Rob Olsen
Background
Rob Olsen is Research Professor in the George Washington Institute of Public Policy (GWIPP) and a part-time faculty member in the Trachtenberg School of Public Policy & Public Administration. In GWIPP, he is conducting research on the external validity of randomized trials of career pathways, job training programs. In the Trachtenberg School, he teaches courses on statistical and econometric methods.
Prof. Olsen’s research has focused on rigorous impact evaluations of social programs, especially randomized trials. He has played leadership roles on large-scale randomized trials of educational programs, and he has contributed to the science and practice of impact evaluations through scholarly publications on evaluation methods, advisory panels for evaluation standards, and evaluation technical assistance to research teams conducting impact evaluations. Prior to joining GWIPP, Prof. Olsen held research positions at Mathematica, the Urban Institute, Abt Associates, and Westat. He received his Ph.D. in labor economics from Cornell University in 1999.
Over the past 14 years, his research has focused on methods for improving the external validity or generalizability of findings from randomized trials. This research has assessed the generalizability of randomized trials, tested the performance of statistical methods for improving their generalizability, and developed new methods to improve their generalizability.
Current Research
Testing Innovative Methods to Project Impacts of Career Pathways Programs for Individual Sites. Randomized trials of social programs often involve multiple sites spread across the country. Findings from these studies can be used to inform local decisions about whether to implement the program. The question addressed by this research project is how accurately the impact of social programs can be projected locally, for individual sites, using data from national studies—specifically, multi-site randomized trials that are national in scope. To address this question, we use data from two multi-site randomized trials of career pathways programs that were conducted for the Administration of Children and Families. To project the impact in individual sites, we use Fixed Intercept Random Coefficients models to explain variation in impacts that is attributable to individual and site characteristics.
Selected Publications
Using a Multi-site RCT to Predict Impacts for a Single Site: Do Better Methods and Data Yield More Accurate Predictions? With Larry L. Orr, Stephen H. Bell, Elena Badilla-Goicoechea, Atsushi Miyaoka, and Elizabeth A. Stuart.Journal of Research on Educational Effectiveness.
Enhancing the Generalizability of Impact Studies in Education (NCEE 2022-003). With Elizabeth Tipton. U.S. Department of Education, Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance.
Using the Results from Rigorous Multisite Evaluations to Inform Local Policy Decisions. With Larry L. Orr, Stephen H. Bell, Ian Schmid, Azim Shivji, and Elizabeth A. Stuart. Journal of Public Policy Analysis and Management.
A Review of Statistical Methods for Generalizing from Evaluations of Educational Interventions. With Elizabeth Tipton. Educational Researcher.
Using Preferred Applicant Random Assignment (PARA) to Reduce Randomization Bias in Randomized Trials of Discretionary Programs. With Stephen H. Bell and Austin Nichols. Journal of Policy Analysis and Management.
Characteristics of School Districts that Participate in Rigorous National Educational Evaluations. With Elizabeth A. Stuart, Stephen H. Bell, Cyrus Ebnesajjad, and Larry L. Orr. Journal of Research on Educational Effectiveness.
On the “Where” of Social Experiments: Selecting More Representative Samples to Inform Policy. With Larry L. Orr. New Directions for Evaluation.
Estimates of Bias When Impact Evaluations Select Sites Purposively. With Stephen H. Bell, Larry L. Orr, and Elizabeth A. Stuart. Educational Evaluation and Policy Analysis.
External Validity in Policy Evaluations that Choose Sites Purposively. With Larry L. Orr, Stephen H. Bell, and Elizabeth A. Stuart. Journal of Policy Analysis and Management.