EVAL 6970: Meta Analysis
Description
This course in the interdisciplinary Ph.D. in evaluation program at °ÄÃÅÁùºÏ²Ê¹ÙÍøÖ±²¥ is an advanced graduate seminar designed to provide students with the knowledge, skills and abilities necessary to conduct basic research reviews, research syntheses and meta-analyses. Topics covered include, but are not limited to:
- The increasing use of meta-analysis in formulating and enacting evidence-based policies and practices.
- The role of meta-analysis in theory development.
- Principles and procedures for planning and executing research reviews and meta-analyses
- Identifying and retrieving literature.
- Coding studies.
- Computing effect sizes (e.g., based on means, binary data, and correlations) and their corresponding confidence intervals for meta-analysis.
- Converting among effect sizes.
- Factors that affect precision (e.g., variance, standard error, confidence intervals).
- Fixed-effect and random-effects models for meta-analysis.
- Identifying and quantifying heterogeneity.
- Prediction intervals.
- Subgroup analysis.
- Meta-regression.
- Meta-analysis with complex data structures.
- Power analysis for meta-analysis.
- Publication bias.
- Psychometric meta-analysis.
Students should have at least a fundamental knowledge of applied statistics and experimental and quasi-experimental design to succeed in the course and will be required to plan and execute a basic meta-analysis. EMR 6550: Experimental and Quasi-Experimental Designs, is a recommended, but not required, prerequisite.
Syllabus
Instructor
Required textbooks
- Bornenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to meta-analysis. West Sussex, UK: Wiley.
- Cooper, H., Hedges, L. V., & Valentine, J. C. (Eds.). (2009). The handbook of research synthesis and meta-analysis (2nded.). New York, NY: Russell Sage Foundation.
- Comprehensive Meta-Analysis 2.0
Required readings
These readings are for instructional purposes only.
- Anderson, C. A., et al. (2010). Violent video game effects on aggression, empathy, and prosocial behavior in eastern and western countries: A meta-analytic review. Psychological Bulletin, 136(2), 151-173.
- Beretavs, S. N. (2010). Meta-analysis. In G. R. Hancock, G. R. R. O. & Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 255-264). New York, NY: Routledge.
- Bornmann, L., diger Mutz, R., & Daniel, H-D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS ONE, 5(2), e14331.
- Cook, T. D., & Leviton, L. C. (1980). Reviewing the literature: A comparison of traditional methods with meta-analysis. Journal of Personality, 48(4), 449-472.
- DuBois, D. L., Holloway, B. E., Valentine, J. C., & Cooper, H. (2002). Effectiveness of mentoring programs for youth: A meta-analytic review. American Journal of Community Psychology, 30(2), 157-197.
- Peterson, J. L., & Shibley Hyde, J. (2010). A meta-analytic review of research on gender differences in sexuality, 1993–2007. Psychological Bulletin, 136(1), 21-38.
- Schwandt, T. A. (2000). Meta-analysis and everyday life: The good, the bad, and the ugly. American Journal of Evaluation, 21(2), 213-219.
- Wilson, S. J., & Lipsey, M. W. (2000). Wilderness challenge programs for delinquent youth: a meta-analysis of outcome evaluations. Evaluation and Program Planning, 23(1), 1-12.
Data sets and supplementary materials
- Effect Size Substantive Interpretation Guidelines
- Effect Size Calculator for Standardized Mean Difference Type Effects (Wilson, 2001)
- Meta-Analysis with Means, Binary Data, and Correlations
- (Wilson, n.d.)
- American Psychological Association Meta-Analysis Reporting Standards
- Sample Code Book
- Data Sets 1-6
- Data Sets for Heterogeneity Analysis
- Data Set for Subgroup Analysis
Homework and projects
Lecture notes
- Lecture 1PPTX
- Lecture2 PPTX
- Lecture 3 PPTX PDF
- Lecture 4 PPTX PDF
- Lecture 5 PPTX PDF
- Lecture 6 PPTX PDF
- Lecture 7 PPTX PDF
- Lecture 8 PPTX PDF
- Lecture 9 PPTX PDF
- Lecture 10 PPTX PDF