Dissertation/Thesis Abstract

The Effect of Time Period, Field, and Coding Context on Rigor, Interrater Agreement, and Interrater Reliability in Meta-Analysis
by London, Jennifer Elise, Ph.D., North Carolina State University, 2016, 88; 10583286
Abstract (Summary)

Meta-analyses are used in all fields of research, and they are typically assumed to be the same regardless of field or time. This paper explores changes in the meta-analytic process over time as well as across the fields of Computer Science, Education, Medicine, and Psychology on a representative sample of meta-analyses. Facets of the process of meta-analysis were also examined as well as how they impact meta-analytic Rigor, interrater agreement (IRA) and interrater reliability (IRR). Data visualization, ANOVAs, multiple regression, and path analysis were used to connect variables and make sense of relationships. Surprisingly, IRA and Rigor were not related, and IRR had a strong negative correlation with Rigor. Furthermore, the reporting rates of IRA and IRR were very low across all fields.

In terms of procedure, meta-analyses in the field of Medicine were the most well-developed and offer several ideas for improving meta-analyses in the social sciences. Suggestions for improving IRA and IRR process and reporting are included. In examining many meta-analyses from different fields, some surprising discoveries were made. Most notably, studies labeled as meta-analyses in Computer Science frequently failed to include the analyses that typify meta-analysis; instead, they would be better described as vote-counting studies or qualitative reviews.

Indexing (document details)
Advisor: Wilson, Mark A.
School: North Carolina State University
School Location: United States -- North Carolina
Source: DAI-B 78/08(E), Dissertation Abstracts International
Subjects: Psychobiology, Behavioral Sciences, Quantitative psychology
Keywords: Interrater agreement, Interrater reliability
Publication Number: 10583286
ISBN: 978-1-369-61981-2
Copyright © 2020 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy