Collaborative efforts like modern scientific research depend on methods to evaluate and absorb participants’ contributions, and at the research frontier this evaluative step is often accomplished through the peer review of grants and manuscripts. With billions of dollars and space in prestigious journals hinging on the decisions of reviewers, the review system has attracted consistent scrutiny. Many of the thousands of studies scrutinizing peer review focus on the reliability, validity, and fairness of the reviewers’ decisions. Largely absent in this debate about peer review’s internal practices are the consequences of these practices for the character and diffusion of published knowledge. This dissertation shifts the focus to the consequences of peer review practices through four case studies. The first case investigates the negotiation of revisions authors of quantitative sociological manuscripts undertake during peer review and reveals that substantial changes concern primarily manuscripts’ theoretical framing, while the data analyses remain relatively stable. The case argues that the greater relative value placed on data and analysis over frames incentivizes investment into the former over the latter. The second case interrogates the common practice of using post-publication citations to evaluate the validity of review decisions. Analysis of the reviews of manuscripts submitted to the American Sociological Review from 1977 to 1981 and the manuscripts’ subsequent citations reveals no relationship. However, reviewers’ comments show that reviewers focused on the soundness of the manuscripts’ arguments, not their potential impact. The case shows that a review process that results in publications of variable impact is not necessarily a failing of peer review, but rather a consequence of reviewers and citers draw on different dimensions of value. The third case study examines the consequences for quantitative sociology of the common bias for positive findings in peer review. Using hundreds of studies that use the General Social Survey, the published statistical relationships are perturbed by slight changes to the model specifications. Results show that at the time of publication, results are relatively robust to this perturbation. Additionally, the published relationships are estimated using waves of the Survey that appeared after publication. Results indicate that published findings are weakened much more by social change. The last case focuses on the consequences of scientific peer review judgments outside of the sphere of science. By measuring rates at which millions of scientific journals are used as sources in Wikipedia, the largest online encyclopedia, I show that Wikipedia editors preferentially use high impact and the more accessible (open access) journals. The case shows that increased accessibility of the scientific literature improves its diffusion to the lay public and that a status ordering that review practices establish in one sphere, science, may be exported wholesale to a disparate context, Wikipedia.
|Advisor:||Evans, James A.|
|Commitee:||Abbott, Andrew, Knorr, Karin|
|School:||The University of Chicago|
|School Location:||United States -- Illinois|
|Source:||DAI-A 77/10(E), Dissertation Abstracts International|
|Subjects:||Sociology, Public policy|
|Keywords:||Citations, Peer review, Reproducibility, Wikipedia|
Copyright in each Dissertation and Thesis is retained by the author. All Rights Reserved
The supplemental file or files you are about to download were provided to ProQuest by the author as part of a
dissertation or thesis. The supplemental files are provided "AS IS" without warranty. ProQuest is not responsible for the
content, format or impact on the supplemental file(s) on our system. in some cases, the file type may be unknown or
may be a .exe file. We recommend caution as you open such files.
Copyright of the original materials contained in the supplemental file is retained by the author and your access to the
supplemental files is subject to the ProQuest Terms and Conditions of use.
Depending on the size of the file(s) you are downloading, the system may take some time to download them. Please be