Dissertation/Thesis Abstract

Models for understanding student thinking using data from complex computerized science tasks
by LaMar, Michelle Marie, Ph.D., University of California, Berkeley, 2014, 93; 3686374
Abstract (Summary)

The Next Generation Science Standards (NGSS Lead States, 2013) define performance targets which will require assessment tasks that can integrate discipline knowledge and cross-cutting ideas with the practices of science. Complex computerized tasks will likely play a large role in assessing these standards, but many questions remain about how best to make use of such tasks within a psychometric framework (National Research Council, 2014). This dissertation explores the use of a more extensive cognitive modeling approach, driven by the extra information contained in action data collected while students interact with complex computerized tasks. Three separate papers are included. In Chapter 2, a mixture IRT model is presented that simultaneously classifies student understanding of a task while measuring student ability within their class. The model is based on differentially scoring the subtask action data from a complex performance. Simulation studies show that both class membership and class-specific ability can be reasonably estimated given sufficient numbers of items and response alternatives. The model is then applied to empirical data from a food-web task, providing some evidence of feasibility and validity. Chapter 3 explores the potential of using a more complex cognitive model for assessment purposes. Borrowing from the cognitive science domain, student decisions within a strategic task are modeled with a Markov decision process. Psychometric properties of the model are explored and simulation studies report on parameter recovery within the context of a simple strategy game. In Chapter 4 the Markov decision process (MDP) measurement model is then applied to an educational game to explore the practical benefits and difficulties of using such a model with real world data. Estimates from the MDP model are found to correlate more strongly with posttest results than a partial-credit IRT model based on outcome data alone.

Indexing (document details)
Advisor: Rabe-Hesketh, Sophia
Commitee: Griffiths, Tom, Wilson, Mark
School: University of California, Berkeley
Department: Education
School Location: United States -- California
Source: DAI-B 76/08(E), Dissertation Abstracts International
Subjects: Educational tests & measurements, Statistics, Quantitative psychology
Keywords: Item response theory, Markov decision processes, Mixture models, Technology-based assessment
Publication Number: 3686374
ISBN: 978-1-321-63124-1
Copyright © 2020 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy