Dissertation/Thesis Abstract

Modeling Multiple Valuation Systems in Human Decision Making
by Guo, Rong, Ph.D., Technische Universitaet Berlin (Germany), 2015, 166; 10951612
Abstract (Summary)

Humans may consider various sources of information when making a decision. Traditional reinforcement-learning algorithms mainly focus on learning the expected reward and ignore other psychophysiological factors that may affect human decisions, such as perceptual interference or emotional regulation. This thesis aims to integrate these other factors into the reinforcement-learning models and addresses two questions: (1) How do conflicting salient stimuli influence reward estimation? (2) How are the counterfactual consequences integrated into economic decision-making? I hypothesize that the neurobiological mechanism of error-correction via reinforcement is commonly utilized by multiple valuation systems. In the study of contextual modulation of prediction-error representations, I designed a value-based choice paradigm that dissociated stimulus-based and reward-based expectations. Participants traded off reward against the predictability of the stimulus location. Behavioral results were analyzed on a trial-by-trial basis using two independent Rescorla-Wagner models, which were then combined by a non-linear weighting function. Using model-based fMRI analysis, I found a co-existence of stimulus and reward prediction errors in the ventral striatum, suggesting that this brain region responded to surprising perceptual events as well as unexpected reward delivery or omission. Furthermore, the amygdala activity correlated with the weighting function, suggesting that it might be negotiating between the initial stimulus saliency based choices and the later reward-driven choices. In the study of valuation with counterfactual learning signals, I extended the Q-learning model by incorporating both counterfactual gains and losses into fictive temporal-difference prediction errors. The model was used to investigate the potential influence of counterfactual valuation using both behavioral and fMRI data from a strategic sequential investment paradigm. The results demonstrated that counterfactual learning signals improved the Q-learning model fit, and this improved model predicted BOLD signal changes that correlated with expected value and reward prediction. Expected values derived from the model robustly modulated activity in the ventral medial prefrontal cortex and orbital frontal cortex. Furthermore, the model showed that individuals had different sensitivity to counterfactual gains and losses, which led to distinct neural correlations with fictive prediction error in the ventral striatum. Together these two studies highlighted the neural correlates of multiple prediction errors in the ventral striatum and re-interpreted them in the form of an information prediction error, thus integrating the multiple valuation systems into a single coherent decision-making framework.

Indexing (document details)
Advisor: Opper, Manfred
Commitee: Obermayer, Klaus, Blankenberg, Felix, Gläscher, Jan
School: Technische Universitaet Berlin (Germany)
School Location: Germany
Source: DAI-C 81/1(E), Dissertation Abstracts International
Subjects: Information science, Cognitive psychology
Keywords: Valuation system, Human decision making
Publication Number: 10951612
ISBN: 9781392379905
Copyright © 2020 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy