Humans may consider various sources of information when making a decision. Traditional reinforcement-learning algorithms mainly focus on learning the expected reward and ignore other psychophysiological factors that may affect human decisions, such as perceptual interference or emotional regulation. This thesis aims to integrate these other factors into the reinforcement-learning models and addresses two questions: (1) How do conflicting salient stimuli influence reward estimation? (2) How are the counterfactual consequences integrated into economic decision-making? I hypothesize that the neurobiological mechanism of error-correction via reinforcement is commonly utilized by multiple valuation systems. In the study of contextual modulation of prediction-error representations, I designed a value-based choice paradigm that dissociated stimulus-based and reward-based expectations. Participants traded off reward against the predictability of the stimulus location. Behavioral results were analyzed on a trial-by-trial basis using two independent Rescorla-Wagner models, which were then combined by a non-linear weighting function. Using model-based fMRI analysis, I found a co-existence of stimulus and reward prediction errors in the ventral striatum, suggesting that this brain region responded to surprising perceptual events as well as unexpected reward delivery or omission. Furthermore, the amygdala activity correlated with the weighting function, suggesting that it might be negotiating between the initial stimulus saliency based choices and the later reward-driven choices. In the study of valuation with counterfactual learning signals, I extended the Q-learning model by incorporating both counterfactual gains and losses into fictive temporal-difference prediction errors. The model was used to investigate the potential influence of counterfactual valuation using both behavioral and fMRI data from a strategic sequential investment paradigm. The results demonstrated that counterfactual learning signals improved the Q-learning model fit, and this improved model predicted BOLD signal changes that correlated with expected value and reward prediction. Expected values derived from the model robustly modulated activity in the ventral medial prefrontal cortex and orbital frontal cortex. Furthermore, the model showed that individuals had different sensitivity to counterfactual gains and losses, which led to distinct neural correlations with fictive prediction error in the ventral striatum. Together these two studies highlighted the neural correlates of multiple prediction errors in the ventral striatum and re-interpreted them in the form of an information prediction error, thus integrating the multiple valuation systems into a single coherent decision-making framework.
|Commitee:||Obermayer, Klaus, Blankenberg, Felix, Gläscher, Jan|
|School:||Technische Universitaet Berlin (Germany)|
|Source:||DAI-C 81/1(E), Dissertation Abstracts International|
|Subjects:||Information science, Cognitive psychology|
|Keywords:||Valuation system, Human decision making|
Copyright in each Dissertation and Thesis is retained by the author. All Rights Reserved
The supplemental file or files you are about to download were provided to ProQuest by the author as part of a
dissertation or thesis. The supplemental files are provided "AS IS" without warranty. ProQuest is not responsible for the
content, format or impact on the supplemental file(s) on our system. in some cases, the file type may be unknown or
may be a .exe file. We recommend caution as you open such files.
Copyright of the original materials contained in the supplemental file is retained by the author and your access to the
supplemental files is subject to the ProQuest Terms and Conditions of use.
Depending on the size of the file(s) you are downloading, the system may take some time to download them. Please be