Dopamine neurons are thought to play a key role in reinforcement learning, in updating the values of options and actions to guide value-based decision making. These neurons are thought to encode a reward prediction error (RPE) signal, the difference between experienced and predicted reward. Neuroimaging studies typically identify neural activity in dopamine projection areas correlated with the predictions of highly parameterized RPE models. Identified brain areas might encode RPEs or alternatively simply have activity correlated with RPE model predictions. To formally test the dopaminergic RPE hypothesis, we used an axiomatic approach rooted in economic theory to test the entire class of RPE models on neural data. Neural activity in the nucleus accumbens, a subregion of the striatum rich in dopamine receptors, satisfies the necessary and sufficient conditions for the entire class of RPE models. Neural activity in the caudate, putamen, medial prefrontal cortex, amygdala, and posterior cingulate cortex also satisfies these conditions and can encode RPEs. However, anterior insula activity falsifies the axiomatic model and therefore no RPE model can account for this activity. Further analysis suggests that the anterior insula might encode something related to outcome salience.
Parkinson’s disease is characterized by loss of dopamine neurons in the substantia nigra. However, it is unknown whether Parkinson’s disease and the dopaminergic drugs taken to treat the disorder affect reinforcement learning in the way predicted by the hypothesis that dopamine neurons encode RPEs used for reinforcement learning. To test a key prediction of the theory, we fit choice behavior from a dynamic foraging task with reinforcement learning models and show that treatment with dopaminergic drugs alters choice behavior in a manner consistent with the theory. More specifically, we found that dopaminergic drugs selectively modulate learning from positive outcomes, but observed no effect on learning from negative outcomes. These results show that neural activity measured in dopamine projection areas can encode RPEs and that dopaminergic drugs affect reinforcement learning in the manner predicted.
|Advisor:||Glimcher, Paul W.|
|Commitee:||Daw, Nathaniel, Delgado, Mauricio, Kiorpes, Lynne, Simoncelli, Eero|
|School:||New York University|
|Department:||Center for Neural Science|
|School Location:||United States -- New York|
|Source:||DAI-B 72/01, Dissertation Abstracts International|
|Keywords:||Dopamine, Neuroeconomics, Parkinson's disease, Reinforcement learning, Reward|
Copyright in each Dissertation and Thesis is retained by the author. All Rights Reserved
dissertation or thesis. The supplemental files are provided "AS IS" without warranty. ProQuest is not responsible for the
content, format or impact on the supplemental file(s) on our system. in some cases, the file type may be unknown or
may be a .exe file. We recommend caution as you open such files.
supplemental files is subject to the ProQuest Terms and Conditions of use.