Dissertation/Thesis Abstract

The neural basis of human reinforcement learning
by Rutledge, Robb B., Ph.D., New York University, 2010, 259; 3428046
Abstract (Summary)

Dopamine neurons are thought to play a key role in reinforcement learning, in updating the values of options and actions to guide value-based decision making. These neurons are thought to encode a reward prediction error (RPE) signal, the difference between experienced and predicted reward. Neuroimaging studies typically identify neural activity in dopamine projection areas correlated with the predictions of highly parameterized RPE models. Identified brain areas might encode RPEs or alternatively simply have activity correlated with RPE model predictions. To formally test the dopaminergic RPE hypothesis, we used an axiomatic approach rooted in economic theory to test the entire class of RPE models on neural data. Neural activity in the nucleus accumbens, a subregion of the striatum rich in dopamine receptors, satisfies the necessary and sufficient conditions for the entire class of RPE models. Neural activity in the caudate, putamen, medial prefrontal cortex, amygdala, and posterior cingulate cortex also satisfies these conditions and can encode RPEs. However, anterior insula activity falsifies the axiomatic model and therefore no RPE model can account for this activity. Further analysis suggests that the anterior insula might encode something related to outcome salience.

Parkinson’s disease is characterized by loss of dopamine neurons in the substantia nigra. However, it is unknown whether Parkinson’s disease and the dopaminergic drugs taken to treat the disorder affect reinforcement learning in the way predicted by the hypothesis that dopamine neurons encode RPEs used for reinforcement learning. To test a key prediction of the theory, we fit choice behavior from a dynamic foraging task with reinforcement learning models and show that treatment with dopaminergic drugs alters choice behavior in a manner consistent with the theory. More specifically, we found that dopaminergic drugs selectively modulate learning from positive outcomes, but observed no effect on learning from negative outcomes. These results show that neural activity measured in dopamine projection areas can encode RPEs and that dopaminergic drugs affect reinforcement learning in the manner predicted.

Indexing (document details)
Advisor: Glimcher, Paul W.
Commitee: Daw, Nathaniel, Delgado, Mauricio, Kiorpes, Lynne, Simoncelli, Eero
School: New York University
Department: Center for Neural Science
School Location: United States -- New York
Source: DAI-B 72/01, Dissertation Abstracts International
Subjects: Neurobiology
Keywords: Dopamine, Neuroeconomics, Parkinson's disease, Reinforcement learning, Reward
Publication Number: 3428046
ISBN: 978-1-124-33286-4
Copyright © 2020 ProQuest LLC. All rights reserved. Terms and Conditions Privacy Policy Cookie Policy