CiteULike: nelmor Doya

Low-Serotonin Levels Increase Delayed Reward Discounting in Humans

Nicolas Schweighofer — 2008-04-24T13:04:57-00:00

J. Neurosci., Vol. 28, No. 17. (23 April 2008), pp. 4528-4532.

Previous animal experiments have shown that serotonin is involved in the control of impulsive choice, as characterized by high preference for small immediate rewards over larger delayed rewards. Previous human studies under serotonin manipulation, however, have been either inconclusive on the effect on impulsivity or have shown an effect in the speed of action-reward learning or the optimality of action choice. Here, we manipulated central serotonergic levels of healthy volunteers by dietary tryptophan depletion and loading. Subjects performed a "dynamic" delayed reward choice task that required a continuous update of the reward value estimates to maximize total gain. By using a computational model of delayed reward choice learning, we estimated the parameters governing the subjects' reward choices in low-, normal, and high-serotonin conditions. We found an increase of proportion in small reward choices, together with an increase in the rate of discounting of delayed rewards in the low-serotonin condition compared with the control and high-serotonin conditions. There were no significant differences between conditions in the speed of learning of the estimated delayed reward values or in the variability of reward choice. Therefore, in line with previous animal experiments, our results show that low-serotonin levels steepen delayed reward discounting in humans. The combined results of our previous and current studies suggest that serotonin may adjust the rate of delayed reward discounting via the modulation of specific loops in parallel corticobasal ganglia circuits. 10.1523/JNEUROSCI.4982-07.2008

Metalearning and neuromodulation

Kenji Doya — 2007-12-06T08:50:56-00:00

Neural Netw., Vol. 15, No. 4. (June 2002), pp. 495-506.

Understanding Neural Coding through the Model-Based Analysis of Decision Making

Greg Corrado — 2007-08-03T09:38:54-00:00

J. Neurosci., Vol. 27, No. 31. (1 August 2007), pp. 8178-8180.

The study of decision making poses new methodological challenges for systems neuroscience. Whereas our traditional approach linked neural activity to external variables that the experimenter directly observed and manipulated, many of the key elements that contribute to decisions are internal to the decider. Variables such as subjective value or subjective probability may be influenced by experimental conditions and manipulations but can neither be directly measured nor precisely controlled. Pioneering work on the neural basis of decision circumvented this difficulty by studying behavior in static conditions, in which knowledge of the average state of these quantities was sufficient. More recently, a new wave of studies has confronted the conundrum of internal decision variables more directly by leveraging quantitative behavioral models. When these behavioral models are successful in predicting a subject's choice, the model's internal variables may serve as proxies for the unobservable decision variables that actually drive behavior. This new methodology has allowed researchers to localize neural subsystems that encode hidden decision variables related to free choice and to study these variables under dynamic conditions. 10.1523/JNEUROSCI.1590-07.2007

Representation of action-specific reward values in the striatum.

K Samejima — 2005-11-30T17:36:20-00:00

Science, Vol. 310, No. 5752. (25 November 2005), pp. 1337-1340.

The estimation of the reward an action will yield is critical in decision-making. To elucidate the role of the basal ganglia in this process, we recorded striatal neurons of monkeys who chose between left and right handle turns, based on the estimated reward probabilities of the actions. During a delay period before the choices, the activity of more than one-third of striatal projection neurons was selective to the values of one of the two actions. Fewer neurons were tuned to relative values or action choice. These results suggest representation of action values in the striatum, which can guide action selection in the basal ganglia circuit.

Reward and Decision Making in Corticobasal Ganglia Networks (Annals of the New York Academy of Science)

Kenji Doya — 2007-07-23T14:24:51-00:00

(01 June 2007)

The neural bases of decision-making processes are currently generating considerable experimental and theoretical interest. Several recent developments in neuroscience, psychology, and economics have helped to focus thinking on this issue: the computational description of cortico-striatal networks in terms of reinforcement learning models, recognition that midbrain dopaminergic activity could reflect an error correction learning signal, improvements in imaging technology, the recognition that multiple controllers of actions and of values contribute to the development of adaptive behavior, and the recognition that theories of value derived from economics can provide a principled means of incorporating intangible factors such as risk, uncertainty, and temporal discounting into computational models of neural system.The contributions to this volume are forward-looking assessments of the current and future issues likely to be faced by researchers in this area. Four current issues are specifically addressed:- the degree to which distinct behavioral and psychological capacities map onto discrete neural systems; - the relationship between midbrain dopamine activity and reward in terms of phasic and tonic activity, prefrontal and striatal targets, and whether and what limits exist in terms of learning; - whether there are multiple prediction error signals distinguishing between, for example, reward and punishment, instrumental versus Pavlovian conditioning, goals and habits, learning with different discount factors, real versus fictive learning; and- the relationship between economics and neuroscience, which has recently emerged in a marriage to form the new field of neuroeconomics. Will this marriage be productive for the long term or swiftly head for divorce?NOTE: Annals volumes are available for sale as individual books or as a journal. For information on institutional journal subscriptions, please visit www.blackwellpublishing.com/nyas. ACADEMY MEMBERS: Please contact the New York Academy of Sciences directly to place your order (www.nyas.org). Members of the New York Academy of Science receive full-text access to the Annals online and discounts on print volumes. Please visit www.nyas.org/membership/main.asp for more information about becoming a member.

The computational neurobiology of learning and reward

Nathaniel Daw — 2007-04-19T16:11:56-00:00

Current Opinion in Neurobiology, Vol. 16, No. 2. (April 2006), pp. 199-204.

Following the suggestion that midbrain dopaminergic neurons encode a signal, known as a `reward prediction error', used by artificial intelligence algorithms for learning to choose advantageous actions, the study of the neural substrates for reward-based learning has been strongly influenced by computational theories. In recent work, such theories have been increasingly integrated into experimental design and analysis. Such hybrid approaches have offered detailed new insights into the function of a number of brain areas, especially the cortex and basal ganglia. In part this is because these approaches enable the study of neural correlates of subjective factors (such as a participant's beliefs about the reward to be received for performing some action) that the computational theories purport to quantify.