preprints:
dual credit assignment processes underlie dopamine signals in a complex spacial environment
ta krausz / ae comrie / lm frank / nd daw / jd berke
biorxiv 528738
episodic retrieval for model-based evaluation in sequential decision tasks
cy zhou / d talmi / nd daw / mg mattar
psyarxiv 3sqjh
effort foraging task reveals positive correlation between individual differences in the cost of cognitive and physical effort in humans and relationship to self-reported motivation and affect
la bustamante / t oshinowo / jr lee / e tong / ar burton / as shenhav / jd cohen / nd daw
biorxiv 517394
disentangling abstraction from statistical pattern matching in human and machine learning
s kumar / i dasgupta / r marjieh / nd daw/ jd cohen / tl griffeths
arxiv 2204.01.01437
a passive computational marker for individual differences in non-reinforced learning
t salomon / a itzkovitch / nd daw / t schonberg
biorxiv 484477
a vector reward prediction error model explains dopaminergic heterogeneity
rs lee / b engelhard / ib witten / nd daw
biorxiv 482378
neural evidence for the successor representation in choice evaluation
em russek / i momennejad / mm botvinick / sj gershman / nd daw
biorxiv 458114
a neural mechanism for the opportunity cost of time
sm constantino / j dalrymple / rw gilbert / s varenese / a di rocco / nd daw
biorxiv 173443
2023:
humans decompose tasks by trading off utility and computational cost
cg correa / mk ho / f callaway / nd daw / tl griffiths
plos computational biology in press
model-based control can give rise to devaluation insensitive choice
n garrett / s allan / nd daw
addiction neuroscience 100070
an item response theory analysis of the matrix reasoning item bank
s zorowitz / g chierchia / sj blakemore / nd daw
behav res 2023
2022:
using natural language and program abstractions to instill human inductive biases in machines
s kumar / c correa / i dasgupta / r marjieh / y michael / rd hawkins / nd daw / jd cohen / k narasimhan / tl griffeths
advances in neural information processing systems 2022
uncertainty alters the balance between incremental learning and episodic memory
j nicholas / nd daw / d shohamy
elife 81679
fast rule switching and slow rule updating in a perceptual categorization task
f bouchacourt / s tafazoli / mg mattar / tj buschman / nd daw
elife 82531
neurocomputational mechanisms of confidence in self and others
d bang / r moran / nd daw / sm fleming
nature communications 13:4238
rat anterior cingulate cortex continuously signals decision variables in a patch foraging task
ga kane / mh james / a shenhay / nd daw / jd cohen / g aston-jones
journal of neuroscience 42:5730-5744
2021:
in for a pound, in for a penny: how the opportunity to gain reward influences the competition for memory resources
d talmi / d kavaliauskaite / nd daw
learning and memory 28:445-456
a model for learning based on the joint estimation of stochasticity and volatility
p piray / nd daw
nature communications 12:6587
linear reinforcement learning in planning, grid fields, and cognitive control
p piray / nd daw
nature communications 12:4942
increased and biased deliberation in social anxiety
le hunter / ea meer / cm gillan / m hsu / nd daw
nature human behavior (online ahead of print)
context-sensitive valuation and learning
le hunter / nd daw
current opinion in behavioral sciences 41:122-127
the temporal dynamics of opportunity costs: a normative account of cognitive fatigue and boredom
m agrawal / mg mattar / jd cohen / nd daw
psychological review (online ahead of print)
experience replay is associated with efficient nonlocal learning
y liu / mg mattar / tej behrens / nd daw / rj dolan
science 372(6544):eabf1357
changes in brain and behavior during food-based decision-making following treatment of anorexia nervosa
k foerde / tb walsh / m dalack / nd daw / d shohamy / je steinglass
journal of eating disorders 9:48
deficient goal-directed control in a population characterized by extreme goal pursuit
k foerde / nd daw / t rufin / bt walsh / d shohamy / j steinglass
journal of cognitive neuroscience 33:463-481
meta-learning of compositional task distributions in humans and machines
s kumar / i dasgupta / jd cohen / nd daw / tl griffiths
international conference on learning representations
2020:
restrictive eating across a spectrum from healthy to unhealthy: behavioral and neural mechanisms
k foerde / j schebendach / l davis / dn daw / t walsh / d shohamy / je steinglass
psychological medicine 13:1-10
beyond the average view of dopamine
aj langdon / nd daw
trends in cognitive sciences 24:499-501
a simple model for learning in volatile environments [code]
p piray / nd daw
plos computational biology 16:e1007963
biased belief updating and suboptimal choice in foraging decisions
n garrett / nd daw
nature communications 11:3417
sympathetic and parasympathetic involvement in time constrained sequential foraging
nm dundon / n garrett / v babenko / m cieslak / nd daw / st grafton
cognitive affective and behavioral neuroscience 20:730-745
anxiety, avoidance, and sequential evaluation
s zorowitz / i momennejad / nd daw
computational psychiatry 4:1-17
a computational role for top-down modulation from frontal cortex in infancy
s jaffe-dax / am boldin / nd daw / ll emberson
journal of cognitive neuroscience 32:508-514
2019:
rats exhibit similar biases in foraging and intertemporal choice tasks
ga kane / am bornstein / a shenhav / rc wilson / nd daw / j cohen
elife pii: e48429
comparison of the association between goal-directed planning and self-reported compulsivity vs. obsessive-compulsive disorder diagnosis
cm gillan / e kalanthroff / m evans / hm weingarden / rj jacoby / m gershkovich / i snorrason / r campeas / c cervoni / n crimarco / y sokol / sl garnaat / n mclaughlin / ea phelps / a pinto / cl boisseau / s wilhelm / nd daw / hb simpson
jama psychiatry 77(1):77–85
hierarchical bayesian inference for concurrent model fitting and comparison for group studies
p piray / a dezfouli / t heskes / m frank / nd daw
plos computational biology 15:e1007043
reward prediction error does not explain movement selectivity in dms-projecting dopamine neurons
rs lee / mg mattar / nf parker / ib witten / nd daw
elife pii: e42992
slow escape decisions are driven by trait anxiety
b fung / s qi / s hassibis / nd daw / d mobbs
nature human behavior 3: 702-708
specialized coding of sensory, motor, and cognitive variables in midbrain dopamine neurons
b engelhard / j finkelstein / j cox / w fleming / hj jang / s ornelas / sa koay / s thiberge / nd daw / d tank / i witten
nature 570:509-513
role of human ventromedial prefrontal cortex in learning and recall of enhanced extinction
je dunsmoor / mcw kroes / j li / nd daw / hb simpson / ea phelps
journal of neuroscience 39: 3264-3276
hippocampal contributions to model-based planning and spatial memory
om vikbladh / mr meager / j king / k blackmon / o devinsky / d shohamy / n burgess / nd daw
neuron 102:683-693
a distinct inferential mechanism for delusions in schizophrenia
sc baker / ab konova / nd daw / g horga
brain 142:1797-1812
a retrieved context model of the emotional modulation of memory
d talmi / l lohanas / nd daw
psychological review 126: 455-485
the opportunity cost of time modulates cognitive effort
ar otto / nd daw
neuropsychologia 123:92-105
2018:
offline replay supports planning: fmri evidence from reward revaluation
i momennejad / ar otto / nd daw / ka norman
elife pii: e32548
are we of two minds?
nd daw
nature neuroscience 21(11):1497-99
prioritized memory access explains planning and hippocampal replay
news and views (j widloski / dj foster)
mg mattar / nd daw
nature neuroscience 21(11):1609-17
more than the sum of its parts: a role for the hippocampus in configural reinforcement learning
k duncan / bb doll / nd daw/ d shohamy
neuron 98:645-657
surviving threats: neural circuit and computational implications of a new taxonomy of defensive behavior
j ledoux / nd daw
nature reviews neuroscience 19:269-282
how cognitive and reactive fear circuits optimize escape decisions in humans
s qi / d hassabis / j sun / f gau / nd daw / d mobbs
proceedings of the national academy of sciences 115:3186-91
neural mediators of changes of mind about perceptual decisions
sm fleming / ej van der puten / nd daw
nature neuroscience 21:617-624
a perceptual inference mechanism for hallucinations linked to striatal dopamine
cm cassidy / pd balsam / jj weinstein / rj rosengard / m slifstein / nd daw / a abi-dargham / g horga
current biology 28:503-514
2017:
predictive representations can link model-based reinforcement learning to model-free mechanisms
em russek / i momennejad / mm botvinick / sj gershman / nd daw
plos computational biology 13:e1005768
the successor representation in human reinforcement learning
i momennejad / em russek / j cheong / mm botvinick / nd daw / sj gershman
nature human behavior 1:680-692
what’s past is present: reminders of past choices bias decisions for reward in humans
am bornstein / mw khaw / d shohamy / nd daw
nature communications 8:15958
chronic and acute stress promote overexploitation in serial decision making
jk lenow / sm constantino / nd daw / ea phelps
journal of neuroscience 37:5681-5689
increased locus coeruleus tonic activity causes disengagement from a patch-foraging task
ga kane / em vazey / rc wilson / a shenhav / nd daw / g aston-jones / jd cohen
cogn affect behav neurosci 17(6):1073-83
computational approaches to fmri analysis
j cohen / nd daw / b engelhardt / u hasson / k li / y niv / k norman / j pillow / p ramadge / n turke-brown / t willke
nature neuroscience 20: 304-313
low lifetime stress exposure is associated with reduced stimulus-response memory
ev goldfarb / gs shields / nd daw / gm slavich / ea phelps
learning and memory 24:162-168
independent neural computation of value from other people’s confidence
d campbell-meiklejohn / a simonsen / cd frith / nd daw
journal of neuroscience 37:673-684
suboptimal criterion learning in static and dynamic environments
eh norton / sm fleming / nd daw / ms landy
plos computational biology 13:e1005304
self-evaluation of decision-making: a general bayesian framework for metacognitive computation
sm fleming / nd daw
psychological review 124:91
reinforcement learning and episodic memory in humans and animals: an integrative framework
sj gershman / nd daw
annual review of psychology 68:101-128
2016:
boredom, information-seeking and exploration
a geana / rc wilson / nd daw / jc cohen
proceedings of the 38th annual conference of the cognitive science society:1751-1756
taking psychiatry research online
cm gillan / nd daw
neuron 91(1):19-23
reduced model-based decision-making in schizophrenia
aj culbreth / a westbrook / nd daw / m botvinick / dm barch
journal of abnormal psychology 125:777-87
instructed knowledge shapes feedback-driven aversive learning in striatum and orbitofrontal cortex, but not the amygdala
ly atlas / bb doll / j li / nd daw / ea phelps
elife 5:e15192
reward and choice encoding in terminals of midbrain dopamine neurons depends on striatal target
nf parker / cm cameron / jp taliaferro / j lee / jy choi / tj davidson / nd daw / ib witten
nature neuroscience 19:845–854
motivational context modulates prediction error response in schizophrenia
ja reinen / vx snellenberg / g horga / a abi-dargham / nd daw / d shohamy
schizophrenia bulletin 42(6):1467-75
The expanding role of dopamine
bb doll / nd daw
elife 5:e15963
from creatures of habit to goal-directed learners: tracking the developmental emergence of model-based reinforcement learning
jd decker / ar otto / nd daw / ca hartley
psychological science 27:848-58
characterizing a psychiatric symptom dimension related to deficits in goal-directed control
cm gillan / m kosinski / r whelan / ea phelps / nd daw
elife 5:e11305
variability in dopamine genes dissociates model-based and model-free reinforcement learning
bb doll / kg bath / nd daw / mj frank
journal of neuroscience 36:1211-1222
coherent neural ensembles are rapidly recruited when making a look-reach decision
yt wong / mm fabiszak / y novikov / nd daw / b pesaran
nature neuroscience 19:327-334
dopamine selectively remediates reward learning: a computational approach
me sharp / k foerde / nd daw / d shohamy
brain139:355-64
fronto-striatal organization: defining functional and microstructural substrates of behavioral flexibility
ls morris / p kundu / n dwell / dj mechelmans / p farve / ma irvine / tw robbins / nd daw / et bullmore / na harrison / v voon
cortex 74:118-133
2015:
of goals and habits (commentary for cushman and morris 2015)
nd daw
proceedings of the national academy of sciences 112:13749–50
deciding how to decide: self control and meta-decision making
y-l boureau / p sokol-hessner / nd daw
trends in cognitive sciences 19:700-710
rethinking extinction
je dunsmoor / y niv / nd daw / ea phelps
neuron 88:47-63
valence dependent influence of serotonin depletion on model-based choice strategy
y worbe / s palminteri / g savulich / nd daw / e fernandez-egea / tw robbins / v voon
molecular psychiatry 21:624-9
motivation and value influences in the relative balance of goal-directed and habitual behaviours in obsessive-compulsive disorder
v voon / k back / j enander / y worbe / ls morris / na harrison / tw robbins / c ruck / nd daw
translational psychiatry 5:e670
human representation of visuo-motor uncertainty as mixtures of orthogonal basis distributions
h zhang / nd daw / lt maloney
nature neuroscience 18:1152-1158
reinforcement learning markers of variation in compulsive and depressive traits
cm gillan / m kosinski / ea phelps / nd daw
biological psychiatry 77:39s
learning the opportunity cost of time in a patch foraging task
s constantino / nd daw
cognitive affective and behavioral neuroscience 15:837-53
integrating memories to guide decisions
d shohamy / nd daw
current opinion in behavioral sciences 5:85-90
model-based choices involve prospective neural activity
bb doll / kd duncan / da simon / d shohamy / nd daw
nature neuroscience 18:767-772
model based learning protects against forming habits
cm gillan / ar otto / ea phelps / nd daw
cognitive affective and behavioral neuroscience 15:523-536
depression: a decision theoretic analysis
qj huys / nd daw / p dayan
annual reviews of neuroscience 38:1-23
2014:
episodic memory encoding interferes with reward learning and decreases striatal prediction errors
ge wimmer / ek braun / nd daw / d shohamy
journal of neuroscience 34:14901-12
cognitive control predicts use of model-based reinforcement learning
ar otto / a skatova / s madlon-kay / nd daw
journal of cognitive neuroscience 27:318-333
representation of aversive prediction errors in the human periacqueductal gray
m roy / d shohamy / nd daw / m jepma / ge wimmer / td wager
nature neuroscience 17:1607-12
the algorithmic anatomy of model-based evaluation
nd daw / p dayan
philosophical transactions of the royal society b 369
disorders of compulsivity: a common bias towards learning habits
v voon / k derbyshire / c ruck / m irvine / y worbe / j enander / l schrieber / c gillan / n fineberg / b sahakian / t robbins / n harrison / j wood / nd daw / p dayan / j grant / e bullmore
molecular psychiatry 20:345-352
multiple memory systems as substrates for multiple decision systems
b doll / d shohamy / nd daw
neurobiology of learning and memory 117:4-13
2013:
the importance of mixed selectivity in complex cognitive tasks
m rigotti / o barak / mr warden / x wang / nd daw / ek miller / fusi s
nature: 497:585-90
testing whether humans have an accurate model of their own motor uncertainty in a speeded reaching task
h zhang / nd daw / lt maloney
PLOS Computational Biology 9:e1003080.
working-memory capacity protects model-based learning from stress
ar otto / cm raio / a chiang / ea phelps / nd daw
proceedings of the national academy of sciences 110:20941-6
cortical and hippocampal correlates of deliberation during model-based decisions for rewards in humans
am bornstein / nd daw
plos computational biology 9:e1003387
the irrationality of categorical perception
s fleming / l maloney / nd daw
journal of neuroscience 33:19060-70
dissociable effects of dopamine and serotonin on reversal learning
h den ouden / nd daw / g fernandez / j elshout / m rijpkema / m hoogman / b franke / r cools
neuron 80:1090-100
extraversion differentiates between model-based and model-free strategies in a reinforcement learning task
a skatova / pa chan / nd daw
frontiers in human neuroscience 7:525
the curse of planning: dissecting multiple reinforcement learning systems by taxing the central executive
ar otto / sj gershman / am markman / nd daw
psychological science 24:751-761
value learning through reinforcement: the basics of dopamine and reinforcement learning
nd daw / p tobler
chapter for neuroeconomics: decision making and the brain, 2nd edition
advanced reinforcement learning
nd daw
chapter for neuroeconomics: decision making and the brain, 2nd edition
multiple systems for value learning
nd daw / jp o’doherty
chapter for neuroeconomics: decision making and the brain, 2nd edition
2012:
action selection in multi-effector decision making
s madlon-kay / b pesaran / nd daw
neuroimage 70:66-79
dynamic estimation of task-relevant variance in movement under risk
m landy / j trommershauser / nd daw
journal of neuroscience 32:12702-11
anterior prefrontal cortex contributes to action selection through tracking of recent reward trends
ck kovach / nd daw / d rudrauf / d tranel / jp o’doherty / r adolphs
journal of neuroscience 32:8434-42
serotonin selectively modulates reward value in human decision making
b seymour / nd daw / j roiser / p dayan / rj dolan
journal of neuroscience 32:5833-42
generalization of value in reinforcement learning by humans
ge wimmer / nd daw / d shohamy
european journal of neuroscience 35:1092-1104
dissociating hippocampal and striatal contributions to sequential prediction learning
am bornstein / nd daw
european journal of neuroscience 35:1011-1023
the ubiquity of model-based reinforcement learning
bb doll / da simon / nd daw
current opinion in neurobiology 22:1075-81
surprise! neural correlates of pearce-hall and rescorla-wagner coexist within the brain
mr roesch / gr esber / j li / nd daw /g schoenbaum
european journal of neuroscience 35:1190-1200
perception action and utility: the tangled skein
sj gershman / nd daw
chapter for principles of brain dynamics
model-based reinforcement learning as cognitive search: neurocomputational theories
nd daw
chapter for cognitive search: evolution algorithms and the brain
2011:
environmental statistics and the trade-off between model-based and TD learning in humans
da simon / nd daw
nips 24
grid cells, place cells, and geodesic generalization for spatial reinforcement learning
n gustafson / nd daw
plos computational biology 7:e1002235
differential roles of human striatum and amygdala in associative learning
j li / d schiller / g schoenbaum / ea phelps / nd daw
nature neuroscience 14:1250-1252
signals in human striatum are appropriate for policy update rather than value prediction
j li / nd daw
journal of neuroscience 31:5504-11
neural correlates of forward planning in a spatial decision task in humans
da simon / nd daw
journal of neuroscience 31:5526-39
model-based influences on humans’ choices and striatal prediction errors
nd daw / sj gershman / b seymour / p dayan / rj dolan
neuron 69:1204-1215
serotonin and dopamine: unifying affective, activational, and decision functions
r cools / k nakamura / nd daw
neuropsychopharmacology 36:98-113
dual-system learning models and drugs of abuse
da simon / nd daw
chapter in computational neuroscience of addiction
multiplicity of control in the basal ganglia: computational roles of striatal subregions
ab bornstein / nd daw
current opinion in neurobiology, 21:374-380
trial by trial data analysis using computational models
nd daw
chapter in “affect, learning, and decision making: attention and performance xxiii”
2010:
tonic dopamine modulates exploitation of reward learning
ja beeler / nd daw / crm frazier / x zhuang
frontiers in behavioral neuroscience 4:170
states vs rewards: dissociable neural prediction error signals underlying mdoel-based and model-free reinforcement learning
j glascher / nd daw / p dayan / jp o’doherty
neuron 66: 585-595
selective impairment of prediction error signaling in human dorsolateral but not ventral striatum in parkinson’s disease patients: evidence from a model-based fmri study
t schonberg / jp o’doherty / d joel / r inzelberg / y segev / nd daw
neuroimage 49:772-781
a closer look at choice
sm constantino / nd daw
nature neuroscience (news and views for krajbich et al. 2010) 13:1153-1154
2009:
the pigeon as particle filter
nd daw / ac courville
nips 20
human reinforcement learning subdivides structured action spaces by learning effector-specific values
sj gershman / b pesaran / nd daw
journal of neuroscience 29:13524-31
reward learning and the novelty seeking personality: a between and within-subjects study of the effects of dopamine agonists on young parkinson’s patients
n bodi / s keri / h nagi / a moustafa / c myers / nd daw / g dibo / a takats / d bereczi / ma gluck
brain 132:2385-2395
a dual role for prediction error in associative learning
he den ouden / kj friston / nd daw / ar mcintosh / ke stephan
cerebral cortex 19:1175-1185
2008:
decision theory, reinforcement learning, and the brain
p dayan / nd daw
cognitive affective and behavioral neuroscience 8:429-453
striatal activity underlies novelty-based choice in humans
bc wittmann / nd daw / b seymour / rj dolan
neuron 58:967-973
the cognitive neuroscience of motivation and learning
nd daw / ds shohamy
social cognition 26:593-620
multiple forms of value learning and the function of dopamine
bw balleine / nd daw / jp o’doherty
chapter in “neuroeconomics: decision making and the brain,”
semi-rational models: the case of trial order
nd daw / ac courville / p dayan
chapter for “the probabilistic mind,” chater & oaksford, eds., oxford
2007:
reinforcement learning signals in the human striatum distinguish learners from non-learners during reward-based decision making
supplemental material
t schonberg / nd daw / d joel / jp o’doherty
journal of neuroscience 27:12860-12867
differential encoding of losses and gains in the human striatum
b seymour / nd daw / p dayan / t singer / r dolan
journal of neuroscience: 27:4826-4831
tonic dopamine: opportunity costs and the control of response vigor
y niv / nd daw / d joel / p dayan
psychopharmacology 191:507-520
dopamine: at the intersection of reward and action
nd daw
nature neuroscience (news and views for roesch et al 2007) 10:1505-1507
2006:
cortical substrates for exploratory decisions in humans
supplemental material / news and views (d lee)
nd daw / jp o’doherty / b seymour / p dayan / rj dolan
nature 441:876-879
the misbehavior of value and the discipline of the will
p dayan / y niv / b seymour / nd daw
neural networks 19:1153-1160
bayesian theories of conditioning in a changing world
ac courville / nd daw / ds touretzky
trends in cognitive sciences 10:294-300
representation and timing in theories of the dopamine system
nd daw / ac courville / ds touretzky
neural computation 18:1637-1677
choice values
y niv / nd daw / p dayan
nature neuroscience (news and views for morris et al 2006) 9:987-988
the computational neurobiology of learning and reward
nd daw / k doya
current opinion in neurobiology 16:199-204
actions, values, policies and the basal ganglia
nd daw / y niv / p dayan
chapter for “recent breakthroughts in basal ganglia research”
2005:
uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control
nd daw / y niv / p dayan
nature neuroscience 8:1704-171
how fast to work: response vigor, motivation and tonic dopamine
y niv / nd daw / p dayan
nips 18
2004:
similarity and discrimination in classical conditioning: a latent variable account
ac courville / nd daw / ds touretzky
nips 17:313-320
matchmaking
nd daw / p dayan
science (perspective for sugrue et al 2004) 304:1753-1754
theoretical and computational neuroscience: learning, action, inference and neuromodulation
p dayan / nd daw / y niv
chapter for “new encyclopedia of neuroscience”
2003:
a computational substrate for incentive salience
sm mcclure / nd daw / pr montague
trends in neurosciences 26:423-428
model uncertainty in classical conditioning
ac courville / nd daw / gj gordon / ds touretzky
nips 16:977-984
reinforcement learning models of the dopamine system and their behavioral implications
nd daw
phd thesis, cmu dept of computer science
2002:
long-term reward prediction in TD models of the dopamine system
nd daw / ds touretzky
neural computation 14:2567-2583
opponent interactions between serotonin and dopamine
nd daw / s kakade / p dayan
neural networks 15:603-616
local analysis of behavior in the adjusting delay task for assessing choice of delayed reinforcement
r cardinal / nd daw / tw robbins / bj everitt
neural networks 15:617-634
timing and partial observability in the dopamine system
nd daw / ac courville / ds touretzky
nips 15:99-106
dopamine and inference about timing
nd daw / ac courville / ds touretzky
second icdl 271-276
combining configural and td learning on a robot
ds touretzky / nd daw / ej tira-thompson
second icdl 271-276
2001:
operant behavior suggests attentional gating of dopamine system inputs
nd daw / ds touretzky
neurocomputing 38-40:1161-1167
2000:
behavioral results suggest an average reward TD model of dopamine function
nd daw / ds touretzky
neurocomputing 32:679-684