4
$\begingroup$

I just recently learned (indirectly from this source) that long-term potentiation/depression only takes place when you have large/small amounts of dopamine present at the relevant synapse. However, my understanding is that early in the process of learning behavior, dopamine isn't released until after a reward is received. If I'm a monkey who has to look one way or the other to get some juice, by the time I drink the juice and get the resulting dopamine, the neurons I used to move my eye are presumably not firing any more, right? So then how does this work? Is it that:

  1. There's a slow bootstrapping process. Something like: first the monkey has to learn that the taste of juice leads to the reward of sugar. Then the monkey has to learn that the sight of juice correlates to the taste, and extend the chain one step further, etc., etc., such that by the time you actually have the monkey in the lab there's a very small inferential step needed to plug into a larger reward chain.
  2. The neurons don't actually stop firing. There is some sort of system in play that keeps the neurons firing afterwards, waiting for fluctuations in dopamine level to tell it whether or not it should increase or decrease long term potentiation.
  3. Something I haven't thought of?
$\endgroup$
  • $\begingroup$ Just a comment to point out that crediting dopamine for neuroplasticity is a tremendous reduction. Does this affect your question at all? $\endgroup$ – anongoodnurse Jul 9 '14 at 11:11
  • $\begingroup$ In a similar vein, much of the cortex doesn't even get any dopaminergic enervation, at least in non-human animals. Yet, plasticity happens. $\endgroup$ – jona Jul 9 '14 at 12:56
  • 1
    $\begingroup$ This is generally known as the temporal credit assignment problem (see scholarworks.umass.edu/dissertations/AAI8410337 ) and many different models attempt to explain how it might be solved in various brain systems. $\endgroup$ – Tim Aug 4 '14 at 17:32
1
$\begingroup$

The ability of dopamine to work on different time scales is discussed by Schultz. More generally, the basic idea of prediction error reinforcement based learning and DA is that the prediction error signal "propagates back in time". There is some doubt for how temporally precise this DA signal actually is, and if DA is fast enough to support RPE TD learning.

Does that answer the question?

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.