Is there a neural network model of Pavlovian Learning?

Question

I am trying to find a computer simulation of Pavlovian learning. i.e. an action such as salivation in response to a stimulus such as a bell ringing.

Most neural network models I've seen seem to be more about trying to recognise things such as handwriting or speech rather than the stimulus-response model. And they don't usually incorporate the temporal nature of Pavlovian learning. e.g. Ring a bell, wait a bit, give dog food.

Are there any computer simulations of this? (Do they have a name?) Is there a connected neuron model of this?

Answer 1

In general, what you're looking for is a biologically plausible model of reinforcement learning and/or conditioning. I know of two publications in particular that address this.

The first is A Biologically Plausible Spiking Neuron Model of Fear Conditioning and the second is A Spiking Neural Integrator Model of the Adaptive Control of Action by the Medial Prefrontal Cortex. They both using the Neural Engineering Framework and a learning rule to modify the connection weights between ensembles of biologically plausible spiking neurons. These connections are modified to create and destroy associations between stimuli and an action. However, in the second paper, it's shown that the mechanism can also learn timing information.

Answer 2

I would classify pavlovian learning as a type of hebbian learning. Where events that occur together positively reinforce each other (different from reinforcement learning).

This idea has been modified into hopfield networks, and then their descendants boltzmann and restricted boltzmann machines. They use an algorithm called contrastive divergence which is effectively hebbian learning. It tries to make events that occur together produce more stable states, and events that do not occur together are made less stable, thus given a partial state, the network will be attracted to the stable state.

So if a bell rings, and food is received the net will learn that this is a (more) stable state and when/if only the bell rings then the net will naturally be attracted to the state, bell + food, as all other states are less stable.

(Unfortunately I do not understand contrastive divergence particularly well, it may be better to look it up yourself).

This work leads into unsupervised learning, deep learning and auto-encoders.

Although, on second thoughts, I should clarify that this doesn't currently work with temporal difference. So I don't know if I really answered the question.

Answer 3

A true classic -- the Configural-Cue model -- uses the Rescorla-Wagner rule to learn associations between cues and outcomes. Link1 Link2 Link3

In my view this is one of the most straightforward (i.e., simplest) models of conditioning, likely a good starting point for you.