I'm trying to program a Hebbian learning rule for a simple fully connected deep neural network (DNN), which is structured as:

$$z_i = W_iy_{i-1}+b_i$$


$$y_i = g(z_i)$$

Based on Hebbian learning rule, weight update is proportional to the product of pre and post-synaptic activations.


Does this mean $W_i$ is updated based on $y_{i-1}$ and $y_i$ or $y_{i-1}$ and $z_i$?


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.