In the Temporal difference learning algorithm (TD-learning), an agent seeks to predict the total value of future rewards that will be received during the current trial. The agent updates the prediction as the trial progresses, based on conditioned stimuli (CS) that have been seen so far. Specifically, the prediction is calculated as a weighted sum of the intensities of past CS presentation. At time $t$, the prediction is calculated as
$$v(t) = \sum_{z=0}^t w(z)u(t-z)$$
where $v(t)$ is the prediction at time $t$, $u(t)$ is the CS intensity at time $t$, and $w$ is an array of weights.
My question is: why are we summing the terms $w(z)u(t-z)$, rather than $w(z)u(z)$, i.e. why isn't each weight associated with the CS strength at a particular time?
[Thanks to user honi's reply, I subsequently understood that each element of the weight array is associated with an elapsed time since seeing the CS. For example, $w(3)$ is the weight given to the CS strength experienced three units of time ago.]
[Original phrasing of the question: I am reading Theoretical Neuroscience by Dayan and Abbot, and I am confused by the use of $w(t)$ in equations (9.6) and (9.7). In the former, it is a window function (linear filter) and in the latter it is the weight function. Is this (a) an unfortunate use of the same name for two different things, or (b) is the weight function really used as a linear filter to calculate $v(t)$?]
