I remember trying using single-epoch measurements with electroretinogram (ERG) recordings. Basically, ERG responses are electrical potential recordings from the neural activity in the retina. ERGs are usually also obtained using ensemble averages.
The single-sweep ERG amplitude measures were, as expected, also really messy due to random noise and eye movement artifacts that are normally averaged out when using ensemble averages. However, when using all 250 available individual epochs I had at the time, I could still show that ERG amplitudes slowly declined along the duration of the recording. So I took advantage of all available recordings by plotting all available amplitudes against time andand performing linear regression. Linear regression is a particularly robust statistical method when a lot of data is available.
In your case, where you basically wish to do an analysis of variance on populations of people, I would try to pool all the data available per subject and determine the standard deviation, or a related variance parameter. Then doing that for all 20 normal subjects will yield a collection of variabilities. This could then be compared to the older population. For the exact statistics I cantcan't advice you on the most appropriate statistical test unfortunately, but for a starters you can simply plot the 2 point clouds and compare them visually. If it looks promising, you can do a t-test (which is likely not the most appropriate test, but it will give you a fair indication). If it all looks promising, you have to find a proper analysis of variance to compare the two.