I found a study by Mullen (1994) which discusses the empirical literature on earliest childhood memories (see also the wikipedia article on childhood amensia).
Here are a few interesting quotes:
The average age of earliest memory is consistently found to be in the vicinity of 3.5 years.
Mullen mentions theories for the amnesia prior to this point focussing particularly on the role of language development and training provided by parents in how to socially construct accessible memories.
Mullen distinguishes between types of memories which in some sense pertains to your question about senses:
White and Pillemer (1979) suggest that there may be a dual system of
autobiographical memory in which one aspect is a verbally accessible
store of interconnected narratives organized with respect to a time
frame and the other aspect is an unorganized collection of fragments
of images, behaviors and emotions.
In study 3, Mullen (1994) examined the content of earliest reported memories.
Here is a table summarising content features of these memories:
Thus, it appears that most of these memories reflect an event in some sense.
Of course there are a bunch of methodological things going on here.
I imagine if you delve into the references or these related articles you will find out more on the topic.
Kihlstrom et al (1982) looks like it might be worth a read.
- Mullen, M. K. (1994). Earliest recollections of childhood: A demographic analysis. Cognition, 52(1), 55-79.
- Kihlstrom, J. F., & Harackiewicz, J. M. (1982). The earliest recollection: A new survey. Journal of Personality, 50(2), 134-148.