Jens
' answer is pretty much spot on, but misses the fact, remembered from my undergraduate lectures, that your ears actually partially 'turn off' when you speak (or chew), in what's called the stapedius reflex (wikipedia).
The most common reference I've seen for this is Møller (2000), which unfortunately is a book, but I'm sure more information could be found with a little digging.
###Reference
Reference
Møller, Aage (2000). Hearing: It's Physiology and Pathophysiology (illustrated ed.). Academic Press. pp. 181–90.