In my mind there are two main explanations of this kind of instinct behaviours.
- The first one is rooted in evolution. There are many examples of human innate behaviours which we can't explain e.g. when we see a lace or tape on the street we automatically jump and feel scared. Although we live in big city our brain associates the lace with a snake. It is an effect of our "prememories", memories of our ancestors.
-> Buss D., Evolutionary Psychology: the New science of the Mind (1999)
- The second explanation is more connected with your question as it refers to animals' innate instincts such as birds' travels (how do they know where to fly?) etc. Some interesting experiments were conducted on a phenomenon called epigenetic inheritance. An example is teaching animals some reactions after specific stimulus (e.g.fear after specific smell etc.). Then we observe the learned behaviours in the descendants of these animals. Offspring of trained animals show the same reaction , although nobody trained them to do that. There is an abstract of such experiment with mice:
http://www.nature.com/neuro/journal/v17/n1/full/nn.3594.html
And there you can read more about it in another article:
http://www.nature.com/news/fearful-memories-haunt-mouse-descendants-1.14272#/b1
-> Griffiths, Gelbart, Miller. Modern Genetic Analysis.(1999)