It may have to do with language.
The idea that language influences thought is called Linguistic Relativity:
The more accepted weak version claims that linguistic categories and usage only influence thoughts and decisions.
Lera Boroditsky summarizes research on the influence language has on direction and time:
English speakers tend to talk about time using horizontal spatial metaphors (e.g., "The best is ahead of us," "The worst is behind us"), whereas Mandarin speakers have a vertical metaphor for time (e.g., the next month is the "down month" and the last month is the "up month"). ... Imagine this simple experiment. I stand next to you, point to a spot in space directly in front of you, and tell you, "This spot, here, is today. Where would you put yesterday? And where would you put tomorrow?" When English speakers are asked to do this, they nearly always point horizontally. But Mandarin speakers often point vertically, about seven or eight times more often than do English speakers.
And in more detail in a Scientific American article:
... my colleague Alice Gaby of the University of California, Berkeley, and I gave Kuuk Thaayorre speakers sets of pictures that showed temporal progressions—a man aging, a crocodile growing, a banana being eaten. We then asked them to arrange the shuffled photographs on the ground to indicate the correct temporal order. We tested each person twice, each time facing in a different cardinal direction. English speakers given this task will arrange the cards so that time proceeds from left to right. Hebrew speakers will tend to lay out the cards from right to left. This shows that writing direction in a language influences how we organize time. The Kuuk Thaayorre, however, did not routinely arrange the cards from left to right or right to left. They arranged them from east to west. ... in Aymara, a language spoken in the Andes, the past is said to be in front and the future behind. And the Aymara speakers’ body language matches their way of talking: in 2006 Raphael Núñez of U.C.S.D. and Eve Sweetser of U.C. Berkeley found that Aymara gesture in front of them when talking about the past and behind them when discussing the future.
This suggests that if you reversed the exercise, binding the left/right keys to movement forward or back, then you might get different results from Hebrew speakers than English ones. And if you presented this exercise to speakers of Kuuk Thaayorre, then the results might depend on which way the keyboard is oriented - ie, you would get different results depending on whether the keyboard faces East or West!