I've read theoretical papers on why constraints are a necessary part of any learning system. The papers were aimed at human learning. I'm having trouble finding relevant references now.
What papers deal with the theoretical issue of whether or not learning must be constrained?
If that suffices, I can give you the classical article from the domain of language learning: Gold 1967.
The basic intuition is that an infinite number of grammars could explain any given set of strings. Analogously, you can probably consider fitting a polynomial to a set of points, or instances of the inverse problem (reconstructing a source from observations). In language acquisition, without some prior constraint on the to be acquired grammar, therefore, no unique convergence will happen. This has led to the Poverty of the Stimulus class of arguments, and the wide range of responses to it. There is a wide range of responses to the Gold paper - if you want more, it really depends on what specifically you're looking for.
In practice, I don't think anyone can imagine a learner in the real world that is without constraints, first and foremost those of storage and computational power.
There's a related saying in statistics:
you cannot do inference without making assumptions.
From Information Theory, Inference, and Learning Algorithms (page 26) by David J. C. Mackay.
Inference about the world is learning, and if you don't assume anything about the world, you can't learn to generalize. Assumptions limit your hypothesis space.
