Combinatorial woes

Question

I am interested in the creation of chunks (aka configural nodes) from smaller chunks and input features (only interested in System 1 cognition).

Unitization studies (e.g. Goldstone (pdf)), suggest that we start with generic features, and slowly combine them into more specific chunks. As we do this unitization, we wouldn't store all of the learned combinations --- a mere 10 input features can be combined in 3.6 million configurations!

Simon suggested generic elements get overwritten by more specific ones at each exposure. but what would you do with the predictions/rules learned about a given chunk? Transfer them to the newly created specific chunk? What if they do not apply at that more specific level?

Nosofsky was suggesting a rule-plus-exception model, where the more generic elements are always stored and their predictions are recorded as general rules; When those predictions are broken, more specific chunks are stored to explain the exceptions to these rules. What does it mean for predictions to be broken? In a probabilistic environment like ours, sometimes a prediction is correct, sometimes it isn't -- updating the weight of the prediction seems much more reasonable than deeming it invalid in favor of an 'exception'.

When do I chunk two co-occurring features? When do I chunk those two with a third? Do I delete the 2-feature chunk in favor of the 3-feature? What do I do with the memories associated with the deleted chunk?

Answer 1

Recently Tom Griffiths and I worked on the problem of how to use distributional cues from the raw sensory data given by the environment to form representational units (features). The basic idea is to form a probabilistic model over possible feature representations. The model uses the observed data and a bias towards simpler feature representations (those with fewer features) to infer a feature representation. The tricky thing is forming a probabilistic model over the complex, infinite-dimensional space of feature representations and once that is defined, the practical part of inferring the feature representation given a set of observed objects.

For more details on how to do this, what it means theoretically, and experiments testing some predictions of the approach, take a look at our recent article in cognitive psychology, doi.

Also, if you're interested, we've done some work looking at inferring transformation-invariant features, pdf. We're nearly ready to submit a larger article on this line of work. Send me an email if you're interested.