Recently i read an article on Bayesian Program Synthesis. It says that Bayesian cognitive modelling would help AI systems to learn from few examples in comparison to deep learning. But i was not able to understand the advantages of using Bayesian vs other formulations in cognitive modeling. I would like to understand why this Bayesian approach stand out ? What are the drawbacks of current approaches to cognitive modeling ?
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
Here's a quick answer from general background knowledge, not from any specific knowledge of "Bayesian Program Synthesis (BPS)"
In general, Bayesian models can use strongly informed priors or diffuse "could be anything" priors. Strong priors specify that a lot of parameter values are very unlikely, while a few other parameter values are possible descriptions of the data. With a strong prior, it takes relatively little data to winnow down the possibilities (i.e., faster learning). Weak priors allow for a much broader range of possibilities, but it takes a ton of data to narrow down the possibilities (i.e., slower learning).
Many Bayesian models of mind use strong priors. Such models come pre-set with particular structure or parameter dependencies that are tuned for the type of problem that needs to be learned. This is not "cheating;" the prior is a crucial part of the theory. Another example comes from computer vision: Figuring out what objects and lighting produced a pixelated 2D image can only be done by assuming strong prior knowledge of the types of objects and types of illumination "out there" in the world, otherwise it's an unsolvable problem. An introductory article is here or here (specifically, see the section on "Prior Knowledge" starting on p. 17).
Deep learning, however, is (as far as I know) a generic approach that can learn just about any possible relationship among variables. The cost of this weak prior is that it takes a ton of training to narrow down the parameter distribution.
