3
$\begingroup$

In the Leabra cognitive architecture, sparse representations are created by simply ignoring all but the k strongest activations, if I understand correctly. In Hierarchical Temporal Memory, instead, each firing neuron inhibits an area of a given radius during this time step. To me, this seems more plausible due to the locality of neuronal connections.

Is the k-winner inhibition biologically plausible?

$\endgroup$
3
$\begingroup$

As mentioned by honi, it is possible to have bio plausible k-means. However, neither Leabra nor Hierarchical Temporal Memory (HTM) use it.

As Section 3.5.1 of How to Build a Brain by Chris Eliasmith notes:

There are basic computational methods underlying Leabra that are of dubious plausibility. The most evident is that Leabra directly applies a k-Winner-Takes-All (kWTA) algorithm, which is acknowledged as biologically implausible: “although the kWTA function is somewhat biologically implausible in its implementation (e.g., requiring global information about activation states and using sorting mechanisms), it provides a computationally effective approximation to biologically plausible inhibitory dynamics” (http://grey.colorado.edu/emergent/index.php/Leabra; see also O’Reilly & Munakata [2000], pp. 94-105 for further discussion). In other words, the actual dynamics of the system are replaced by an approximation that is computationally cheaper on digital computers.

The same problem occurs with the HTM as discussed in this forum post by Travis DeWolf:

In the second step of their spatial pooling, they find the k most active columns, to apply learning to only these columns. Dynamically, setting up WTA with lateral inhibitory connections is notoriously very tricky, and isn't something that can be done in a single time step. On top of that, controlling the learning so that it's only applied after the network has settled on a set of winners is a whole other issue. It might be the case it works running the whole time as the WTA circuit settles, but the dynamics are complex and can't just be assumed to work.

To summarize, k-winner inhibition is possible, but it's not used in a biologically plausible manner in either Leabra and HTM.

$\endgroup$
2
$\begingroup$

Yes, see "A Second Function of Gamma Frequency Oscillations: An E%-Max Winner-Take-All Mechanism Selects Which Cells Fire" by de Almeida, et al., 2009 for a biologically plausible implementation.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.