Difference between reverse inference and decoding (e.g. MVPA) in fMRI

Question

Reverse inference, i.e. the use of brain activation (fMRI) data to infer the engagement of a certain mental function, is heavily criticised (e.g., Poldrack 2011, Neuron).

At the same time, decoding, the process of applying machine learning algorithms to such brain activation patterns in order to infer something about the mental function that elicited this response (e.g. to distinguish between two different conditions in an fMRI experiment), is extensively used and actively developed - multivariate pattern analysis (MVPA) being just the most widespread of these techniques.

The two concepts appear, however, to be one and the same; or rather, that the second (decoding) is a special case of the first (reverse inference). Is there an additional nuance to their difference? Also, why is decoding not criticised in the same way as reverse inference?

Answer 1

Short answer: Decoding is not a special case of reverse inference.

The difficulty with interpreting neuroimaging results is that there is a tremendous amount of variability (noise) in the data. For example, say we attempt to determine the brain areas associated with the emotion of romantic love by showing subjects images of close friends (condition 1), or images of their loved ones (condition 2), and comparing the results. Each brain scan may show 5-10 active regions, which regions are active and to what degree varies between subjects even in the same condition, and there is even variability in brain scans of the same subject across multiple trials.

To deal with this variability, the first step in just about any neuroimaging experiment's data interpretation process is a statistical analysis. This can range from an "averaging" or "noise-cancellation" analysis, to a multi-voxel / multi-frame machine-learning pattern-matching classifier (MVPA). The data analysis is used to determine a predictor of the independent variable, and also to calculate significance (p-value) based on the consistency level in the data. For example, we may learn that subjects viewing images of their loved ones (condition 2) have 5 areas of their brain reduce activity and 4 areas increase activity relative to subjects viewing images of other close relations (condition 1). Note that the decoding step does not involve "labelling" the pattern as any particular mental state - a classifier like MVPA does not care what the patterns mean, it's just a mathematical function used to distinguish between them.

In a typical cognitive neuroscience experiment, the next step is to attach a label to the mental state defined by the pattern found - this is forward inference. To do this, researchers typically determine the mental state through some other validated measure - for example, a questionnaire or some other test that is known to reliably measure love. The researchers may conclude: "These regions of the brain are associated with the emotion of romantic love when subjects view images of their loved ones." Immediately, a constraining factor arises: This pattern of activation is relative to the control condition only!

While forward inference involves labelling the pattern from the independent variable, reverse inference is used to label the independent variable from an activity pattern. Reverse inference gets into trouble when it is used to label a particular pattern of activation in a different context. For example, "the same areas of the brain associated with the emotion of love in an experiment involving showing subjects images of their loved ones, also show the same pattern of activation when subjects view their iPhones, and therefore we conclude that subjects are in love with their iPhones." Note that statistical analysis of the data (decoding) must take place first in order to be able to make either kind of cognitive inference.

Reverse inferences suffer from several potential pitfalls, among them:

• A particular pattern of activity usually indicates different mental states in different contexts, so assuming that the same pattern indicates "love" when applied to iPhones is invalid.
• Patterns isolated using a statistical analysis usually do not fully describe the mental state, so assuming that the pattern is sufficient to indicate "love" is also invalid.
• Mental state labels themselves are context dependent, subject-dependent, and even temporally variable, so assuming that subjects would interpret their own mental state as "love" is invalid as well.
• Mental states are not discrete, as evidenced by all the "noise" in neuroimaging data, so subjects may identify their mental state using a variety of different labels depending on what part of their mental state they attend to, "love" being only one.

Reverse inference is subject to these issues no matter what statistical analysis is used, MVPA is not immune. However, many of these issues can be overcome using a variety of techniques. MVPA is very valuable for dealing with the first issue - thanks to much higher "resolution" (level of detail, scope of data, linearity, etc), this decoding technique is significantly less likely to confuse different mental states that appear to have the same activity pattern at a lower resolution. Another important strategy is using meta-analysis (data gathered from many different studies) to determine how useful a particular pattern is for reliably inferring a mental state. Solving the remaining problems will likely require per-subject training data, and even more advanced analysis techniques, but already reverse inference has been a useful methodology in many studies, when carefully applied.

An additional interesting side-effect of machine-learning pattern analyzers is that they tend to remove the temptation to make unwarranted reverse inferences. In the early days, when patterns of activation were "informal" - based on gross anatomy - it was too easy to make sweeping statements such as: "... the insular cortex of the brain, which is associated with feelings of love ..." With MVPA, patterns of activation are practically impossible to describe in human language, so machines are required to make inferences, which are then inherently more objective.

Answer 2

Note. I initially scan read the question, I have rewritten my answer as a consequence, and due to the comments given.

As highlighted by others here Multi-voxel pattern analysis (MVPA) is an application of machine learning, used for decoding vast quantities of complex information (neural activation patterns to particular asks). This is a form of decoding may be used to infer a cognition, otherwise known as reverse inference.

The problem of reverse inference is largely summed up by the following comment from Poldrak's (2011)...

The use of reasoning from activation to mental functions, known as “reverse inference”, has been previously criticized on the basis that it does not take into account how selectively the area is activated by the mental process in question.

Poldrack (2011) goes on to explain that informal reverse inference, which is based on a researchers knowledge, is flawed because an individuals knowledge is limited by what the remember and have read. Additionally poor interpretations are compounded from one researcher to another.

The problem with reverse inference comes not from understanding general cognitive processes, such as sight, movement, language, decision-making etc. General patterns for general processes have been established, the problem as Poldrack (2011) points out is when we interpret patterns to more well defined cognitions, for example instead of looking at merely reward processing, we might want to compare patterns of pleasure derived from seeing highly palatable e.g. cake, and less palatable food e.g. fruit. At this level, comparing the data requires far more specific analytical approach. If we were a researcher making an inference based on our knowledge, we would have a strong chance of error.

MVPA deals with a far higher resolution of data than an individual could deal with, and compares data to prior trials or previous experiments, see Fig 1. However it is critical to remember that we are comparing participants within similar contexts.

Fig 1. MVPA diagram of testing and inference Norman et al (2006)

Poldrack (2011) gives an excellent example of how to use MVPA, which was conducted by Kay et al (2008). Simply Kay et al (2008) scanned participants viewing natural images n= 1750. In the following trial 120 images were added, MVPA of the neural data was able to accurately predict which images were being viewed. This method has seen substantial development to the point that using a similar method researchers have accurately analysed neural patterns observed in a participants waking brain into decoding what participants are dreaming. Basically the quantity and quality of when taking account of context may be used to conduct reverse inference, but this does require previous data, within context, for the MPVA and other machine learning methods to compare too.

This does not mean that machine learning can not be wrong if incorrectly applied. It is a statistical method in which humans set the parameters, the post below by Arnon Weinberg accurately defines the issues and pot holes that must be avoided to make this method viable for reverse inference.

Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006). Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends in Cognitive Sciences, 10(9), 424–430. http://doi.org/10.1016/j.tics.2006.07.005