5
$\begingroup$

I'm using likelihood ratio testing to assess whether a behavioral model is a better description of my data than a simpler (so called restricted) model.

How should results of such statistical tests be reported?

$\endgroup$
  • 1
    $\begingroup$ You might have more luck getting an answer at CrossValidated. stats.stackexchange.com $\endgroup$ – Seanny123 Jun 26 '15 at 15:41
  • 1
    $\begingroup$ Thanks, Seanny123, but I strongly doubt that CrossValidated will have a useful answer. My question has to do with reporting a common statistical test in psychology journals. $\endgroup$ – blz Jun 28 '15 at 14:30
5
$\begingroup$

General reporting recommendations such as that of APA Manual apply. One should report exact p-value and an effect size along with its confidence interval. In the case of likelihood ratio test one should report the test's p-value and how much more likely the data is under model A than under model B.

Example: The data is 7.3, 95% CI [6.8,8.1] times more likely under Model A than under Model B. The hypothesis that the data is equally likely under the two models was rejected with p=0.006.

The above statements already indicate that likelihood ratio test does not tell you which

model is a better description of my data

as the likelihood is $p(\mathrm{Data}|\mathrm{Model})$ and to learn which model is a better description of the data you need to compute $p(\mathrm{Model}|\mathrm{Data})$.

$\endgroup$
  • $\begingroup$ Thanks! Isn't the distinction of $p(\mathrm{Data}|\mathrm{Model})$ vs $p(\mathrm{Model}|\mathrm{Data})$ somewhat academic? In principle, the probability of data given a model is closely related to the probability of a model given data, no? Colloquially speaking, is it really that incorrect to say that a likelihood ratio test helps us choose a better model for our data? And another question: I've run LRTs independently for 6 conditions over 20 subjects, so it's impractical to give precise p-values, CIs and probability ratios for each. How can I summarize these results? $\endgroup$ – blz Jun 29 '15 at 13:04
  • 1
    $\begingroup$ @6 conditions: one should avoid multiple tests. It's cumbersome to present such results and you are running into problems with multiple comparisons. Instead formulate a single hypothesis about the pattern of results you expect to emerge across all 6 conditions and test it. If you don't have such a hypothesis then avoid testing and do exploratory analysis. For instance I would plot the log-likelihood for each condition. Differences on a logscale translate into multiplicative differences on the original scale. Then it's easy to derive "A is X times more likely than B" statements from such graph. $\endgroup$ – matus Jun 29 '15 at 13:47
  • $\begingroup$ I'm not sure I understand in concrete terms. We expect model A to better fit data from any condition (2x3, i.e. congruency by SOA, as this is a Posner cueing task), so we run a LRT for each subject and condition. We find that each LRT result favors model A at p < .001. Could you please formulate your recommendations in concrete terms? What should I be doing differently, here? Thanks again -- your comments are very useful! $\endgroup$ – blz Jun 29 '15 at 13:55
  • 1
    $\begingroup$ @blz, then just do a single likelihood ratio test on all data and all conditions. Strictly speaking, the influence SOA and congruency is not identifiable anyway. A voodoo solution would be to fit both models to each subject and each condition and then run multivariate regression with loglikelihood as DV and SOA and congruency as IV. $\endgroup$ – matus Jun 29 '15 at 15:50
  • $\begingroup$ While it's strictly true that for a single likelihood it's the likelihood is of the data | model, the test keeps the data constant and the ratio solves this issue. It can be interpreted as a comparison of the models. $\endgroup$ – John Jun 6 '17 at 3:30
1
$\begingroup$

The likelihood ratio test is distributed as χ²with degrees of freedom = the change in degrees of freedom between the two models. So, to give an example dropping one parameter from a model, you would report it like this:

χ² (1) = 3.4, p = 0.065

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.