12
$\begingroup$

Are there any official APA guidelines for reporting linear mixed model results?

Can I take an approach similar to what would be done with an ANOVA (i.e., report the p-value, standard error and .95-CI for each main effect and interaction)?

What about the intercept? Any additional recommendations?

$\endgroup$

3 Answers 3

7
$\begingroup$

The APA style manual does not provide specific guidelines for linear mixed models. Additionally, a review of studies using linear mixed models reported that the psychological papers surveyed differed 'substantially' in how they reported on these models (Barr, Levy, Scheepers and Tily, 2013). It depends greatly on your study, in other words. Normatively speaking, I find this checklist (Ferron et al., 2008) extremely helpful in such cases.

Besides the more general checklist, Barr, Levy, Scheepers and Tily (2013) also provide some guidelines for reporting on LMMs. The whole paper is well worth a read, but excerpts of their "Reporting results" section are provided below.

One needs to provide sufficient information for the reader to be able to recreate the analyses. One way of satisfying this requirement is to report the variance–covariance matrix, which includes all the information about the random effects, including their estimates. ... A simpler option is to mention that one attempted to use a maximal LMEM and, as an added check, also state which factors had random slopes associated with them.

If it is seen as necessary or desirable in a confirmatory analysis to determine the random effects structure using a data-driven approach, certain minimal guidelines should be followed. First, it is critical to report the criteria that have been used, including the a-level for exclusion/inclusion of random slopes and the order in which random slopes were tested. Furthermore, authors should explicitly report the changed assumptions about the generative process underlying the data that result from excluding the random slope (rather than just stating that the slopes did not ‘‘improve model fit’’), and should do so in non-technical language that non-experts can understand. Readers with only background in ANOVA will not understand that removing the random slope corresponds to pooling error across strata in a mixed-model ANOVA analysis. It is therefore preferable to clearly state the underlying assumption of a constant effect, e.g., ‘‘by excluding the random slope for the priming manipulation, we assume that the priming effect is invariant across subjects (or items) in the population.’’

Searching the Barr article, I also found a highly relevant CrossVal.SE question which asks for examples rather than guidelines. Since no guidelines appear to exist, I would suggest that.

References

$\endgroup$
3
$\begingroup$

As an update, this paper may be helpful, though it comes from the medical field.

References

Monsalves, M.J., Bangdiwala, A.S., Thabane, A. et al. LEVEL (Logical Explanations & Visualizations of Estimates in Linear mixed models): recommendations for reporting multilevel data and analyses. BMC Med Res Methodol 20, 3 (2020).

$\endgroup$
2
$\begingroup$

There are some informative examples in Baayen, Davidson and Bates (2008), though some of their advice is outdated, having been supplanted by Barr et al. (2013) cited in the answer above. I found it useful to read these two papers together, though.

I'd like to add my voice to @Christian in stressing that one common gap in reporting such models is which random effects included just an intercept, and which (if any) included an intercept and a slope, along with a justification of of these decisions. Baayen et al. provide some examples of such justifications.

References

Baayen, R. H, Davidson, D. J., Bates, D. M. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390-412.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.