How to analyse 3 by 3 design with Likert scale ratings?

Question

I want to understand what would be the best way to analyze the results of an online likert-scale ratings of pictures on 3 dimensions. I have 3 groups of participants from different countries. They are not equal. I got the data into SPSS already but now I wonder what would be the appropriate way to deal with the data.

Answer 1

Without knowing more, I imagine you could do a 3 x 3 mixed ANOVA with three levels to the within subjects factor "dimension" three levels to the between subjects factor "group". This is all predicated on the idea that each dimension is rated on the same scale.

Unequal group sizes typically wont be a major issue. The important consideration is that you have sufficient sample size in each group to give you reasonable precision. That said, it might be more of something to think about if you have unequal group variances.

There is also quite a lot of discussion on whether you can analyse Likert scales or Likert-type items using linear model approaches like mixed ANOVA. Perhaps see some of these discussions. In general, if you each variable is the mean of multiple items, then you should be less concerned about using the linear model. If you are dealing with single Likert-type items, then there may be a small amount of imprecision in your p-values. This can be a particular problem if you have floor or ceiling effect issues.

Here's a tutorial on mixed ANOVA in SPSS.

Answer 2

A rating scale is not necessarily a Likert scale. It sounds like you have a single rating for each picture across each dimension but I could be wrong. More details would be needed for better advice and Ana's comment should be answered in your question.

You would be best off doing a multi-level cumulative link model (multi-level ordinal regression). You can include subject and picture as random variables. See the ordinal package in R. The vignettes will be very helpful.

Keep in mind that with the pictures you also have item level variance and generalization issues that people doing research on words struggle with as well. Even if you can get reasonably good looking data for your analysis that satisfies an ANOVA you'll have to consider random and fixed effects issues of both the people and the pictures. The multi-level modelling can include both and avoid the sort of patchwork hoops required with ANOVA. Therefore, regardless of whether you have a true likert scale and you satisfy requirements for treating the responses continuously or you should be treating it as ordinal data, multi-level modelling is your best bet.