I am analysing data on an emotion perception task, where participants must decide if a given face is happy, sad, angry or fearful (essentially a forced choice between 4 options).
I am confident that I want to calculate an unbiased hit rate for each emotion (as in Wagner 1993). This takes a confusion matrix (toy example below):
Happy Sad Angry Fearful Happy 4 1 0 1 Sad 0 3 2 1 Angry 0 0 6 0 Fearful 2 0 0 4
And the unbiased hit rate is then the correct answers squared, divided by the number of times that answer is used multiplied by the number of trials where that answer was correct. So in the above example:
Hu(Happy) = 4^2 / ((4+0+0+2)*6) = 16/36 = 0.444 Hu(Sad) = 9/24 = 0.375 Hu(Angry) = 36/48 = 0.75 Hu(Fearful) = 16/36 = 0.444
These are then converted with an arcsine tranformation on the square root for use as the dependent variable in regression.
However, I want to use this data to assess general ability at emotion recognition in faces - what is the correct way to combine the unbiased hit rates? That is, can I generate a single value that accounts for all off these, or should I run four separate analyses and examine the overlap in the results?
Wagner, H. L. (1993). On measuring performance in category judgment studies of nonverbal behavior. Journal of Nonverbal Behavior, 17(1), 3-28.