I'd like to know what would be the less biased measure of central tendency to compare RTs in two conditions with a lot of trials (~700). Mean is really sensitive to outliers, and median really robust, so I always used this last one. However it seems to me that I'm not really catching what's happening on the tail of the distribution with this measure. I'm not familiar with diffusion model though.
$\begingroup$
$\endgroup$
4
-
$\begingroup$ Less biased how? It seems like you're looking for a measure of disperson not central tendency, since you want to capture information in the tail and that's fundamentally not what central tendency speaks to. $\endgroup$– KrystaJul 23, 2013 at 22:01
-
$\begingroup$ Should this question be moved to CrossValidated? $\endgroup$– GalaJul 24, 2013 at 9:56
-
1$\begingroup$ I think Jeromy's really excellent answer below--which fits here very well, and points to some RT-specific properties of the population distribution--is argument enough that the question is OK here (although there would also probably be some interesting answers on CV). $\endgroup$– KrystaJul 25, 2013 at 14:20
-
$\begingroup$ I think it is on topic here and would be on topic on CrossValidated. Therefore it should stay here based on the general principle that where a question is on topic on multiple sites it is up to the asker to decide where they want to ask the question. There was a metapost about the issue of statistics questions on cogsci. $\endgroup$– Jeromy AnglimJul 26, 2013 at 1:15
Add a comment
|
1 Answer
$\begingroup$
$\endgroup$
You can find an accessible overview of some of the issues in Whelan (2008) which contains further references discussing the issue. Note that from a statistical perspective, the sample mean and median are unbiased estimators of their population equivalents. That said, with outliers, skew and the like, the standard error of the sample mean can be quite a bit larger than the sample median. Furthermore, you typically the population distribution should be seen as a multiple component mixture distribution (e.g., fast guess trials, "real trials", distracted trials). In such cases, you may be particularly interested in the central tendency of the "real trial" distribution. In that case, sample mean and median may be biased estimates of "real trial" equivalent population distributions.
Whelan (2008) suggests the following in order of preference when estimating a measure of central tendency:
- Mean after trimming the raw data for outliers (i.e., taking the middle 85% to 95% of RT data)
- Mean of log or negative inverse transformation of raw reaction time
- Median of raw RT
That said, the justification for a given method depends on the underlying distribution of the data and the nature of the effect of condition. For example, if condition is mainly influencing prevalence of outliers or skew, this will effect the mean more than the median in the population.
A good general strategy for exploring these issues and justifying your decision is to estimate central tendency in reaction times using a range of methods. Then correlate the scores you get for each individual using the different methods. If you are getting very high correlations (e.g., r>.95 or .99) then you will know that the method makes little difference.
Whelan (2008) also discusses strategies for estimating all parameters in the RT distribution.
If you are studying choice reaction time, then LBA or Ratcliff's diffusion model may be more appropriate.
References
- Whelan, R. (2008). Effective analysis of reaction time data. The Psychological Record, 58, 475-482. PDF
answered Jul 23, 2013 at 23:42