0
$\begingroup$

I have two groups (13 experimental and 13 control participants) performing on two cognitive tasks. I have accuracy (omissions, commissions) and reaction time measurement from each test for each person. To avoid type-I error, I was suggested to decrease my number of DVs, I now have d-prime (d') and reaction time (RT) as the DVs (both continuous). I would like to compare performances on both the tasks, across both the groups. My questions pertaining to this design are:

  1. Is d-prime a good measurement to substitute accuracy scores ? or could you please suggest a unitary index that would reflect behavioral performance (from omissions, commissions, RT data)

  2. Should I do a one way MANOVA? (IV- groups; DV- test1 d', test2 d', test1 RT, test2 RT)?

  3. or Should I do a mixed model MANOVA? (if yes, how to do this, what are my DVs)?

Improve this question
$\endgroup$
2
  • $\begingroup$ If you only do stats on d', you could miss an effect on the bias (e.g., beta) that you would catch if you used both omission and commission errors. If you do stats on both d' and beta, then you are not reducing the number of DVs. That said, I personally like d' and beta over error rates. $\endgroup$
    – StrongBad
    Commented Mar 28, 2016 at 17:59
  • $\begingroup$ Thank you very much for your suggestions. Therefore you would suggest that my first DV is d’ (calculated from Probability of corrects and Probability of false positives/commission measures). And the second DV is beta (which is probability of false negatives i.e. omissions). Am I correct? Also if you could please suggest if I had two DVs from each test (which is total of 4 DVs), should I do a one way MANOVA (considering there are 4 different DVs) or do a mixed model MANOVA (considering there are two repeated measures for each test)? I would really appreciate your feedback. $\endgroup$
    – Mibo
    Commented Mar 29, 2016 at 20:45

1 Answer 1

Reset to default
3
$\begingroup$

If you have single-trial data, the drift-diffusion model/DDM and related models, originating with Roger Ratcliff (1976/1978), can simultaneously fit the whole response distribution, both RTs and accuracies. It captures phenomena such that in some experiments, errors are systematically faster or slower than correct responses.

Fitting and interpreting the DDM can be non-trivial, but it has many advantages, such as

  • accurately accounting for the distribution of RT data
  • directly relating to cognitive processes (e.g. evidence accumulation speed, sensory encoding speed)

The DDM works by modeling the decision process as a random walk next to (usually two) decision thresholds (e.g. corresponding to the correct and the incorrect button in a 2-alternative false choice task), which after an initial period of encoding begins drifting towards the correct boundary at a speed corresponding to the effectivity of taking up evidence. Occasionally, the drift process reaches the wrong boundary. When a boundary is crossed, execution of the corresponding response is initiated.

The DDM is fitted to the whole RT distribution and the resulting parameters can be submitted to statistical tests between conditions. For an example of hierarchical Bayesian estimation of the model, consider HDDM.

References

Ratcliff, R. & Murdock, B. B., Jr. (1976). Retrieval processes in recognition memory. Psychological Review, 83, 190-214.
Ratcliff, R. (1978). A theory of memory retrieval. Psychological Review, 85, 59-108.
Forstmann, B. U., Ratcliff, R., & Wagenmakers, E.-J. (2016). Sequential sampling models in cognitive neuroscience: Advantages, applications, and extensions. Annual Review of Psychology, 67, 641-666.
Ratcliff, R., Smith, P.L., Brown, S.D., & McKoon, G. (2016). Diffusion decision model: Current issues and history. Trends in Cognitive Science, 20, 260-281.
Wiecki TV, Sofer I and Frank MJ (2013). HDDM: Hierarchical Bayesian estimation of the Drift-Diffusion Model in Python. Front. Neuroinform. 7:14. doi: 10.3389/fninf.2013.00014

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.