What are the state-of-the-art approaches to analyze reaction choice experiments?

Example of a reaction choice trial:

  1. Fixation cross (randomized length between 500 and 1500ms)
  2. Choice stimulus: Indicates to press left or right button (200ms)
  3. Response: Which and if one of the two buttons is pressed (max 1000ms after stimulus onset)

This procedure produces the following types of data:

  1. Response times, of
    1. correct responses
    2. incorrect responses
    3. both
  2. Correctness of responses
  3. Omitted responses

What are valid ways to analyze these interdependent yet disjunct types of results? Ideally including references to examples.

  • 2
    $\begingroup$ You may want to have a look at signal detection theory, which is also covered in this question: cogsci.stackexchange.com/q/541/11318 $\endgroup$ – Robin Kramer Nov 19 '17 at 9:43
  • 1
    $\begingroup$ What is your research question? In other words, data is generally gathered and, perhaps, structured to answer one or more questions. What's that question? With the question in mind, you can start to make a structured battle plan to tackle that question. I think it's not so much a matter of state-of-the-artness that answers your post here, really. Your analysis strategy will depend more on what you wish to know. $\endgroup$ – AliceD Nov 19 '17 at 19:32

Omitted responses are not usually analyzed (what does that even mean, no response before a deadline?, a "does not want to respond" button?).

If you want to analyze jointly choices and RTs, you need a model. I would say the most admitted model right now is the drift-diffusion model (Ratcliff & Rouder, 1998), which models choice and RTs as the accumulation of noisy evidence toward a decision boundary. These models are actually quite old (the late 50s), but they regained a lot of interest in the past 10 years. Some toolbox (e.g. Vandekerckhove & Tuerlinckx, 2008) will do that for you. Note that these models originally predict equal RTs for correct and incorrect choices, which is typically not what is found. You'll have to add parameters to predict these aspects which will require more data do produce good fits. You can also tweak the model to predict opt-out choices (Kiani, Corthell & Shadlen, 2014). You can tweak it some more to make it predict trials that failed to produce a response before a deadline, but I doubt any toolbox will do it out of the box.

These models are especially usefull if you have various conditions on the same scale (for example 5 levels of luminance), because that will strongly constrain your fits. If you want to analyze differences between a small number of conditions, especially if they are not on the same scale, I would avoid using them. (1) Fitting a 10+ parameter model to compare just 2 conditions is highly dubious (despite that people do it all the time), and (2) your results become model dependent. What if the model is wrong, are your conclusions still valid? What you can do is simply compare z-transformed ratio correct responses and median of RTs (because the are not normally distributed, you can also take their inverse which make them more or less normal) in your favorite statistical test and make sure there is no speed-accuracy tradeoff.

Ratcliff, R., & Rouder, J. N. (1998). Modeling response times for two-choice decisions. Psychological Science, 9(5), 347-356.

Vandekerckhove, J., & Tuerlinckx, F. (2008). Diffusion model analysis with MATLAB: A DMAT primer. Behavior Research Methods, 40(1), 61-72.

Kiani, R., Corthell, L., & Shadlen, M. N. (2014). Choice certainty is informed by both evidence and decision time. Neuron, 84(6), 1329-1342.

  • $\begingroup$ Thanks for your answer! Concerning: "what does that even mean, no response before a deadline?" yes $\endgroup$ – thando Nov 25 '17 at 21:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.