I'm looking at analyzing the just noticeable differences (JNDs) in temporal order judgment (TOJ) studies. A modern way to do this is with a multilevel model. And, I am going to do that. But I also need to actually extract and analyze JNDs by participant in a more traditional way.

I'm doing it by generating a logistic regression for each participant with whether X was selected as first as the response and the TTOA (target-target SOA) as the predictor. Then I go through the model function and find the TTOA for when the probability of selecting X first is just less than 25% and when it is just greater than 75%. The difference between those two TTOAs is 2 x JND.

But what do people do when the found values from the regression are actually outside the times used in the experiment? Or, what does one do if the function seems excessively shifted so that the point of subject simultaneity is well off centre?

I'd be happy for a pointer to where this sort of analysis is explained as well.

  • $\begingroup$ I'm not familiar with this type of experiment, but on the statistical model side I don't see why you wouldn't have exactly the same issue with a MLM. But yeah, if your regression can give numbers outside the possible range that's a good hint that the model assumptions aren't met by your scenario, so either you have to live with it and consider it good enough, or use a different approach. Is it possible, though, that you're dealing mostly with a complete separation problem? $\endgroup$
    – Bryan Krause
    Oct 26, 2021 at 1:08
  • $\begingroup$ @BryanKrause, the reason that the MLM won't likely have the issue is that the issue only occurs in the occasional participant's data, not generally. The fixed effect from the MLM would very likely end up not having the issue. $\endgroup$
    – John
    Oct 27, 2021 at 2:05
  • $\begingroup$ Got it, well then it seems like it may not be possible to estimate from the single participants - sounds like there isn't enough data per subject. $\endgroup$
    – Bryan Krause
    Oct 27, 2021 at 2:18
  • $\begingroup$ @BryanKrause you're probably right. That's true of the vast majority of cognitive or cog-neuro studies. But keep in mind I'm not trying to design a study. I'm trying to find out how people typically analyze this data that I have and deal with the difficult bits. $\endgroup$
    – John
    Oct 27, 2021 at 20:14
  • $\begingroup$ Sure, but given the issues you've raised it seems likely to me that either A) they do so (analyze by individual participant) only after collecting substantially more data per participant, or B) they don't at all. You could also go the Bayesian route but I'd be careful in interpreting estimates where the priors are dominant unless you have a good reason for them to be. $\endgroup$
    – Bryan Krause
    Oct 27, 2021 at 20:52


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.