4
$\begingroup$

Most resources mention using 20 'practice' trials categorizing both targets and attributes, followed by 40 'real' trials categorizing both targets and attributes. Note that this is after practicing with targets and attributes alone. Is there a downside to using more than 20/40 trials? (I know some recommend using extra practice trials when learning the new target/button assignment.)

$\endgroup$
  • $\begingroup$ Loss of attention caused by boredom and fatigue. $\endgroup$ – user3116 Feb 25 '15 at 10:20
2
$\begingroup$

The main threat to a design's validity from increasing the amount of trials in any experiment comes from participant motivation and attention. After sitting in front of a monitor for a while, participants get tired, as anyone would. As a personal rule of thumb, a session should therefore not go beyond 40 minutes without breaks if possible. Rather than going with my anecdotal advice, though, I suggest you run planned analyses to check and/or control for any decline in performance.

Given recent developments, it would seem there is generally more of a downside to using the typical rather than to not doing so. The methodological skeletons have come tumbling out of the field's closet one after the other the last few years, and as we are seeing in recent open access efforts such as Many Labs, power matters.

Instead of going with the tried and (maybe, possibly) true, run a power analysis to determine your ability to detect an effect size of the magnitude you expect to find and set your sample size and trial length appropriately. If you don't find an effect or a different sized effect with a well-powered study, well, that's basically publishable these days. Especially if you pre-registered it!

Besides that, there is rarely a downside to gathering more data, provided your statistical methodology is appropriate to your research question and you know how to interpret it correctly. The more observations you make, the more confident in your increasingly narrow inference you can be. If performance declines substantially, it will show up in the data.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.