0
$\begingroup$

I'm currently designing an experiment with several conditions - how do I decide how many repetitions / stimuli within each condition?

I know that I can justify sample size with an a priori power analysis...is there any common way to choose how many trials within each of my conditions?

I was using sample size/power as an example of something where there is a particular approach to help decide.....what I am actually asking about is the number of trials i.e. if I have three conditions in my experiment, how many stimuli should I design for each of the conditions?

Improve this question
$\endgroup$
4
  • $\begingroup$ If you don't want do any calculations you could look at similar researches and see how many trials they have used. For ways of calculating effect size/power or the like, you may want to check Cross Validated, the statistics SE $\endgroup$
    – Robin Kramer-ten Have
    Mar 27, 2017 at 16:05
  • $\begingroup$ Related: cogsci.stackexchange.com/q/3384/11318 $\endgroup$
    – Robin Kramer-ten Have
    Mar 29, 2017 at 7:07
  • $\begingroup$ Obviously, the choice of sample size is a compromise between the results of the power analysis (larger sample size = higher power) and the costs of the larger sample (larger sample = higher cost). It's not clear, what kind of answer are You @LennaKB looking for? Are you looking for a formal decision model that will tell You what sample size to choose given a specific cost function? Does this question ask what a power analysis would suggest? Are there some aspects of the current scenario (eg the hierarchical structure of the exp.) that make standard solutions from power analysis not applicable? $\endgroup$
    – matus
    Mar 29, 2017 at 15:42
  • $\begingroup$ I am asking about trials rather than sample size (just used sample size as an example of something that you can use a certain approach to decide). $\endgroup$
    – LennaKB
    Mar 30, 2017 at 17:01

1 Answer 1

Reset to default
1
$\begingroup$

A power analysis is typically based on the difference in the mean to be detected and the expected standard deviation of the data. The amount of variation in the data (i.e., the size of the standard deviation) typically depends on individual differences in the subjects and measurement noise. The measurement noise depends on a number of things, but in many experiments can be reduced by running more trials. In the minimal risk experiments I am most familiar with, we run so many trials that we reduce the trial related noise to inconsequential amounts. We tend to use wildly inefficient methods for collecting the trials also (our stimuli and response interval are too long, we use multiple intervals, and our adaptive procedures are not optimized).

In order to minimize the data collection/maximize power (and this should not be the only goal) the stopping rule generally cannot be a number of trials, but rather a precision criteria. There are procedures (e.g., ZEST) which can be set up to use a fixed number of trials or a target precision.

Improve this answer
$\endgroup$

Not the answer you're looking for? Browse other questions tagged or ask your own question.