I need to perform a questionnaire as part of my final project.
What are the quick-and-dirty rules for knowing how many respondents I need to get meaningful results?
$\begingroup$
$\endgroup$
3
-
1$\begingroup$ At its heart this is a good question. It's tough to answer without knowing what your metric for "meaningful" is. A starting point would be Cohen's d, but I don't know how accepted that is in the survey community. $\endgroup$ – Chuck Sherrington Aug 24 '14 at 8:05
-
$\begingroup$ What is "Cohen's D"? A book, a resource...? $\endgroup$ – Teusz Aug 24 '14 at 13:54
-
$\begingroup$ en.wikipedia.org/wiki/Effect_size#Cohen.27s_d $\endgroup$ – Chuck Sherrington Aug 25 '14 at 3:02
Add a comment
|
$\begingroup$
$\endgroup$
2
Obviously, this question is highly under-specified. The sample size you need depends a lot on the aims of your analysis.
In general, when thinking about sample size requirements, you need to think about power analysis and desired precision of estimation. This in turn requires you to think about your research question and expectations about results (e.g., effect sizes, expected means, and so on).
That said, you asked about a few simple rules of thumbs for questionnaire style research.
These would be my rough rules of thumb:
If you are wanting to do a proper survey study of the population and are aiming to get good estimates of proportions, then I'd think you'd want at least 1,000 participants in order to get reasonable standard errors. But if you are doing this kind of study, you are going to want to know a lot more about sampling than this question implies.
More likely, you are thinking about rules of thumb for a standard observational psychology study. In these kinds of studies, you are typically measuring several psychology scales perhaps along with some demographics. A typical analysis might involve correlations, regression-type models, and perhaps some reliability analysis and factor analysis to check the scales. In these kinds of studies, I generally think of a sample size of 100 as a reasonable starting point, 200 as good, and 300+ as very good, and of course even bigger samples are better. These rules of thumb are related to the precision with which you are estimating correlations which in turn relate to the precision of predictive models and so on.
For more discussion of rules of thumb, see this question on rules of thumb for regression.
That said, the above are just rough rules of thumb. There are trade-offs between required resources and sample size. More importantly, if you are serious about these things you should analyse your problem in terms of either power analysis or desired precision.
answered Aug 10 '14 at 3:18
-
$\begingroup$ THanks, Jeromy. You wrote "The sample size you need depends a lot on the aims of your analysis." -- what information about the analysis would help me determine correct sample size? $\endgroup$ – Teusz Aug 10 '14 at 11:05
-
2$\begingroup$ It depends on your research question, study design, planned analysis, alpha level or desired precision, and effect sizes that are either expected or of interest. $\endgroup$ – Jeromy Anglim♦ Aug 14 '14 at 5:56
