How many samples are there when test subjects come through different recruitment strategies?

Question

Let's assume I want to conduct a survey. To find test subjects, I post an ad in a local newspaper, one on a locally relevant website, distribute flyers in front of the train station, and have managed to get one complete class from a local school to partake.

Due to the different recruitment methods, each will probably have drawn quite different samples from the population: the local newspaper is mostly read by older people with medium to high education; the institute website is mostly visited by students and researchers; the flyer is given to commuters; then there is that group of pupils, all of the same age and education level.

In my test, I ask my test subjects to specify how they knew of the test:

1. ad in newspaper
2. ad on website
3. flyer
4. they were tested in class

So I can group my data according to the recruitment method, if I want to.

Is this all one sample? Or have I drawn four samples?

And if this is one sample, how then do you draw two or more samples?

Answer 1

This depends on your research question, as well as the population you want to generalize your conclusions to.

If you care about whether complex sounds are easier to discriminate when they are longer, you can regard all the participants to be one sample because there is no theoretical reason to assume that people differ in this. In principle, the more universal the subject of your study is, the less you need to care about your sampling technique because any person should be as good as any other. In practice, the sample will likely be first year psychology students :)

But if you care about computer literacy or car ownership or attitudes towards standardized testing (i.e. if your question relates to individual differences or group differences), then clearly your samples are drawn from very different parts of the population. In that case you can say that you have drawn four samples. Or perhaps less - perhaps you drew two: people unlikely to have cars (school kids and train takers) and people likely to have cars (American newspaper readers and internet users). This is not the best example but you can see the logic.