Meta-analysis is a tool that is applied across a wide range of disciplines, especially those involving human subjects. In general, meta-analyses aim to sample all relevant studies that have been conducted.
In some fields, the norm is to use undergraduate students (especially psychology students). Many psychology programs have a research participation pool; so for a lot of basic lab based research (e.g., cognitive psychology, social psychology), this represents a convenient sample. Note of course, that other disciplines will use other samples. Organisational psychology researchers often use samples of employees from organisations. Developmental psychologists study children. Social science researchers often rely on large scale panel surveys.
More recently, online samples are becoming particularly popular. This includes online convenience samples as well as paid samples. In particular, Mechanical Turk is particularly popular in the United States, but other platforms such as prolific exist.
In general, the choices that researchers make about sample are driven by many factors:
- convenience/resources
- discipline expectations
- interests in generalisation (some researchers are mainly interested in generalising to their own culture)
- where the researchers have traditionally come are from (e.g., most researchers in psychology come from the U.S., followed by UK/Europe/Australasia)
- relevance of sample to research question (e.g., sometimes a convenience sample is fine; other times you really need a representative sample); in basic science, it's often fine, but if you're doing organisational or clinical research, you'll generally want a relevant org or clinical sample.
So in short, this phenomena is going to vary a lot across research questions. And in some cases it's probably not going to matter much; in other cases it will.
More generally, there is often a trade-off between convenience and rigour. For example, most researchers who measure job performance would love to have a rigorous objective measure of objective performance, but it's much easier to get self-report. So you'll often find that meta-analyses will be summarising studies that have opted for one point on the convenience/rigour trade-off.
Meta-analysts can and do perform moderator analyses if they think a particular factor might be relevant to the meta-analytic finding (e.g., whether the sample are undergraduate students or not).
