In experimental psychology and cognitive neuroscience today, with the exception of a few specific subdisciplines, the consensus seems to be that an experimental study has to be performed on a group of subjects which are considered as a sample from an underlying population. Statistics is then applied in such a way as to allow an inference on properties of the distribution in that population of a certain psychological function or feature, most often the population mean.

My question is: When was this consensus established, with which reasoning, and on whose initiative?

In my experience, current textbooks on statistics in psychology discuss the statistical procedures in great detail, but contain only a few cursory remarks on the psychological (not statistical) motivation of these procedures. They do not discuss why a cognitive scientist should be interested in a population-average reaction time, a population-average event-related potential etc. – instead of e.g. effects that are present in every single individual in the population, or a specific sub-population. Or maybe I'm just looking at the wrong books?

  • $\begingroup$ I think I can provide an answer, but I'm trying to understand what the difference is between investigating a population-average response time and 'effects that are present in every single individual in the population' $\endgroup$ Aug 26 '14 at 13:28
  • 1
    $\begingroup$ @Josh: I think the difference is between effects that vary across individuals (hence are distributed in ways that make characteristics such as a mean and SD useful) and effects that do not (i.e., are invariant and universal / human nature), or at most vary nominally (i.e., differ in ways that can be described with discrete, unordered categories; e.g., sex, or maybe species). $\endgroup$ Aug 26 '14 at 16:41
  • $\begingroup$ @Josh, assuming a symmetrical distribution, if the population average is larger in one condition than in another condition, this merely means that the response is larger in more than 50% of the individuals, and smaller in less than 50% of the individuals. If the effect was present in every single individual, this would mean that the response is larger in 100% of the individuals. $\endgroup$
    – A. Donda
    Aug 26 '14 at 18:52
  • $\begingroup$ But the question is not about this specific alternative goal, since many different alternatives are conceivable, but just who decided when that it is specifically the population mean that we are interested in. If you can provide an answer, please do so. $\endgroup$
    – A. Donda
    Aug 26 '14 at 18:53
  • 3
    $\begingroup$ Preaching to the choir :) but a lot of the (false) consensus probably built up around traditional practices that were "established" when both psychology and statistics were young sciences, borrowing heavily from biological research methods I'm sure. This isn't meant to be an answer, but I worry that a good one will be elusive. So much of statistical practice is a matter of finding a useful approximation of the phenomenon of interest; even basic improvements in these models like choosing to focus on the median instead of the mean seem to occur gradually and without consensus. $\endgroup$ Aug 26 '14 at 19:18

What you are actually asking about is the debate surrounding the question:

Can psychological quantities be measured?

Up until about 1800 psychological questions where discussed by philosophers. A separate psychological discipline did not yet exist. Answers to questions relating to perception, emotion and cognition where attempted on the basis of religious beliefs, philosophical tradition and unsystematic observation and introspection.

Over the course of the eighteenth century, the body-mind dualism was slowly being resolved, which allowed first physiologists and then physicists over the course of the nineteens century to begin to apply their methodology of experimenting with plants and animals and mathematical principles, as they had been developed within the natural sciences, to psychological questions. In the middle of the nineteenth century, the first psychological institutes were founded, the first by Wundt (in Leipzig, Germany), and the discipline of psychology came into existence.

Looking at the history of psychology, you will realize that while psychological questions are as old as humanity, the academic discipline of psychology came into existence when scholars began to measure, i.e. quantify, the psyche. This was in accordance with the rise of the natural sciences during the Enlightenment and their dominance over our whole worldview (almost?) until today.

In his classice monography, Psychometric Methods (1936), J. P. Guilford writes:

The progress and maturity of a science are often judged by the extent to which it has succeeded in the use of mathematics. The “psychometric methods” are procedures for psychological measurement. Measurement means the description of data in terms of numbers and this, in turn, means taking advantage of the many benefits that operations with and mathematical thinking provide.

Thus, while psychological questions have traditionally been considered in the humanities and arts, the academic discipline of psychology has been created and finds its home within the natural sciences and shares their basic approach of quantitative measurement and statistical analysis. Philosophical and artistic "psychology" have continued to exist parallel to institutionalized academic psychology and have given rise to psychoanalysis (Freud), humanistic psychology (Rogers) or critical psychology (Holzkamp), among others.

The principles and standards of psychological experimentation and statistical analysis as we know them today where not discovered instantaneously but developed and refined until the middle of the twentieth century. In 1936 Guilford found that the natural sciences lacked a common theory of measurement:

Measurement in the physical sciences has come so naturally that in that connection little thought has had to be given to what measurement really is. There are some physical scientists who maintain that what is called measurement in psychology is not measurement at all. It is true that the term measurement is sometimes defined in such a way that it does not apply to most of the operations in psychology commonly known as measurement. Definition of an abstract term such as measurement is an arbitrary matter, however, and psychologists will either define it to cover what they are doing under that concept or they will invent a new term for what they are doing with numbers and arithmetical operations. Fortunately, measurement can be defined sufficiently broadly to include the operations known as psychological measurement.

Four years before this statement, in 1932, the British Association for the Advancement of Science had established a committee of scientists that should consider the problem of measurement in the empricial disciplines. The committee was especially asked to resolve the question, whether a measurement of sensation was possible. The answer to this question would decide wether psychology was a natural science coequal to physics or chemistry. Psychologist S. S. Stevens (1946) writes about the committee's discussions:

For seven years a committee of the British Association for the Advancement of Science debated the problem of measurement. Appointed in 1932 to represent Section A (Mathematical and Physical Sciences) and Section J (Psychology), the committee was instructed to consider and report upon the possibility of “quantitative estimates of sensory events” — meaning simply: Is it possible to measure human sensation? Deliberation led only to disagreement, mainly about what is meant by the term measurement. An interim report in 1938 found one member complaining that his colleagues “came out by that same door as they went in,” and in order to have another try at agreement, the committee begged to be continued for another year. For its final report (1940) the committee chose a common bone for its contentions, directing its arguments at a concrete example of a sensory scale. This was the Sone scale of loudness (S.S. Stevens and H. Davis, Hearing. New York: Wiley, 1938), which purports to measure the subjective magnitude of an auditory sensation against a scale having the formal properties of other basic scales, such as those used to measure length and weight. Again the 19 members of the committee came out by the routes they entered, and their views ranged widely between two extremes. One member submitted “that any law purporting to express a quantitative relation between sensation intensity and stimulus intensity is not merely false but is in fact meaningless unless and until a meaning can be given to the concept of addition as applied to sensations.”

So, while the committee could not agree on the question wether sensation can be measured, they made it clear that an empirical equivalent to arithmetic addition would constitute measurement. This refers to the foundation of the measurement of physical quantities by Hermann von Helmholtz (1887) and defined axiomatically by Hölder (1901).

I don't want to go into the details of this argument, because it goes beyond the scope of this question. In short, Helmholtz sought to base measurement on the empirical comparison of so-called "extensive quantities", for example physical length or mass. He correlated arithmetic addition to physical combination (e.g. lineal apposition of staves to measure lengths) and the equality of two numbers to a method of "physical comparison", for example the comparison of weights on a beam balance).

How these principles of extensive measurment can be applied in psychology, and if it should, has been heavily debated, and different "test theories" have been developed, such as classical test theory, latent trait theory, or the theory of knowledge spaces. Foundational texts for a representational theory of measurement are those by Krantz, Luce, Suppes, and Tversky (1971, 1989, 1990) and Robers (1979), if you are interested in this aspect. All books feature some general introduction but require mathematical understanding to follow.

This requirement of a broad and deep understanding of mathematics is the reason why all psychology looks the same to you.

Most currently publishing psychologists have only a superficial understanding of methodology. Gigerenzer, Krauss and Vitouch (2004) explain some of the methodological shortcomings and mistakes:

One of us once had a student who ran an experiment for his thesis. Let us call him Pogo. Pogo had an experimental group and a control group and found that the means of both groups were exactly the same. He believed it would be unscientific to simply state this result; he was anxious to do a significance test. The result of the test was that the two means did not differ significantly, which Pogo reported in his thesis.

In 1962, Jacob Cohen reported that the experiments published in a major psychology journal had, on average, only a 50 : 50 chance of detecting a medium-sized effect if there was one. That is, the statistical power was as low as 50%. This result was widely cited, but did it change researchers’ practice? Sedlmeier and Gigerenzer (1989) checked the studies in the same journal, 24 years later, a time period that should allow for change. Yet only 2 out of 64 researchers mentioned power, and it was never estimated. Unnoticed, the average power had decreased (researchers now used alpha adjustment, which shrinks power). Thus, if there had been an effect of a medium size, the researchers would have had a better chance of finding it by throwing a coin rather than conducting their experiments. When we checked the years 2000 to 2002, with some 220 empirical articles, we finally found 9 researchers who computed the power of their tests. Forty years after Cohen, there is a first sign of change.

Editors of major journals such as A. W. Melton (1962) made null hypothesis testing a neces- sary condition for the acceptance of papers and made small p-values the hallmark of excellent experimentation. The Skinnerians found themselves forced to start a new journal, the Journal of the Experimental Analysis of Behavior, to publish their kind of experiments (Skinner, 1984, p. 138). Similarly, one reason for launching the Journal of Mathematical Psychology was to escape the edi- tors’ pressure to routinely perform null hypothesis testing. One of its founders, R. D. Luce (1988), called this practice a “wrongheaded view about what constituted scientific progress” and “mind- less hypothesis testing in lieu of doing good research: measuring effects, constructing substantive theories of some depth, and developing probability models and statistical procedures suited to these theories” (p. 582).

The student, the researchers, and the editors had engaged in a statistical ritual rather than sta- tistical thinking. Pogo believed that one always ought to perform a null hypothesis test, without exception. The researchers did not notice how small their statistical power was, nor did they seem to care: Power is not part of the null ritual that dominates experimental psychology.

The articel runs on and explains what the p-value can and cannot do (read it, if you are a psychologist). I only quoted the beginning in length to show you that a methodologically savvy minority among psychologists believes that the majority of scholars publishing in their discipline does not know what they do. They follow a "ritual" of blindly applying the same methods to all problems, thus hampering the progress of the discipline and reducing the value of their findings.

Nevertheless, while widely ignored, progress has been made (see my brief remarks about different theories of measurement above).

If you want to learn more about the development of psychology as a natural science, grab any book on the history of psychology from a nearby university library. Make sure you pick one that starts at least in the nineteenth century, but for a better understanding it is necessary to look back until Leibnitz and his contemporaries.

If you want to learn more about measurement in psychology, books such as Joel Michell's Measurement in Psychology: A Critical History of a Methodological Concept provide a good introduction. I have not read any of those books and cannot recommend one. My own knowledge (and the summary given here) stems from a couse in psychometrics I took while studying psychology.

To summarize, academic psychology uses experimentation and statistical analysis because it is a natural science and adheres to the basic principles that define these disciplines. The belief that the universe is probabilistic (and events are distributed and these distributions can be described through arithmetic means and such) rules most of the natural sciences today and is not unique to psychology.

Beyond this unifying foundation, the different fields of psychology have quite different goals and methods. For example, general psychology (including the psychology of perception) is not interested in the distribution of a trait (such as intelligence) across a population, but in understanding, for example, the principles that govern learning in all individuals.

It is not true that experiments have to be performed on a group of subjects. In the psychology of perception, for example, which does not deal with interspecies differences but with the basic principles of human psychology, it is common to use only one single subject, because it is assumed that certain aspects of perception, such as trichromatic vision, are the same in all members of the population and can be investigated in any (healthy) individual. But of course this one test subject has to perform multiple trials, which are then analysed statistically.

Statistical analysis and the application of mathematics to reality itself goes back to ancient times. Bakker (2003) finds predecessors of contemporary statistical concepts as far back as ancient Indian texts:

In an ancient Indian story, Rtuparna estimated the number of leaves and fruit on two great branches of a spreading tree (Hacking 1975). He estimated the number on the basis of one single twig, which he multiplied by the estimated number of twigs on the branches and found a number, which after a night of counting turned out to be very close to the real number.

Aristotle, in the Nichomachean Ethics calculated the mean as "a point equally distant from either extreme", and this was developed further by Ticho Brahe and others until Legendre introduced the least squares method, that we use today, in 1805.

If you want to know about the history of statistics, that is beyond the scope of this site.

The answer to your question in detail:

  1. Why group level statistics?

    Because there is no other way of measurement. As a physicist you know that every measurement is a sample from a population. If, for example, you measure weight, each measurement of the same weight will be different, if your scale is sensitive enough. You will then calculate the distribution and its characteristics (mean, slope, range etc.) to describe that distribution. If you do only one measurement, then that is because the differences are negligible for your purpose (e.g. human body weight) or measurement is too expensive and time consuming.

  2. Since when group level statistics?

    The ancient Indian example shows that group level statistics has been done since the dawn of time: a representative sample of leaves where drawn from a tree to understand the whole population of leaves. Of course that Indian "mathematician's" estimations where primitive, since the relevant mathematics hadn't been invented yet. The arithmetic mean was defined as late as 1805. But the basic idea and principles are identical to ours: estimate the properties of a population from a representative sample.

  3. Why group level statistics in psychology?

    The reason is not a psychological but a methodological one, and it is given in the quote by Guilford: It is the dominating belief of our culture that a true science must employ mathematics. So as soon as psychology was created as a separate scientific discipline, it had to use statistics. Otherwise it would have remained an area of speculation, in the eyes of the scientific community.

  4. Since when did psychology employ group level statistics?

    Building on the groundbreaking work of Ernst Heinrich Weber, Gustav Fechner defined the absolute threshold as the statistical mean of all threshold values in the method of limits in his work Elemente der Psychophysik (1860). Fechner also developed the notion of the median (in 1878).

    But my answer describes that the use of mathematical methods in psychology was not so much a singular event that affected the rest of the science like an explosion, but a gradual process that happened as psychology was slowly detached from philosophy and grew into a natural science from around 1800 to around 1950, when the process was largely finished (with Stevens' publication).

    And again, you must understand that the theory of measurement – and even statistics itself! – was in development in all of the natural sciences at the same time. For example, the arithmetic mean was defined in 1805, Legendre published the least squares method in the same year, and Gauss defined the normal distribution and maximum-likelihood estimation in 1809. This ongoing development was why the British committee on measurement was set up in 1932: because no satisfying theory of measurement had been created until then!

  5. Why the popularity of the mean?

    Because most psychologists don't understand statistics well enough to do more than that.

  6. Why population average instead of effects that are present in every single individual in the population, or a specific sub-population?

    You obviously did not read a lot of psychological texts. The field of general psychology focusses on effects that are present in every single individual and often does experiments of one proband. Only the field of differential psychology explores the distribution of traits among the population. Other fields look at sub-populations (e.g. children or old people, psychiatric patients, workers of a certain occupation, and so on).

    In the history of psychology, the first studies (all through the 1800s) explored general psychology: perception, cognitition, memory. Beginning around 1900, differential psychology began first with the measurement of intelligence, and then the study of medical psychology in psychiatric patients. Around the First World War occupational psychology began to study work behaviour and pedagogical psychology began to apply the findings to the training of future workers and soldiers. Workers and soldiers are groups, so there is also an economic and political reason for group-focussed psychology. Advertising and Human Resources are still today the largest areas of psychology beside clinical psychology, which partially explains the continuing focus on population based studies.

Finally, of course there are scholars that employ a more refined statistics and do not focus on the mean so much, and there are scholars that believe there are aspects of human psychology that are not covered by a statistical approach. If you are interested in them, go deeper into psychology or create your own alternative methodology that represents what you believe about the human psyche.

All of this is of course my personal opinion. I am no historian of psychology and do not have the overview to judge the correctness of my interpretation. I used the small sample of publications that my professor for methodology provided and that I hope is representative. From this sample, I estimated the history of psychology, and my estimation may, of course, be wrong.


  • Bakker, A. (2003). The early history of average values and implications for education. Journal of Statistics Education, 11(1), 17-26.
  • Fechner, G. Th. (1860). Elemente der Psychophysik [Elements of Psychophysics]. Leipzig: Breitkopf und Härtel.
  • Gigerenzer, G., Krauss, S., & Vitouch, O. (2004). The null ritual. The Sage book of quantitative methodology for the social sciences, 391-408.
  • Guilford, J. P. (1936). Psychometric Methods. New York: McGraw-Hill.
  • Helmholtz, H. (1887). Zählen und Messen erkenntnistheoretisch betrachtet. In Philosophische Aufsätze Eduard Zeller zu seinem fünfzigjährigen Doctor-Jubiläum gewidmet. Leipzig: Fues.
  • Hölder, O. (1901). Die Axiome der Quantität und die Lehre vom Mass". Berichte über die Verhandlungen der Koeniglich Sächsischen Gesellschaft der Wissenschaften zu Leipzig, Mathematisch-Physikaliche Klasse, 53, 1–46.
  • Krantz, D.H., Luce, R.D., Suppes, P. & Tversky, A. (1971). Foundation of measurement. Vol. I. Additive and polynomial representations. New York: Academic Press.
  • Luce, R.D., Krantz, D.H., Suppes, P. & Tversky, A. (1990). Foundation of measurement. Vol. III. Representation, axiomatization, and invariance. New York: Academic Press.
  • Roberts, F.S. (1979). Measurement theory with applications to decisionmaking, utility, and the social sciences. Reading: Addison-Wesley.
  • Stevens, S. S. (1946). On the theory of scales of measurement. Science, 103, 667-680.
  • Suppes, P., Krantz, D.H., Luce, R.D. & Tversky, A. (1989). Foundation of measurement. Vol. II. Geometrical, threshold, and probabilistic representations. New York: Academic Press.
  • $\begingroup$ Dear what, thanks a lot for your very detailed and clear answer. There are a lot of interesting pointers here, I'll be going through all of them, and I'm sure I'll learn plenty. However, I'm afraid that you missed the focus of my question. I did not ask why psychologists measure, I assumed tacitly that of course they do. The reason may be that my background is in physics and my work is in cognitive neuroscience (EEG and fMRI), so my work is all about measured data. I also didn't ask why psychologists use statistics; again, that should be obvious to anyone who has ever seen psychophysiological $\endgroup$
    – A. Donda
    Aug 31 '14 at 12:40
  • $\begingroup$ or even behavioral data. My question is specifically: Since when and with what reasoning do psychologists and cognitive scientists do group-level statistics. As you write, there are subdisciplines who focus on one or a few subjects, and I acknowledged this fact in my question. However, unless I'm mistaken, the majority of psychological disciplines collects data and applies statistics to infer something about the distribution of an effect in a population of subjects. My secondary question is: Why are psychologists and cognitive scientists primarily interested in the population average? $\endgroup$
    – A. Donda
    Aug 31 '14 at 12:46
  • $\begingroup$ See my edit, which I hope gives my answer more clearly. $\endgroup$
    – user3116
    Sep 1 '14 at 7:03
  • $\begingroup$ again, thanks a lot, but again, I think I didn't really get my point through. By "group-level" statistics I mean statistics done across several subjects. The complement is "subject-level" statistics, where one does statistics across a set of trials performed by one subject. If there are several trials from several subjects, one can apply a hierarchical, two-level model, which often can be reduced to entering subject-level summary statistics into group-level statistics. In event-related potentials research, people speak of single-subject averages that are then combined into $\endgroup$
    – A. Donda
    Sep 1 '14 at 13:00
  • $\begingroup$ a "grand-average", what I would call a population-average, where "population" refers to a population of humans. In this sense – correct me if I'm wrong – Fechner did not do group-level statistics; I believe he mainly experimented on himself as the single subject of his studies? $\endgroup$
    – A. Donda
    Sep 1 '14 at 13:02

In simplest terms, they are searching for objective, observable truth. Wikipedia seems to have quite a few references to this, see the quote/link below.

In terms of the One-Factor-at-a-time, OFAT method was first recorded to have been used by James Lind.
According to Wikipedia:
"In 1747, while serving as surgeon on HM Bark Salisbury, James Lind carried out a controlled experiment to develop a cure for scurvy. In this study his subjects' cases "were as similar as I could have them", that is he provided strict entry requirements to reduce extraneous variation. The men were paired, which provided blocking. From a modern perspective, the main thing that is missing is randomized allocation of subjects to treatments.

Fast forward a hundred years or so, and:
"As is well know, statistic methods--and by "statistical methods" I mean probability-based modeling and inference--entered into psychology in 1860 in the work of Gustav Fechner (1860) in psychophysics (Boring 1961;Stigler 1986)."
A Historical View of Statistical Concepts in Psychology and Educational Research

American Journal of Education, 101, 1992.

  • $\begingroup$ Hello Cbaker510, thanks for your answer. However it does not address my question: Why group statistics (instead of e.g. single subject examinations), why population average (instead of other distributional properties). The Stigler pointer seems to be correct though, I justed picked up a copy of his book on The History of Statistics before 1900, which includes a discussion of how psychology and social sciences were not only fields of application of statistics but also involved in the development of the field. $\endgroup$
    – A. Donda
    Aug 31 '14 at 12:54
  • 1
    $\begingroup$ Try looking at page 66-67 of the AJE article. $\endgroup$
    – Cbaker510
    Aug 31 '14 at 18:56
  • $\begingroup$ Yes, it seems like Quetelet has introduced the idea of an "average man" – not into psychology though but into the social sciences; apparently opposed by the founder of sociology, Comte. I'm currently reading up in Stigler's monograph if and how this line traces from Quetelet into psychology. $\endgroup$
    – A. Donda
    Sep 1 '14 at 13:17

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.