How do people perceive subjective scales?

Question

Let's take a picture of apples:

Are there experiments conducted that study the distribution of the following answers to the question "How many apples are on this picture?":

• Very little
• Little
• Normal
• Many
• Lots of

Well, of course, this specific picture doesn't matter, but this type of questions is important for many applications, like in surveys, when people rate things on subjective scales, so that in particular situations we can compare neither across questions with the same scale nor across persons answered the same question.

For example, think of 360-degree surveys when a person rates himself and is rated by others, so to compare in the end the two ratings.

Answer 1

If I understand you correctly, you are trying to learn about good practice in designing studies involving subjective scales? Then getting actual answers to such surveys might not be very helpful. You might rather want to know the impact of your survey design and the answering-environment on the answers of the respondent.

These are broad questions. Neither do I have the expertise nor the space to give you comprehensive answers here. Some suggestions though. As far as study design is concerned, you may want to consult Survey Methodology. In particular, section 5.3.5 deals with the implications of the type of scales used as possible answers. In fact, any other Survey Methodology textbook should cover this topic.

A lot of experiments also study the impact of social environment on perceptions and the construction of subjective opinions. Among others, Solomon Asch is a pioneer of experimental investigations of subjective perception. His most famous experiment is related to the conformity effect. He shows that people tend to conform to the perception of the group they are put in, even when this goes against strong visual evidences, see http://www.youtube.com/watch?v=FnT2FcuZaYI.

In another famous experiment, Asch studied the "recency" and the "primacy" effect. The subject of the experiment where enumerated a list of personality traits (generous, peaceful, wise, mean,...) and subsequently asked to described the person. Asch question was whether, in describing the fictitious individual, people would rely more heavily on the first enumerated traits (the "primacy" effect), or would rather focus on the last ones they where presented (the "recency" effect). Asch results tend to show a dominance of the "primacy" effect.

Again, it would be too long to present a list of the numerous experiments and theoretical attempt to grasp perceptions mechanisms here. The literature is broad but the topic is classical. If you read french, it is notably covered in chapter two of "Psychologie Sociale" by Jacques-Philippe Leyens and Vincent Yzerbyt, but it should be covered in any good (social) psychology textbook. Unless you have a more specific question, this might be the right way to start learning about it.

Answer 2

Well, of course, this specific picture doesn't matter, but this type of questions is important for many applications, like in surveys, when people rate things on subjective scales, so that in particular situations we can compare neither across questions with the same scale nor across persons answered the same question.

But we can, and do. Note that the point is not really to assess how many apples there are based on subjective answers. Usually the point is to see whether the type of people who say that there are many apples are also the type that will say something else (e.g. that the sky is blue). You are just looking for a relationship in the way a person answers different types of questions, and whether there are some general patterns that people follow when doing this. Then you go on to conclude something very general (and bold) about people, for example that they can be divided into optimists and pessimists.

Usually tests like this explain very little variance in actual behaviour, so there is a lot of measurement noise - and a large part of it stems from subjectivity. But still there may be some patterns underneath all this noise, and those are of interest to psychologists.

Note also that you would probably not get just one question aiming to measure one construct. If the construct is ability to discern amounts of apples, you would get a series of questions with pictures. In this case, you would go through a process of scaling based on the pictures you saw: you'd say there are many apples on the pictures that show the most apples in the series, and few on pictures that show the fewest. So your subjective inner scale adjusts to the scale presented to you. This takes away some measurement noise.

Answer 3

Schwarz (1999) reviews a variety of research on frequency estimation scales. That's not exactly the same kind of scale, but both request estimation of a count variable by choice among ordered, polytomized response options. Spector (1976, 1980, 1992) may also have something useful to say on this topic: he researched the effect of providing even vs. uneven interval categories for rating scales of frequency. Meanwhile, Moskowitz and a different Schwarz (1982) have compared the reliability and validity of behavior counts and ratings in personality research.

Getting somewhat off-topic, culture may be important to consider for Likert scale response interpretation in general (see for example Lee, Jones, Mineyama, & Zhang, 2002).

Reference

Lee, J. W., Jones, P. S., Mineyama, Y., & Zhang, X. E. (2002). Cultural differences in responses to a Likert scale. Research in Nursing & Health, 25(4), 295–306.

Moskowitz, D. S., & Schwarz, J. C. (1982). Validity comparison of behavior counts and ratings by knowledgeable informants. Journal of Personality and Social Psychology, 42(3), 518–528.

Schwarz, N. (1999). Self-reports: How the questions shape the answers. American Psychologist, 54(2), 93–105. Available online, URL: https://edit.ethz.ch/oat/education/material/material_06_07/material_empirical_methods_06_07/American_Psychologist_Self-Reports.pdf.

Spector, P. E. (1976). Choosing response categories for summated rating scales. Journal of Applied Psychology, 61(3), 374–375.

Spector, P. E. (1980). Ratings of equal and unequal response choice intervals. The Journal of Social Psychology, 112(1), 115–119.

Spector, P. E. (Ed.). (1992). Summated rating scale construction: An introduction (No. 82). Sage.

Answer 4

I'd narrow this question: how people perceive different things in different situations (as well as set of things is narrowly specified). Because there is no simple answer - it is a cognitive function of the state in the states.

As for that picture (or similar), when I'm hungry I tend to see little apples (cause driven by the instincts I'm worring about me to be not satisfied) so when I had a king-size meal I'd say a lot when looking at that basket.