Does work exist on relative valuations versus numeric valuations?

Question

I've been trying to use Google Scholar to find something on this issue, but I fear I may not have found anything due to using incorrect terms. I am looking for any work which looks at rating systems and people's use of them (I hope this is the right forum). We are all familiar with the idea of rating things numerically, like out of a scale of 5.

But it is also possible for people to rate things relatively ie A > B and B > C. Other preferences can then be inferred, ie A > C.

In certain instances, where people may be reluctant to provide low ratings, these relative valuations may therefore be useful. I am wondering if this is something which has been looked at in any work that anyone knows of? Is 'relative valuation' the correct term for this?

Answer 1

Short answer
Sensory systems in general operate on a relative basis. Hence, seemingly objective rating scales are often still relative, because subjects implicitly, or explicitly compare previous trials to the current one. Perception is per definition always relative. So numeric ratings are similar to ''larger than'', or ''smaller than'' ratings. The advatnage of numeric ratings is that it is a (semi-)continuous scale, allowing for quantitative analyses.

Background
You are asking about rating scales and specifically to the difference between numbering opposed to larger-than / smaller-than ratings.

You have to realize that when subjects rate on a scale from, e.g., 0 to 10, the S is already quite explicitly indicating whether a trial's stimulus is larger or smaller than a previous trial. Even if you force an absolute number scale, the result is a comparison nonetheless.

Moreover, even if you as an experimenter force an absolute rating, Ss will always start comparing later conditions to previous ones (this is larger/bigger/brighter/more intense than the last trial, etc.), either consciously or subconsciously. The only really absolute one, in absence of a baseline measure, will be the very first trial. It is a known phenomenon that previous trials affect subsequent ratings because of that. Hence, true absolute ratings are a rarity.

Also - perception is physiologically and per definition always relative. Sensory systems operate at relative measures at the neural level.

And to conclude, as a comment - What often happens in rating experiments is that the experimenter assigns a certain baseline. For example, consider the rating of speech-understanding. First, the listener can be instructed to rate speech in silence from 0 (unintelligible) to 10 (highly intelligible). You might urge the S to rate it 10, as speech in silence is per definition highly recognizable and understandable. After that, you present various distorted speech forms, or speech in variable amounts of noise. Under these conditions, Ss will (likely) rate these trials lower, because of lower signal-to-noise ratios. This way, the absolute ratings are relative less than/larger than comparisons nonetheless, because Ss are rating the subsequent trials relative to baseline. i.e., they are comparing the noisy conditions to 10 - per definition a relative measure, whether you are asking them to give numbers, or larger than, or smaller than answers. The big advantage of numbers is that it allows for (semi-)quantitative analyses, while '<', '=' or '>' will make your life a lot harder to do stats.