I am concerned that when Likert-scale type questions are used in self-evaluation questions, the results are ambiguous and would like to know if it's possible to address this ambiguity.

Interpersonal Reactivity Index

Although I have seen this in multiple questionnaires, in this question, I would like to focus on it's use in the Interpersonal Reactivity Index (IRI), which is used to measure empathy. The index is a set of questions that must be answered according to a 5-point Likert-scale ranging from "Does Not Describe Me Well" to "Describes Very Well".

Ambiguity Concerns

Consider this question from the IRI:

  1. I daydream and fantasize, with some regularity, about things that might happen to me.

In this case, my concern is the temporal descriptor "with some regularity". How is someone filling this survey supposed to determine what the population mean of "regular" means? Is it once a week? Once a month?

I have similar concerns about the following question:

  1. I often have tender, concerned feelings for people less fortunate than me.

What is "often"? What depth/duration do the "tender, concerned feelings" require to qualify for this question?

How to Address Ambiguity

How is this ambiguity of language, in regards to personal assessment, addressed in these types of surveys? It seems like a better designed survey would first try to establish the person's view of the general population in terms of range of feelings it's trying to describe, as well as the duration and frequency. Consequently, the person's assessment of the general population could be taken into account for their own personal assessment.

Have concerns similar to this been addressed before in other contexts?

  • $\begingroup$ Yes, these questions are less than precise. Question 2 almost looks double barreled to me. Consequences of less precise questions are increases in error rates (e.g., the variation being captured is not solely from the underlying construct, but also from individual interpretations of the question). For example, #2 you respond no for tender (but not concerned), while I respond yes. Extra, unaccounted for variation. $\endgroup$ – mflo-ByeSE Jul 19 '17 at 17:38
  • $\begingroup$ Ugh and just glanced at 1 again: definitely double barreled. I'll add that your concerns about the time are also a big deal (no less than the ambiguity). For me, much better items may say: "how often do you daydream about..." or "... fantasize about..." (then the response is the frequency, not the agreement). $\endgroup$ – mflo-ByeSE Jul 19 '17 at 17:38

I believe that in the first part there is some confusion of empathy and self-concept ("Does Not Describe Me Well" to "Describes Very Well"), although the basic statement refers to empathy that question is made to measure self-concept of empathy. The vast majority of tests (which do not measure performance) measure self-concept dimensions. Of course you can not measure empathy by a test, what you measure are dimensions of self-concept in relation to empathy. Empathy could be measured through behavioral registers (see attachment studies).

With regard to the second part:

  • Ideally the test should not be ambiguous in any item (although it is common for people to comment in this regard).
  • Given the impossibility in most of the tests to request frequencies of behavior are usually asked similar questions.
  • It must be taken into account that these questions are made in test format likert, it would be a serious mistake to ask similar questions in dichotomous format.
  • If there is ambiguity in the test this has to be reflected psychometrically as a low correlation and error variance, provided that questions are asked that correlate with the reference question, that is to say, that usually several questions of this type are made and their Relationships when designing the test (you can also study it on your own for a class work).
  • The administration of the test must be performed under conditions that allow questions to be asked and the professional (or student who must know the test in detail) should clarify any doubts to the person to whom the test is administered, therefore in some countries, someone who has not studied psychology can not administer or correct (legally or validly) a psychological test.

Personally, when I require empathy measurements I always use the TECA (acronym in Spanish) https://web.teaediciones.com/TECA--TEST-DE-EMPATIA-COGNITIVA-Y-AFECTIVA.aspx It allows to measure cognitive and affective components (essential in the present moment): adoption of perspectives, emotional compression, empathic stress and empathic joy. Try to translate.

| improve this answer | |
  • $\begingroup$ Certainly a higher error variance, yes? $\endgroup$ – mflo-ByeSE Jul 19 '17 at 17:34
  • 1
    $\begingroup$ I'm a bit confused by your second and fourth bullet point. Would you mind re-phrasing them? $\endgroup$ – Seanny123 Jul 19 '17 at 17:59
  • $\begingroup$ @Seanny123 In second point I mean that you can not incorporate a good analysis of the frequency of a problem behavior to a test (based on a registry), basically asks for significant frequencies or not (very frequent, frequent, Neither frecuent nor infrequent , Not infrequent) (or which the person considers to be a problem even if it is not a long time) and formulated differently or with questions that "should" correlate. $\endgroup$ – hexadecimal Jul 19 '17 at 19:04
  • $\begingroup$ @Seanny In the fourth point I mean that the person administering the test must always keep in mind that the person who answers may have doubts. I forgot to add that the person can choose not to answer a question. $\endgroup$ – hexadecimal Jul 19 '17 at 19:06
  • $\begingroup$ @mfloren It depends on the subject of the test and how the experts suspect that these questions should correlate, for many "organic problems" no doubt, in the empathy a larger amount may be admitted because they are many factors, are usually suspected 4 (two dichotomous poles: cognitive and affective empathy) and seem to be many more. $\endgroup$ – hexadecimal Jul 19 '17 at 19:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.