3
$\begingroup$

I recently conducted a user study as part of my research in computer science. Respondents were asked to rate articles from the web on a five-point Likert scale from "Very Poor" to "Very Good".

Only 15 percent of ratings were on the negative side. My first conclusion was that most articles must be good. But then I discovered that there was considerably less agreement between users (each article was rated by a couple users) within negatively rated articles than within positively rated ones. So it seems to be harder for the users to accurately rate a bad article.

Is there a psychological effect that could (in part) explain this? References to papers are appreciated. Note that I don't mean bias issues regarding the Likert scale; it is carefully designed.

I'm a layman in psychology, but I was thinking that maybe there was a subconcious fear of rating too low so user rated too high... or something like that.

Improve this question
$\endgroup$
3
  • $\begingroup$ So, just for clarity, the overall distribution of site evaluations is skewed toward the positive side and the variance of the within-site responses is negatively related to the evaluation mean of the individual sites? Is that your result? Or how did you draw your conclusion if else? $\endgroup$
    – user7759
    May 14 '15 at 18:29
  • $\begingroup$ @MaríaAnt Yes, that is correct. It is my conculusion I obtained by looking at the survey responses. $\endgroup$
    – PhillipM
    May 16 '15 at 9:20
  • $\begingroup$ Positive bias? As measuring two constructs (bias and article) instead of 1, variability increases? $\endgroup$
    – mflo-ByeSE
    May 5 '17 at 20:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.