The Stack Exchange websites has people anonymously up vote or down vote whether questions and answers are good or bad and everyone builds their reputation based on this alone. I saw a question on the mathematics board that was not off topic but, in the beginning, was being down voted due to either being too elementary or perceived as what I consider people to be saying as a "stupid question". Do you think that removing this shawl of anonymity would change the way people interact with each other on Stack Exchange? For better or For Worse?
I have had the same question a while back when I observed similar behavior in a forum on a language learning website (I know that the question here is specific about stack-overflow).
There were two things I noticed- One with regards to my own behavior. I started giving my votes without having to worry that I will offend or please someone. -Two was that I have seen people downvote some of the questions which were relevant to the topic.. while some irrelevant content was upvoted.
So I believe the anonymity works both ways- good and bad.
Of course it would change the way people interact with one another. Social and situational settings strongly affect behaviour.
It's funny you see it this way. One can argue that it's exactly the opposite! Like with parliamentary elections, anonymity means that people act based on their own beliefs without being swayed by social pressure.
What's more without anonymity, people may start keeping score, which will further bias the objectivity of the votes.
When it comes to services like Stack Exchange, the general idea is that the various opinions will eventually regress to the mean - so the majority has it. Sure, for this to happen you need quite a lot of votes, but my experience is that usually gross down/up votes are justified.