What is the most robust method for peer grading?

Question

I intend to provide a group task for 30 people, split into 5 or 6 groups.

Ultimately, I want them to assess which group delivered the best result (after presenting results in front of other groups) as well as grade themselves within groups based on their contribution to the team effort.

Now, both game theory and my simple experience of observing people being unobjective would suggest there are plenty of ways this can go wrong if I don’t introduce some robust system that would deal, at least, with problems like:

• group likely to be second will give lowest grade to group they see as objectively the best so that they boost their chances of winning

• group likely to be the last will give lowest grades to second to last group so that they boost their chances to replace them and not finish last

• group members will tell me each person contributed to the same extent, but it will be untrue in most scenarios. Especially the best group has no reason to lower the grade of their colleague no matter how marginal his contribution had been. On the other hand, system where they score each other with percentage contribution is also wrong as sometimes indeed all group members collaborated on the same, high level and there is no need to point best or worst performing members and lower their grades, thus even if they all worked hard they would get the same, median mark for their performance.

Does such a voting/grading system exist?

Answer 1

In order to incentivize the desired behavior for your first point, you can do the following.

• If the group that comes out second, gave the lowest (possible) grade to the group that came out first, the punishment is at follows <specify punishment here>. An alternative condition would be: If the first group received the lowest grading from the second group. This is more stricter, but also fairer. Maybe add some tolerance (e.g. 1 grade divergence is okay).

You could try the same strategy for the second point, but that won't work. If the groups can't vote for themselves (as a group) and if we assume that the end result is a somewhat objective representation of the performance, the last group would give the second to last group the lowest grade. You could try the alternative of the first approach, though:

• If the last group received, the worst grade from the second to last group and if that grade diverges by more than 1, then <specify punishment here>.

In fact, you could introduce this rule for all the ranks. First and second place, second and third, etc. But the groups would need more time then, also this assumes the performances of the groups vary to such extent that it is indeed possible to subjectively rank them.

In order to have the groups assess them themselves, you could introduce a finite number of grades or classifications (e.g. top performer, medium performer). If the grades/classification can only be awarded a fixed amount of times, each participant will have a unique rank (relative to that group). The grade will probably not compare to group members of other groups (maybe if you weigh it with the group grade). It is possible to limit the number of possible grades to number of group members, or limit it to 2 or 3 (top, medium, bottom) and have multiple grading rounds.