I’ve always had a feeling that judged Olympic events like figure skating were subject to certain nationalist pressures that seem rampant at such events, but the Freakonomics Blog has helped put things in perspective:
In response to allegations of vote-trading and home-country bias among figure-skating judges at the 2002 Winter Olympics in Salt Lake, the International Skating Union changed judging procedures. Scores are now reported anonymously, and not all the scores are used in the final tally. Unfortunately, as Ray Fisman writes in Slate, the reforms have been less than effective: the Dartmouth economist Eric Zitzewitz finds that anonymity actually results in a higher home-country bias, as it allows judges to hide from a “scrutinizing press.”
I feel like there should be some axiom that states that if an agency claims to reform itself with measures that do nothing or make the problem worse, for whatever reason, they like the problem.
For the first reform: they’re trying to enforce or promote integrity by granting anonymity? These people really think that a lack of accountability will lead to more responsible judging? Risible.
As for the second reform: let’s analyze this numerically.
Judges A, B, C (from Country A, B, and C, respectively) judge a performer from Country D. It’s an average performance, so, on a scale of one to ten, they give ratings of 5.6, 4.3 and 4.7 respectively. Let’s see what happens when we successively drop one score from each calculation:
Three scores that all hover around 10, appropriate for a so-so performer.
But now a performer from country B goes on and gives a similarly average performance. But B hasn’t won the gold in this event in 16 years and there is great pressure for them to win, or else lose face. So the judge from country B gives the average performance higher-than-deserved marks.
Their scores are as follows:
Let’s look first at the case of dropping B’s score: the average performance gets a score of 10.2 out of 20. Fine. In one of three outcomes, there is “fair” judging.
But let’s then consider the other two cases: the judge from Country B has his score kept in and one of the “fair” scores dropped. This means that his unfair score is now being averaged out by one less fair score a majority of the time (2 out of 3 cases). By definition, this means that, instead of his score contributing one-third in 3 out of 3 cases, his score counts one-half in 2 out of 3 cases.
So, statistically, this is a wash: in aggregate, dropping the unbalanced score a few times evens out with it being included relatively often. On a macro level, this policy makes no significant, statistical difference whatsoever.
So given that the counter-measures introduced to increase the “fairness” in judging either a) exacerbate the problem or b) have no effect on solving the problem, that leaves one of two options. Either the International Skating Union is stupider than a 20-year-old recreational statistician, or they aren’t really interested in solving the problem in the first place.