Premise: I have not watched the competitions at the Nebelhorn Trophy and I have no idea how the skater skated. I just came across a Tweet from SkatingScores that raised some doubts in me:
In a Rhythm Dance program there are 25.14 points of difference between the most generous judge and the most severe judge. Did all the judges watch the same program? Because from the protocol I just am not able to understand. Did Holubtsova/Bielobrov skate well, and therefore did they deserve positive GOEs, or did they make mistakes, and therefore deserve negative GOEs? And even outside the technical elements, are they able of skating at a good level, or their skating is poor?
All the marks are a mess, but two elements stand out because they were judged positively by two judges, without infamy and without praise (they awarded a 0) by two judges, and judged to be incorrect by two other judges. And then should we always blindly trust the judgment of the judges? Which judge? Because unless we suffer from a split personality, it is impossible to trust everyone at the same time.
Since there is much more information on SkatingScores than on the official website, I went to check the protocol in that version. Speaking of SkatingScores, ISU, have you ever thought about asking whoever manages SkatingScores to collaborate with you? That he can do a very useful job on protocols is clear, and I’m sure he could also fix your site, which isn’t exactly the most organized site in the world when you look for some information.
Okay, end of digression. Protocol in the SkatingScores version, with indicated, in addition to the total scores awarded by each judge, also their nationalities. I don’t write anything about the American, Canadian, Australian and German judges. In the protocol of the ice dance competition I did not notice anything particular about them, apart from the possible national bias and the fact that they were much more severe with the Ukrainian dancers than the first two judges. The marks of the first two judges, however, are interesting. These are the Ukrainian judge Irina Medvedeva (about whom I remember having already written, but I don’t remember why) and the British judge Christopher Buchanan.
That the Ukrainian judge helps his compatriots is not surprising. National bias is quite common, but also easy to notice if someone want to check it out. But why the British judge? Out of curiosity, I went to check the marks received by the British ice dancers Fear/Gibson. That in this competition Fear/Gibson are the strongest and deservedly are at the first place is out of question, but even if the ranking is correct, there is a score and a score, also because in a few months Fear/Gibson will almost certainly fight for a medal at the European Championship. So a high score now can aid them in January, because the judges can become accustomed to award them high scores.
For four judges Fear/Gibson were good, for two they were extraordinarily good. Which? The British judge, unsurprisingly, and the Ukrainian judge. A case? Maybe, but if I were in the ISU I would ask myself a few questions. And I would ask to the judges a few questions, also because when there are two judges who assign strange marks, there are two consequences.
The first is that the strange marks seem less strange. Can we talk of national bias for Medvedeva, who awarded Ukrainian skaters 11.83 points more than the average, when the British judge awarded them 14.62 points more? The British judge’s marks normalized those of the Ukrainian judge. Can we talk of national bias for Buchanan, who awarded British skaters 4.19 points more than the average (in one program, let’s remember this), when the British judge awarded them 4.07 points more? The Ukrainian judge’s marks normalized those of the British judge. Those marks no longer stand out, the national bias is less evident.
Not only. Since there were two judges who gave strange marks, the marks of one of the two were not counted, but the others entered the score, so both pairs received a higher score than if only one of the two judges had been in the panel of judges.
I have not watched the competition and I do not intend to watch those programs that have yet to be skated, but these scores make me seriously doubt the reliability of the judges. I close with one last detail.
I wrote that it is not surprising that the Ukrainian judge is more generous with Ukrainian skaters, and that the British judge is more generous with British skaters (it must be said, however, that Buchanan was very generous with almost everyone, and he treated Fear/Gibson no better than Reed/Ambrulevicius, or the skaters who stood behind them, so maybe he was just in the mood to give gifts to everyone, especially Ukrainians, treated with impressive generosity). But that something is not surprising does not mean that it is right.
I did a quick check on all judges, based on another of the table published by SkatingScores. I highlighted the scores that the judges assigned to their compatriots. If a judge has assigned their countrymen the same ranking position that those skaters actually achieved, I have highlighted the box with a blue border. Obviously, if a judge’s countrymen are in the lead and the border is blue, it does not automatically mean that his marks are correct. Better than first place cannot be done, so that Buchanan has given Fear/Gibson first place, as the others have done, does not mean that he was impartial (but he has raised almost all scores by at least 5 points). If the border is green, it means that the judges have assigned their compatriots a better ranking position than the real one. If the border is red, it means that the judges have assigned their compatriots a worse ranking position than the real one.
Oops! I don’t see red borders, who knows why (but, to be honest, if I did the same job with the other competitions there would be some red border). What I see is a general tendency to help one’s compatriots. Maybe conscious, maybe unconscious, only the judges could say this. Perhaps it is time for the ISU to introduce a more objective way to evaluate competitions. Maybe even asking Hanyu, because from the little I know about his thesis, he has some good ideas. He is no longer interested in participating in the competitions, but I’m sure that if asked for some advice to improve the evaluation system, he would more than willingly collaborate on the project. And perhaps it would be time for the ISU to start paying judges, turning them into professionals and designating them personally, so that the judges don’t have to accountable to the wishes of the national federations that appoint them, and they don’t have a tendency to award marks that would please their federations. The season has just begun, and from the very first competitions it is evident that the judges’ evaluations cannot be trusted. Perhaps it is time to take action.
Added after FD
Did I write something yesterday about the fact that judges tend to give higher marks to their countrymen? This is what happened today in Free Dance:
Judges have a tendency to help their compatriots, sometimes a lot, and the ISU should take action. And there are almost 27 points of difference between the best and the worst scores received by Ukrainian skaters. How is it possible? Are both of these assessments really correct according to the ISU?
There is another detail that I have highlighted in this table. The two purple circles indicate the score awarded by the British judge to Ukrainian skaters, and the score awarded by the Ukrainian judge to British skaters. Even today both have favorably judged the fellow countrymen of the other. I suppose it’s simply a situation where the two judges have very similar views on how competitions should be judged, right?
The last table I propose, also from SkatingScores, is the one relating to the national bias, and I remember that both judges have lower national bias values than they could have because the Ukrainian judge’s marks have raised the final score of British skaters, and the British judge’s marks have raised the final score of Ukrainian skaters. And if it is true that in Rhythm Dance the British judge was very generous with almost all skaters, to the point that, according to this graph, on average with Fear/Gibson he was less generous than with the other ice dancers of 2.63 points, it is also true that his generosity was mainly directed towards Ukrainian dancers. If we exclude Holubtsova/Bielobrov from the calculation, the British judge on average awarded to the non-British dancers 7.97 points more, not 8.66, so Fear/Gibson were “damaged” (in a score strange, with marks from the Ukrainian judge so generous that they make the British judge’s marks seem normal ) by 1.94 points, not 2.63.
I don’t know why, but I would like ask them a few questions.
