The title words are not mine. These are words spoken by Alexander Lakernik, vice president of the ISU, during an interview. According to him
there is an accepted assessment corridor
in which the judges can give their marks. If they stay in that corridor, everything is fine, otherwise discussions are needed. Then? After the discussions, if the answers are not convincing, does the ISU take action? Mine is just curiosity, I want to understand how things work.
I have already commented on these words, noting how small, at times, is the difference that separates whoever wins a gold medal from whoever wins a silver medal, or whoever wins a silver medal from whoever wins a bronze medal, or even whoever wins a medal from whoever does not win any.
In the same post I focused on some obvious mistakes made by the judges. But the obvious mistakes aren’t the only ones done by the judges. What would happen in case of small variations, acceptable according to Lakernik, but very widespread, present on all or almost all marks? And is it possible that there are mistakes done by all the judges?
Yes, as I have written countless times, in the 2019 Grand Prix Final Free Skate, Nathan Chen stumbled in the middle of the choreo sequence, and despite the stumbling all the judges gave him a +4 or a +5. All judges were wrong that day, so all judges can be wrong, and in certain circumstances the judges can be led to be wrong.
I know, I write very long posts, and sometimes in my posts I space on different topics. Reading me can be challenging. Anyway I suggest you to read the beginning of this long post, at least if you are not familiar with Anchoring, a type of cognitive bias. I start from this premise, and it is an important premise. If you haven’t done so yet, read Daniel Kahneman’s Thinking, Fast and Slow. Even if he hasn’t written a single word about figure skating, there are plenty of food for thought.
We all let ourselves be influenced by what surrounds us, even if we don’t realize it. If this were not the case, there would be no point in advertising. Of course, some people can be influenced more easily than others, and some events have a greater ability to influence people.
I repeated, for the 2019 World Championship, the same type of check I did last December on a program by Kazuki Tomono (I warn you, the post in which I wrote about Tomono’s marks it’s a really long post, and in this case it is not really necessary to read what I wrote there to understand what I am writing here, I entered the link only to note that it is a type of control that I have been thinking about for some time already).
What happened at the 2019 World Championship? I post only part of the results, with the difference of points that there was between Nathan Chen and Yuzuru Hanyu.
Chen won both programs and took gold with a very clear lead over Hanyu. Apparently, the competition couldn’t have ended any differently. But I wondered if some kind of bias did not come into play, something that led the judges not to evaluate what the skaters did on the ice, but what the judges expected to see. How do I know what they expected to see? For this I can only make hypotheses, I have no certainty. Who were the main contenders for the title?
One was Hanyu. Hanyu was the reigning Olympic champion, true. However the last free skate skated by Hanyu, at the Rostelecom Cup in November 2018, was a free skate full of mistakes. Hanyu was injured, we know it, but on the one hand it is easy for our mind to reduce the injury to something not too serious (he skated, he even did three quadruples, how bad could the injury be? Knowing Hanyu, a lot, but for many it is easier to remember the triple axel with fall, the other axel just simple, the downgraded euler, rather than thinking about what Hanyu has always done on the rink), on the other hand to think that maybe he has not yet recovered from injury and therefore in Saitama he cannot skate at his best.
Hanyu started his short program, and the first thing he did was a mistake on the salchow. I confess, at that moment I cursed. But the rest of the program was perfect. His triple axel… does it need to be said? It was his triple axel, no one is capable of doing anything like that. There have been other skaters capable of doing remarkable triple axels in the long history of figure skating, but none was never able of do what Hanyu does when he does the triple axel well, and that day he did it well. Did he received by the judges the unanimous +5 he deserved? Not a chance. Six of them allowed themselves to assign a +4, and I would like the ISU to ask the Czech Miroslav Misurec, the Croatian Antica Grubisic, the Israeli Albert Zaydman, the Swiss Bettina Meier, the Spanish Saoia Sancho and the French Philippe Meriguet what was missing to not give the +5. Which bullets were missing between
4) steps before the jump, unexpected or creative entry
5) very good body position from take-off to landing
6) element matches the music?
Two must be missing, only one is not enough for a +4. The ISU should ask them, and let us know the answer. The marks have to make sense based on the rules and based on what the skaters do, otherwise we don’t have a competition anymore, we have a farce. Among other things Grubisic, Meier, Sancho and Meriguet also managed to assign a +4 to Chen’s triple axel, Meier and Sancho also assigned a +4 to Jason Brown.
Just to understand the meaning of these marks, Henni has just made a beautiful analysis of them:
Only one of the triple axel performed that day deserved a +5, none a +4, and the marks Chen and Brown received are in no way justified. My suspicion, something I can’t prove, is that, influenced by the mistake on the salchow, the judges lowered Hanyu’s subsequent marks as well. And that they also lowered them in the free skate, after all Hanyu had just made a serious mistake in the short program, so I think there were few who expected an extraordinary program from him.
Over the past four years Hanyu has received inexplicably low marks, something I have dwelled upon when writing about the GOE and the PCS. Now let’s make a hypothesis. What would have happened if Hanyu had received only slightly higher marks? What if, because of that salchow, the Anchoring effect had led the judges to lower their marks? Not so much, by 1 unit in the GOE, by 0.25 points in the PCS. I did the calculations, being careful not to assign absurd marks (where there is already a +5 I have not raised the mark, it is impossible. Next to those marks I wrote ok in the protocol to explain that I have kept them as they were). Keep in mind that I have kept the calls from the technical panel for good, so I have not reviewed the rotations of the jumps or the position of the spins. What I have added is clearly visible in red. I deleted only one thing, the word factored in the upper area, above the components score, to have the space in which to write my number.
Hanyu’s Short Program revised:
Hanyu’s Free Skate revised:
By this I am not saying that Hanyu should have received these marks, to be sure which marks he deserved I should look back at the program (anyway, I’m sure that he was underscored, only I’m not sure of how much he was underscored without a serious check). What I’m saying is, if Hanyu had received these marks, no one would have thought they were wrong. Now I look at Chen’s scores.
How did was present Chen in the judges’ minds before this competition? Chen was the reigning world champion, therefore a very strong skater, perhaps as strong as Hanyu. Chen had also won the last two Grand Prix Final. I’m not sure many people remembered that Hanyu missed all three of these competitions because he was injured. Of course, Chen was only fifth at the Olympic Games, but because of a bad short program. It was he who got the highest score in the free skate.
Who knows why, I suspect that the judges dismissed that short program as a bad day that sometimes happens, and focused on the champion who skated an incredibile free skate and who had been almost unbeatable in the last two seasons. With what scores had Chen won all his competitions in the 2018-2019 season? In this case, instead of publishing the entire protocols, which anyone can easily consult on internet, I am gathering the data in a way that seems interesting to me. The competitions are Skate America 2018, Internationaux de France 2018, Grand Prix Final 2018, US National Championship 2019 and World Championship 2019. I ignore the 2019 World Team Trophy because it was held later, so it has no relevance for the World Championship.
One consideration before the table. At the World Championship Chen didn’t make any serious mistakes. No falls, no step outs, no two-foot landing. He has completed two clean programs, and that’s something I don’t discuss. In the first competitions of the season he received several negative marks because he made mistakes, it is obvious that by not making mistakes the negative marks disappear. But, leaving aside the clearly wrong elements, of what quality are the elements executed by Chen? Is Chen able of performing quality elements, or does he just did his elements without big mistakes? To understand this, I looked at the marks assigned to him. Competition by competition, I counted how many +5 he received on each element, how many +4, how many +3… Here, let’s ignore the negative GOE and let’s focus on the number of high marks (even in the PCS) he received, a sign that according to the judges he skated well and he performed quality elements.
I highlighted with the yellow background the number of +5 (or 10.00 and 9.75) received by Chen, with a light yellow background the number of +4 (or 9.50) received by Chen.
Skate America:
The quadruple flip of the short program is clearly wrong, it is underrotated, otherwise there are no particular problems, but there is no particular quality either. The marks Chen received indicate a generally good level, not extraordinary. The same goes for the PCS, and also for the free program.
Internationaux de France:
The Chinese judge Bing Han must have been greatly fascinated by Chen, he was the one to assign marks in the components really generous, he probably saw a different competition from the one that the other judges saw. And no, he hasn’t been generous to everyone. He tended to be strict, the skaters he was most generous with were Boyang Jin, Jason Brown and Nathan Chen. If you feel like doing it, a check with SkatingScores is quick.
Grand Prix Final:
I ignore the occasionally wrong element, sometimes mistakes happen and it is normal that, in a program without errors, the final score is higher. The ones that interest me are the positive marks, and mostly they are not that positive. Chen does his job well, but does nothing more, he does not add quality to his elements.
The next competition was the national championship and, as I have already noted, at least in the Men’s competition (I have not checked the other disciplines) there is no nation that is on par with the United States in giving their skaters marks very high and not justified by what has been done on the ice, with the express purpose of influencing international judges.
Nothing to complain about (at least without watching the programs, I don’t know if watching the videos would make me change my mind) about the disappearance of all the marks from 0 down, I suppose Chen didn’t do anything wrong. But have he really done such high quality elements? Did he really deserve such high marks in components? And really, as many say, since everyone knows that the marks in the national championship are higher than they should, do the international judges manage not to be influenced? Okay, let’s look at what marks Chen received in the next competition, the World Championship.
True, the marks are lower than in the national championship. They are also a lot higher than what Chen received before the national championship. To understand it better, I made some graphs. In the first graph I counted all the marks between +5 and +2 in each of Chen’s programs.
I have highlighted the national championship with a gray background. The horizontal red line indicates the highest number of +4 Chen received before the national championship, the blue line indicates the highest number of +5. After the national championship, Chen received more high marks than before.
Is this a fact that matters little because before Chen had made several mistakes and therefore some of his marks could not have been high? Yes, I am aware that the number alone does not tell us everything. For this I also calculated the percentage of his highest marks. I only considered the marks among +2 and +5, so the cases in which the judges felt that in the element performed by Chen the positive aspects outweighed the negative ones. This way the clearly wrong elements no longer have any influence on my chart. On this small number of marks I calculated the percentage of +5 received by Chen, the percentage of +4, that of +3 and that of +2.
Even so, it is evident that the percentage of high marks received by Chen has grown a lot.
I like the graphs, you may have understood. But I don’t know if everyone who reads me can follow my explanations, also because I’m afraid my grammar sucks. The tenses of verbs are a lottery, if I’m lucky I guess them, otherwise… let’s say that for me in English there are past, present and future, but distinguishing between past and past is beyond my ability. And I certainly make other mistakes.
Okay, since I don’t think I can explain myself better with words, I explain myself better with another graph. I divided the marks into two groups, +4 and +5, so excellent elements, and +2 and +3, so good elements. In the next graph we see the relationship between the excellent elements and the good elements performed by Chen. I changed the type of graph because in this case a line graph seemed clearer than a histogram, and I highlighted the national championship with the gray vertical lines.
Before the national championship, Chen received mainly good marks, after that he received mainly excellent marks. Even if they think not, even though they know they have given to Chen lower marks than the marks Chen received at the national championship, the judges have been swayed by the absurd marks of the national championship. And this has not only happened in the GOE, but also in the PCS.
Chen received much higher marks. The anchoring effect sought by the US federation to make their athlete win worked perfectly. What marks did Chen receive in Saitama? With him I did the opposite calculation compared to what I did with Hanyu, I lowered the GOE by one unit and the PCS by 0.25 points. This is the short program:
These are hypothetical marks, I know. Without watching the program, we cannot say that the marks had to be those indicated by me in red. Perhaps in some cases this type of change is too severe towards Chen, but in other cases, such as the triple axel analyzed by Henni, these marks continue to be generous, so in my opinion this is a good approximation. An approximation that no one, at the time, would have found shocking if this was the real protocol. Of course I did the same calculations with the free skate.
I repeat, maybe in some GOE I was a bit slightly severe with this calculation, but with Chen leading the competition after the short program, in the PCS the judges were all too generous and gave to Chen well over 0.25 points more than the score deserved in each of the items of the components. And I also remember that at that moment a score above 200 points had been reached only once, by Hanyu a few minutes earlier, so the score I indicated was still a very high score. How would the competition have ended if the marks had been the ones I indicated?
So a competition that, if someone only look at the final score, has one clear winner and seems that in no way could have ended otherwise, would have had a different winner. And this by making changes so small that at that time no one could have shouted the scandal. Sure, I changed all the marks (not the calls of the technical panel, and there is at least one rotation that I have serious doubts about), but I could have changed a few less, I could have not touched the step sequences and the choreographic sequence, the elements on which Chen received the highest marks over the course of the season, and I would still have found a score lower than Hanyu’s. And on the PCS I really should have been stricter.
Due to the way the ISU judging system is structured, if we are lucky, at the end of the competition the best skater wins, otherwise the skater the judges want to see win wins. And this even if their desire is unconscious, even if they are perfectly in good faith and do not understand that they have been manipulated by a propaganda built by someone interested in seeing a specific skater win.
After this competition, the idea of an invincible Chen who always deserves high marks solidified in the judges’ minds. We know the results of the last few seasons. That the scores assigned by the judges really reflect what the skaters did on the ice is another matter.
Dear Ms Martina Frammartino.
I am an vietnamese translator and a big fan of figure skating, so I want to ask you that could i translate it in vietnamese ? I promise that I I will write the full source of information below the translation.
Thank you for reading, I look forward to your response.
Hello eojwalyan
I would like the judging system to be improved, and the more people who talk about it, the more likely it is that something will be done. You can translate my posts.
Thanks
This is a comment in response to a comment I hid at the request of the person who wrote me.
I’m not at home at the moment and I can’t calmly review the program, more on that later.
Thank you so much.🙏 ❤️
Ok. I watched closely all the jumps. They’re at the limit, but for no one I can say with certainty that they would have deserved a call, or to exclude that there should be a call. Watching the jumps at a slow speed during the program, I got an impression. Watching the replays, in several cases I have doubts.
I’m more inclined to “fully rotated”, for every jump, but I’m not absolutely sure. I would say that it is yet another confirmation of the fact that we need a technology capable of telling if a jump has been rotated correctly or not. What I would like to happen does not matter. The results of the competitions, from the winner to the last classified, must be related to what the skaters do on the ice that day, not to my tastes or the tastes of the judges.
Mistakes of judgment can exist for all skaters, both for and against. If the ISU deigned to use the technologies, errors could be greatly reduced. That said, in recent years we have seen a clear tendency to have errors that go in a specific direction. This has not been the case all the time, but it has happened all too often not to give rise to bad thoughts.