Among the very few programs of the last French national championship that I have watched is the free dance of Gabriella Papadakis/Guillaume Cizeron. The two dancers won the title, as we all knew they would. Eugeniia Lopareva/Geoffrey Brissaud are excellent dancers, it is a pity that they can’t go to Beijing, but the difference between the two couples is evident even to an inexperienced eye like mine. What surprised me wasn’t the result, but Gabriella’s fall on the second twizzle. It happens. Even though the best skaters manage to make it seem that what they do is easy, they actually do things difficult, and a mistake can happen. I wish them to be able to skate at their best in Beijing, there the result is far from obvious. After watching the competition, I went to take a look at the protocol.
First I looked at the Rhythm Dance:
Papadakis/Cizeron are good, but twenty +5 and only five +4? Four of the five elements received the maximum possible GOE, the fifth a GOE only slightly lower. And also in the PCS the score is almost the maximum possible, only in skating skills they didn’t received a perfect 10.00.
In free dance there has been a fall, how high can the marks be?
The judges are Ségolène Duprez, Natacha Pontonnier, Alexandra Volle, Clément Perrigouard and Mélanie Lambert. Duprez, Pontonnier and Perrigouard have several years of international experience as judges. Four of them (the exception is Perrigouard) awarded to Papadakis/Cizeron 9.75 in performance, all of them awarded 9.75 in interpretation of the music. And this is the rule:
For me there’s something wrong…
It is well known that the marks in national championships are often inflated. They are not inflated equally in all nations, and they are not inflated equally for all skaters, because the internal politics of the various federations also plays its part, but these marks cannot be taken too seriously. It shouldn’t be, the marks should always be correct, but it is it. Why shouldn’t the ISU accept national championship with too high marks?
These judges also judge international competitions, and the rules are the same. When a judge, for whatever reason, starts assigning marks that don’t respect the rules, then he gets used to it, and the more time passes, the more absurd his marks become. And, especially in the case of the results publicized in international the press, the marks of the national championships can influence the other international judges. This graph is dedicated to the marks in components assigned to Nathan Chen in all competitions of the 2018-2019 season. First (at the top of the table, on the left of the graph) I indicated the marks of the short program, then those of the free skate. In the graph I separated the competitions with a vertical line and highlighted the national championship with a gray background.
Before the national championship, almost all of the marks Chen received did not reach 9.00, his highest mark was a 9.25 in Interpretation, at Skate America. I have highlighted this mark with a red horizontal line. At the national championship Chen received very high marks. In subsequent competitions almost all of his marks, even if lower than those of the national championship, are higher than 9.25. Out of 20 marks, 16 are higher than the highest mark he had previously received, one is the same, three are lower, but they are still higher than almost all of Chen’s previous marks. Has Chen really improved that much? Maybe, but it is more likely that the judges were influenced by the anchoring effect explained by Daniel Kahneman in Thinking, Fast and Slow.
Kahneman explained this behavior, called the anchoring effect, by recalling an experiment in which two groups of people were asked whether, according to them, the tallest sequoia in the world was more than 365 meters (first group) or 54 meters (second group). On a subsequent question as to which, according to them, was the height of the tallest sequoia, the average estimate of the first group was 257 meters, that of the second of 86. The presence of two very different anchors had led the two groups at different estimates of 171 meters. A judge who remembers the 10.00 in performance Chen received in the national championship free skate may think that 9.50 is a fair mark because it is lower, but previously Chen had hardly ever exceeded 9.00. The ISU should not accept such absurd marks. How? Preventing the judges who assign absurd marks in the national championship from judging international competitions could be a first step, possibly accompanied by some sanction. In the worst cases to the whole federation and not only to a single judge.
I’m not able to say if Papadakis/Cizeron really deserved their +5 and their 10.00, I don’t know ice dance so well that I can tell. I know that in free dance nine of the marks they have received definitely don’t respect the rules. The high marks awarded to Papadakis/Cizeron on this occasion can be as damaging as those awarded to Chen in 2019, and they are as wrong as any marks that do not reflect what the skaters did on the ice according to the rules, regardless of competition or identity of the skater. What interests me at the moment, however, are some very specific marks, those 9.75 in Performance and Interpretation, when, in the presence of a fall, according to the rules those marks cannot be assigned. With a fall, the maximum mark is 9.50, the equivalent of a 10.00 for those who skate a perfect program. If in this case such a simple rule has not been respected, what happens when skaters deserve lower marks? Do the judges remember to assign a lower mark in components because there was a fall? I decided to check it out.
If for a perfect performance the maximum is 10.00, in the presence of a fall the maximum marks is 9.75. This means that, according to all the judges, Papadakis/Cizeron that day deserved a 10.00 in skating skills, a lower mark appears on the protocol simply because, by rules, the judges deducted 0.25 to the mark the two skaters deserved.
I applied this reasoning to the protocols (of the Men’s competition) I checked. In the presence of a fall I hypothesized that the judges applied the deduction and lowered the score, therefore to the marks we read on the protocols I added 0.25 points in skating skills, transition and composition, 0.50 in performance and composition. If the falls were two or more, I added 0.75 and 1.25 points respectively. In fact, this limit on the high marks in components says that a fall causes the PCS of a short program to drop by 1.75 points, in the presence of two falls of 4.75 points. At the last World Championship for his short program Shoma Uno received 44.60 points in the PCS. He fell once. Applying the rules, without falling he should have received 46.35 points. Vincent Zhou has fallen twice, his 40.47 is equivalent to 45.22 for a skater with the same skating skills but who made no mistakes. Zhou. 45.22. They are a name and a number that cannot be in the same sentence.
Okay, I took the 46.35 from Uno, the 45.22 from Zhou, but also the 44.22 actually received from Mikhail Kolyada, or the 46.39 from Jason Brown (and so on)… and divided them by 5. I did the same thing with the free skate, only that I doubled the numbers: with one fall I added 3.50 points to the PCS, with two or more falls I added 9.50 points, then I divided the total by 10. Why did I do this work? To have marks in components comparable between skaters, without worrying about whether the marks have been awarded to a short program or a free skate.
I gathered all the marks in a table which you can find below, distinguishing between the scores actually received by the skaters and the correct ones considering the mistake made by the skater. If the skater has fallen once (+1.75), the square is colored light yellow, if he has fallen two or more times (+4.75) the square is colored deep yellow. Let’s look at the rule. According to the ISU
Serious errors are falls, interruptions during the program and technical mistakes that impact the integrity/continuity/fluidity of the composition and/or its relation to the music.
I can’t watch all the programs, either because I don’t have the time, or because not all programs have video available. If a fall is undoubtedly a serious mistake, I wondered about the other mistakes I could identify from the protocol. If for an element at least half the marks are -4 or -5, I assume that element was performed poorly. This is where the discretion of the judges comes into play (and also the fact that I don’t know what kind of error it is). Is this a serious mistake? Maybe yes maybe no. When in doubt, I have not touched the mark in the components but I have highlighted the presence of an imperfect element using red and not black to write the mark. I also used red for those jumps that received at least half of -2 (or -3, -4 and -5) as long as there are no calls on the edge or rotation. This is because the calls necessarily involve a lower GOE, but they are details that specialists see, that most of the public does not notice and therefore do not affect the fluidity of the element. If the jump does not have these problems, but still has a negative GOE, it means that some problems were clearly visible in the landing. In case a program contains two or more elements with low GOE, the writing is in red and in bold. The writing is in blue if there are no low GOE elements in the program, but there are a no value element.
I distinguished the two types of errors because a no value element can be visible for those who know the rules, invisible for those who do not know it. Take the flip performed by Shoma Uno in the Skate America short program. It is a double, so the value of the element is rightfully deleted from the score. However, someone like my mother is unable to notice the mistake because she does not distinguish the number of rotations of a jump and does not know the rules. Uno did not go out of balance and did not make any strange movements. He did a good double flip. For my mum that’s a good jump. So a no value element maybe breaks the fluidity of the program, maybe not. The programs should be looked at. I just note that something strange has happened, but less serious – for the components – than the other errors.
Which skaters have I watched? All the strongest in the men’s category in all international competitions in the last Olympic four-year period. I know that the rule on components has changed, in the 2018-2019 season (Communication 2186) the maximum mark was 9.75 for each of the five component with one error, while with two or more errors was 9.50 for skating skills, transition composition and 9.00 for performance and interpretation. In recalculating the score taking this deduction into account, I paid attention to what the values were for that specific season. As for the skaters I checked, they are all those who have made it to the podium at least once at the World Championship, the European Championship, the Four Continents Championship or in a Grand Prix competition. There are 27 skaters, too many for a legible screenshot, so I have not entered the least significant data, those of the skaters who have participated in a few competitions. These are Makar Ignatov, Shun Sato, Kazuki Tomono, Sergei Voronov and Han Yan. In theory I should have removed Semenenko and not Tomono, but I suspect that the Russian federation care for Evgeni more than the Japanese one care for Kazuki, so I left outside the data of the Japanese skater. As for the competitions, I have distinguished them by importance. A competition-by-competition check tells a lot of interesting things, for example in the Rostelecom Cup the Russian skaters (and Kvitelashvili) tend to have higher ratings than the other Grand Prix competition they participate in, but distinguishing all competitions would have meant having six lines per season for the Grand Prix competitions instead of two, for a final list that is 102 lines long and not 37. Too long.
The blue band highlights the Grand Prix competition of the 2020-2021 season. In theory it is a Grand Prix competition, in practice it was a competition with local skaters and judges, almost an anticipation of the national championship, so even if I have indicated them, those marks cannot be taken seriously. 77 of the yellow boxes have a black border. These are the times in which the skater has fallen one or more times and, with the marks of the PCS revised to respect the maximum possible GOE, he obtained a score higher than his season best score of at least 0.10 points (which, multiplied by five and with the factorization, it correspond to 0.50 points in the short program, 1.00 in the free skate). I also did the reverse check, they are the boxes with the red border. On 13 occasions the revised and corrected score remains below the lowest score obtained by the skater that season. Sometimes a skater skates better, sometimes worse, it’s normal, otherwise there would be no need to assign marks competition by competition. But when the skaters fell, on 13 occasions they proposed their worst performance, in 77 they proposed their best performance of at least 0.50, or 1.00 points (if I consider the best performance regardless of the difference, then I also count votes like that 8.35 obtained by Dmitri Aliev in the first Grand Prix competition of 2018, which hasn’t a black border because in the previous competition Aliev had scored an 8.33, the total number goes from 77 to 90). This means that either the skating quality goes up in case of falls, or that the judges don’t always remember to apply the rule on the maximum mark for the components.
Is the rule being applied arbitrarily? I suspect so. Let’s look at the protocol of the free skate of the Autumn Classic International 2019 in the SkatingScores version:
Two of Hanyu’s elements had (rightfully so) a negative GOE, four of Aymoz’s elements had a negative GOE. Of the two skaters, which one skated a more imprecise program? Hanyu also received three underrotated calls (and now I don’t argue about the correctness of those calls, or of one no call for Aymoz), but the GOE remained positive because the jumps were beautiful, and their execution had no impact on the integrity/continuity/fluidity of the composition and/or its relation to the music. Only two of Aymoz’s elements have received a call, but the rule does not distinguish between two and three, it only distinguishes between singular and plural, and if there are no other problems a downgraded jump is certainly less correct than an underrotated jump.
Considering these protocols, I have the impression that neither of them deserved the application of the rule relating to the lowest marks in components. Aymoz made more mistakes, or more serious mistakes, so if the rule was to be applied to only one, it had to be applied to Aymoz. In the case of particularly severe judges, it could be applied to both. In my opinion it would have been too severe, but if applied to both of them the rule would have been applied fairly, or slightly unfavorably to Hanyu, but not really incorrectly. Has the rule been applied? And if so, for whom?
In two seasons Hanyu has never received such low marks in components. In two programs he received only slightly higher marks, at the Autumn Classic International a year earlier, but on that occasion in the free skate he had fallen once, as well as having had another negative GOE and committed a few more small imprecision. Lower marks in the 2018 competition were fully justified, here not so much. On the other hand, for Aymoz, these were higher marks not only than those of the entire previous season, but also those that he would have received in almost all subsequent competitions. Only in the free skate of the Grand Prix final, and in the very bad short program of the European Championships, would he have received higher marks. In a completely arbitrary way, the judges lowered only Hanyu’s marks, while with Aymoz they opted for generosity. The judges who preferred Aymoz are the Icelandic Halla Bjorg, the American Hal Marron and the Mexican Sasha Martinez (in interpretation 8.75 to Hanyu, 9.75 to Aymoz, and she could be one of the Olympic judges), while for the Australian Karen Anne Harris the two programs are equivalent.
There are many strange marks and it would be nice to be able to look at them all. The ISU should do it, otherwise it makes no sense that it exists, I (compatibly with the time available to me) control what strikes me in some way. And, staying in the 2019-2020 season, I was struck by the marks of American judge Doug Williams at the NHK Trophy. Williams, like Marron, could judge Women and Pairs in Beijing (Marron could also judge Ice Dance). The protocols from SkatingScores:
Here too are Hanyu and Aymoz, but I watch the best four skaters.
To be sure not to make mistakes, I made some calculations. For each skater I indicated the mark awarded by Judge 4, Williams, the average mark that entered the skater’s score, and the difference between the two marks. In the last row is the sum of the five differences.
So Williams was only strict with Hanyu, generous with Aymoz and Brown and very generous with Sadovsky. Another detail is really fascinating, for all the judges. I really like, Jason Brown, but for Brown this wasn’t exactly one of the best competition, he fell twice. And This is a screenshot from
Communication No. 2254
SINGLE & PAIR SKATING
Levels of Difficulty and Guidelines for marking
Grade of Execution, season 2019/20
The following Communication replaces Communication No. 2186
With two falls, his highest possible mark in Interpretation, assuming everything else was perfect, was 8.75. How is it that he received 8.79 as the final mark? The German judge Tobias Bayer gave him the equivalent of 10.50, the Italian Claudia Brambati the equivalent of 10.25, the Australian Kylie Yacopetti, the Israeli Albert Zaydman, the Russian Igor Dolgushin, the American Doug Williams, the Japanese Yoki Kuno and the French Florence Vuylsteker the equivalent of 10.00, only the Canadian Deborah Islam awarded to him the equivalent of 9.50. All marks higher than those received by Hanyu.
Since I’m here, I see that among the errors for which the maximum starting GOE is +2 there is the sign <<, but not the sign <, therefore it is confirmed that one of Aymoz’s mistakes at the Autumn Classic international was more serious than Hanyu’s mistakes.
Could these be occasional mistakes? Maybe, even if the US judges and Sasha Martinez have a curious habit of always lowering Hanyu’s marks. Those programs are relatively old. Has the judgment improved? Let’s go back to more recent competitions, maybe now the judges have got used to the rules on the maximum PCS. I checked the marks in the components at the World Championship and at the World Team Trophy 2021. Not of all, only of the skaters who got on the podium at least once in the most important competitions, without distinction between men and women. The 8.86 obtained in the short program of the World Championship by Rika Kihira in skating skills is not so different from the 8.89 received by Mikhail Kolyada, even if in the overall score of the PCS for her the factorization is 0.80 and for him 1.0. The two disciplines do not have different technical requirements, so if I exclude the factorization from the statistics, the marks become comparable.
For the skaters present in both competitions I have listed four marks, for those present only at the World Championship two, for Vincent Zhou one because he did not qualify for the World Championship free skate. In the average marks that I indicate (to be multiplied by five and to be factored) I took into account the maximum possible mark in the presence of falls, highlighted by the yellow boxes, while the red writings indicate the presence of negative GOEs, with the system I explained earlier. In the two columns on the left (DZ-EB) the skaters are listed in alphabetical order, for each of them the marks go from highest to lowest. In the two columns on the right (ED-EF) the order is from highest to lowest mark, without distinction of skater. I’ll explain the colours under the screenshot.
For skaters who in at least one program have made errors visible on the protocol – whether they are falls or negative GOEs – I have highlighted the highest score. This led me to not highlight any marks for six skaters, Brezina (two programs, both with a fall), Kagiyama, Kvitelashvili, Messing, Sakamoto and Semenenko (two or four programs without errors) because the marks they received were comparable only with those of programs in which they skated in the same way. It is fascinating to note that for Chen, Shcherbakova, Brown, Kolyada, Kihira, Tuktamysheva, Yan, Rizzo and Cha, nine skaters, the program with the highest score is not a program without errors but a program with a fall. For Uno, Trusova, Sadovsky, Miyahara and Grassl, five skaters, the highest scored program is one in which they have fallen two or more times. Only Hanyu, Aymoz, Tennell and Jin, four skaters, made mistakes in at least one program but their best program was one where they made no mistakes (actually Jin’s case is a bit different, in a program he fell once, in the other two, so he didn’t skate programs without errors, but the highest marks came in the program with fewer errors). As for Zhou, having performed only one program it is not possible to make comparisons with his other performances, but in my opinion it is important to emphasize that – taking into account the maximum possible score – his average score is higher than 9.0, and higher than that of most of the other skaters.
While on the one hand there are the unfairly low marks Hanyu received in the 2019 Autumn, on the other hand there are absurdly high marks of Zhou, but also of Trusova, Uno and Chen, at the last World Championship. I do not believe that – except for a few cases – the judges have knowingly raised the marks of the skaters who have made mistakes. I believe that the judges do not remember to apply the rule on the maximum mark, or that they are unable to apply it correctly in the time available to them. And when those who have to judge are unable to apply the rules, there is a huge problem. The ISU should change this rule, or the way in which scores are awarded, because as things stand, its application is completely arbitrary, and figure skating risks ceasing to be a wonderful sport and turning into a farce.
