Nationalistic bias in figure skating judging, 2018-present

[bardtoob]

YouTube

Enjoy the videos and music you love, upload original content, and share it all with friends, family, and the world on YouTube.

youtu.be

"Now, you know, Nationalism is going to come here, probably in the US and Canadian judges, but that's alright, that's alright. Their personal opinion. That's okay."

~ Dick Button

 

[SkateFanBerlin]

I prefer the look of North American skating. I'll take clean lines, quality of position, jump quality and strength over the busyness and barely completed 3A/4's. Is that bias or preference?

 

[Aussie Willy]

But it IS a blog post. That doesn't make it less valuable. There's no need to criticise it for not conforming to the stylistic norms of a different, unrelated genre of writing.

But I think you can question the inherent biases that the author has. They make it quite clear that they think judges are corrupt in the first paragraph and used a particular methodology to justify that. However that is where the problems start with it.

I think they would have been better off using another way to describe what they were trying to achieve. However it comes off as a bit of a witchhunt.

 

[Orm Irian]

I prefer the look of North American skating. I'll take clean lines, quality of position, jump quality and strength over the busyness and barely completed 3A/4's. Is that bias or preference?

Since you're not formally assessing skaters' work during competitions and affecting their placements and scores, it doesn't matter what you call it. Feel free to enjoy what you enjoy!

 

[Orm Irian]

But it IS a blog post. That doesn't make it less valuable. There's no need to criticise it for not conforming to the stylistic norms of a different, unrelated genre of writing.

I'm not criticising it. Quite the contrary, I'm pointing out that it's exactly fit for its stated purpose and publication venue.

 

[Deleted member 40371]

The fact that Australia is placed among the top nations in itself is an anomaly. They hardly have a athlete in top 10 at the worlds. So author adds Australia to say they are not biased just as. Proxy rather than add anything substantive.

I prefer the look of North American skating. I'll take clean lines, quality of position, jump quality and strength over the busyness and barely completed 3A/4's. Is that bias or preference?

clean lines and North American skating, jump quality when most of the top ladies are getting hit by under rotations ?? It surely looks like a conscious bias.

 

[tafattsbarn]

Having had a quick look at the article itself, my main concern is the way it is written. It starts off basically accusing judges of bias in a negative context. It would be better to explain what bias is first rather than launch into accusations. As the first comment about the article says it appears the author is not completing unbiased when it comes to their methodologies and what they are trying to seek in relation to skating judging. It also names and shames which I don't think is helpful and could put people on the defensive if what they are trying to do is be helpful rather than just being accusatory. In fact naming and shaming and putting it in writing could go against member protection, code of conduct and the rules of fair play.

I do think the intention is good and it could be used as part of judges training however it doesn't appear entirely objective in what it is trying to achieve.

Also the author while suggesting they hope to improve accountability, do not list their credentials or even their name. If they want to give credibility to this "project" they need to show these things.

A couple of thoughts on the methodology:
- A judge's marks being an outlier is not evidence in and of itself of that judge being biased. The author's explanation that "outlier scores only “count against” judges if they align with expected patterns of nationalistic bias" doesn't make any sense, because it sounds like s/he is only including outlier scores when those conform to his/her theory that there is national bias.

- Calculating a judge's scores for the same skater across multiple competitions is meaningless, because a skater doesn't perform identically all the time. A judge's score for a skater might vary across time because the quality of the skater's performance varies, not because the judge is biased for or against that particular skater.

- The analysis only looks at the "top" federations, i.e. those with the most judges. IIRC some of the most blatant examples of cheating judges in recent years have been judges from smaller federations - federations that have a lot to gain (e.g. assignments to more competitions) by having their skaters do well. The analysis should include all judges if the author really wants to see whether there is national bias.

- Maybe a biased judge is going to be a biased judge no matter what country they judge for. IMO the problem is that there is biased judging at all, regardless of whether it's nationalistic bias, helping-out-a-coach-friend bias, gender bias, anti-Carmen bias, whatever. I'm not convinced that a focus on nationalistic bias alone is really getting at the problem of biased judging.

Feel free to forward this as a comment on the authors blog, they've been really good about responding to criticism on the platforms they've personally posted their findings. Commenting it here won't help them in any way.

 

[love_skate2011]

History with most home advantage inflations

1 Canada
2 USA
3 Russia

non big country with compatriot skater inflations would be Italy and Australia
politically weak include Japan and China

of course its a given culturally close countries will side with its affiliated countries
like USA, Canada, with puppet countries like Australia, New Zealand
or Russia with Ukraine, Kzakhstan, Uzbekistan, Belarus and all other stans

 

[Aussie Willy]

Feel free to forward this as a comment on the authors blog, they've been really good about responding to criticism on the platforms they've personally posted their findings. Commenting it here won't help them in any way.

I already have. But the point of this thread is discussion about it so that is what we are doing.

 

[binbinwinwin]

On Reddit, the author posted in an earlier thread when they posted the raw data that Ukraine and Kazakhstan's judges were significantly biased towards Russian skaters. This is the only case in which any other country's judges were biased in a way that consistently benefited another (specific) country. Interesting considering the close ties of the federations...

On those same lines I wonder if there's data about whether or not certain judges are biased towards specific skaters from another country as opposed to that country as a whole. I know a lot of judges have a "type" of skater they like so I wonder if it's consistent regardless of country.

Even if they are it's not hard to believe that a judge from Kazakhstan or Ukraine trained in Russia before just because Russia is a very well established figure skating country. They may prefer their style or they may have known the skaters for a long time so they're more exposed to their skating.

As for this actual post itself, I think people keep trying to prove that national bias is a thing without acknowledging the paths that the judges take to get to being an international judge. In terms of national bias, if a judge judged Anna Shcherbakova as a novice and junior in Russia and then eventually at a GP, her gradual improvement will be more obvious to that judge than a judge from Canada. You can swap out Anna, Russia, Canada with other countries and skaters too. I think these kind of things are inevitable in the sport and honestly I think we need different schools of thought (within reason of course, eg chinese judge scoring Boyang as Olympic champion is bad). I learn something new from different people all the time, from coaches at the rink across town too.

Also what do they want to do about nationalistic bias? Have they skated and passed tests and actually judged figure skating before? What improvements do they want to the system?

 

[Vagabond]

The fact that Australia is placed among the top nations in itself is an anomaly. They hardly have a athlete in top 10 at the worlds. So author adds Australia to say they are not biased just as. Proxy rather than add anything substantive.

Australia is in that graphic because it is a federation with at least ten (ISU) judges, not because of the quality of the skaters. Finland is in there for the same reason.

Even so, Australian judges do have a reputation for fairness in judging.

 

[overedge]

Your first and second points are addressed in the analysis.

I don't agree with how they're addressed, which was the point of my post.

The third point is actually addressed in the discussion section of the post as well, though you may not be satisfied with the author's explanation.

No, I'm not.

 

[Rock2]

History with most home advantage inflations

1 Canada
2 USA
3 Russia

Now this would be considered a biased viewpoint unless supported by fact. Should you care to go down that path, please feel free to include an analysis of the PCS and GoE scores of Sotnikova at Sochi vs what she got at every international comp over the prior 12 months including especially Europeans not a month earlier.

Not quite sure where else I have seen such a miraculous transformation in scoring. But I'm sure you'll educate me with your data so thanks in advance.

 

[gkelly]

Not quite sure where else I have seen such a miraculous transformation in scoring.

Although it was a different scoring system and it might be difficult to find the complete protocols from pre-Internet era, it might be interesting to look at how Paul Wylie scored at 1991 Worlds and fall competitions vs. 1992 Olympics.

 

[Deleted member 40371]

Australia is in that graphic because it is a federation with at least ten (ISU) judges, not because of the quality of the skaters. Finland is in there for the same reason.

Even so, Australian judges do have a reputation for fairness in judging.

Where is this reputation, on this board? which in itself can be considered bias as it is English speaking board dominated by Canadians and Americans, so favourable to them. I am yet to see ISU confer such honour to Aussie judges.
Since Finland and Australia has been mentioned here, let us talk about the scores from US international were Angelique Clyde Smith's (AUS) score from SP were she tried to put large differential in scores between Ashley Cain/Leduc and T/M. Is this her nationalistic bias as Ashley has strong Aussie connections?.
Finland too did a lot of mischief with scoring at the Finlandia, their judge Saara Ehlt scored almost all top competitors low compared to the other judges and then scored Jenni Saarinen write on the average, the is also a huge bias as what she did was reduce the gap between Jenni and other top competitor, which the exercise mentioned in this thread would completely miss.

 

[Deleted member 40371]

Now this would be considered a biased viewpoint unless supported by fact. Should you care to go down that path, please feel free to include an analysis of the PCS and GoE scores of Sotnikova at Sochi vs what she got at every international comp over the prior 12 months including especially Europeans not a month earlier.

Not quite sure where else I have seen such a miraculous transformation in scoring. But I'm sure you'll educate me with your data so thanks in advance.

Well she didn’t perform like that as well, we could talk about Daleman’s score or Chan’s score when he was whipping the floor 3-4 times in on routine and getting huge cushion as well. Chan’s score are the biggest analogy of all.

 

[thvu]

Riddled with poor analysis presented with poor context for so many of the reasons already mentioned.

And for the most part, my reaction is “so what?”

 

[overedge]

Although it was a different scoring system and it might be difficult to find the complete protocols from pre-Internet era, it might be interesting to look at how Paul Wylie scored at 1991 Worlds and fall competitions vs. 1992 Olympics.

Another example, in the opposite direction, might be Moniotte/Lavanchy, who won two world ice dance medals and whose skills then seemed to dramatically decline when Anissina/Peizerat came along

 

[Holy Headband]

I don't agree with how they're addressed, which was the point of my post.

Honestly, your criticism sounds like you didn’t understand the methodology because it doesn’t make sense if you know a tiny bit about statistics or data science. Same goes for some of the other people in this thread, e.g. pointing out the author expected to find national bias before testing for it, which therefore taints the results of their analysis, which is not how either the scientific method or data analysis works.

Oh well, I tried to help.

 

[overedge]

Honestly, your criticism just sounds like you didn’t understand the methodology. Same goes for some of the other people in this thread. Oh well, I tried to help.

Get off your high horse. I do statistical analysis as part of my job, and I understand the methodology used here. Not agreeing with the methodological choices is not the same as not understanding them.

 

[thvu]

Oh well, I tried to help.

Data is meaningless without context.

 

[Holy Headband]

Get off your high horse. I do statistical analysis as part of my job, and I understand the methodology used here. Not agreeing with the methodological choices is not the same as not understanding them.

You said the analysis is flawed because skaters may do better or worse in different competitions when that has no effect on the results whatsoever. You said the analysis ignores outliers that disprove the bias hypothesis when it doesn’t.

 

[MacMadame]

Oh well, I tried to help.

 

[overedge]

You said the analysis is flawed because skaters may do better or worse in different competitions when that has no effect on the results whatsoever. You said the analysis ignores outliers that disprove the bias hypothesis when it doesn’t.

The difference across competitions not having an effect on the results in this study does not mean that difference is meaningless. Results come out of the way an analysis is designed, not because the data are infallible and are going to say the same thing no matter what type of analysis is conducted.

Similarly, I don't agree with how the author has dealt with the question of outliers. I didn't say the author "ignored" them; I said (paraphrasing myself) that s/he selectively included them in the analysis in a way that IMO didn't capture their full potential impact.

I will add too that if I got a data set from someone posting on a Hanyu fan site, I would verify that data independently before using it for analysis. The data set may be fine and the data collector may not be a rabid Fanyu, but in a study on bias, it's important for the researcher to be or to appear to be as unbiased as possible themselves.

Regardless, you seem very invested in defending everything about this study and dismissing any criticism of it as uninformed, so have at it. I don't see any point in repeating myself again to you.

 

[Aussie Willy]

Honestly, your criticism sounds like you didn’t understand the methodology because it doesn’t make sense if you know a tiny bit about statistics or data science. Same goes for some of the other people in this thread, e.g. pointing out the author expected to find national bias before testing for it, which therefore taints the results of their analysis, which is not how either the scientific method or data analysis works.

Oh well, I tried to help.

I am sorry I am not sure what you are getting upset about.

The person writing the article is kind of guilty of bias themselves. They think judges are corrupt and trying to prove it. And they are quite open about it. Which undermines their credibility because they are not objective.

Interesting I am sure they were one of the people who could have carried on like crazy about anonymous judging, but don't have a problem not revealing who they are. Pot calling the kettle black.

And BTW - I have made all these comments on that blog.

 

[Japanfan]

I think these kind of things are inevitable in the sport and honestly I think we need different schools of thought (within reason of course, eg chinese judge scoring Boyang as Olympic champion is bad).

I expect that Boyang is likely to at least medal at Beijing 2022. I think that fortunately for him, the timeline is favorable in that he is still growing as a skater/artist. I do worry a bit that S/H may have peaked this past season. But OTOH, they are fierce competitors - warriors on ice, really - and capable of reinventing themselves.

 

[Theoreticalgirl]

These blog posts about judging bias are such aggrieved Fanyu-borne bullshit. Stop giving these bad ideas oxygen.

Don't @ me.

 

[Rock2]

Well she didn’t perform like that as well, we could talk about Daleman’s score or Chan’s score when he was whipping the floor 3-4 times in on routine and getting huge cushion as well. Chan’s score are the biggest analogy of all.

I'm not sure what you are referring to with Gabby and home/national bias. Her PCS steadily climbed as her performances improved over the 2016 and 2017 seasons. Internationally she was pulling mid to high 7s going in and then mid to high 8s two years later. Her performances, packaging and presentation were generally improving in tandem so I'm unclear on what you saw as an inexplicable jump overnight as a result of a home skate.

Not sure where Chan fits. There was no inexplicable jump in scoring and his scores reflected what he did. He was penalized for his missed jumps in TES and there was no mandate at that time to deduct unduly on PCS for missed jumps. That rule came into effect in the next cycle so the scoring was correct and not inflated. People just had a conceptual problem that Chan could fall and win but the scoring system allowed for it.

People forget that Yuzu almost won his GP event in Asia when he had that awful collision with the Chinese skater, fell 3-4 times in his program and skated in a complete fog the entire time. Judging in this regard has been consistent and not favoring any skater from any nation. Can you show me your posts complaining about Yuzu hitting the podium in Asia for a skate that, all things considered, was beyond terrible and should have placed him out of the top 5? Likely not. See how bias works?

Sotnikova's skate wasn't much more special than anything she did up until that point. Even if you think it 'was' special in some regard can you argue her SS, TR and CH changed and improved in a month? OK give her a bit more for PE but in fact she removed TR and some of her CH between events leading up to Olympics to help her skate more cleanly I guess. She should have been ratcheted downwards if anything in most of her components.

My perspective is it was the home crowd that made a difference, roaring for every Russian performance and sitting almost quietly for everyone else. Go back and watch Yuna's free and focus on the crowd almost immediately after the music stopped. Only a handful of people clapping and cheering for her performance with a TON of people sitting motionless. If you were not on your feet for that performance (and Sotnikova's equally) then you are inherently biased or otherwise don't know figure skating. This to me is the perfect example of home bias where the crowd willingly played a factor in perception of performances. I have not seen this kind of behavior from any other country. Note even the Russian curling team stated they hate playing at home because the crowds are awful, cheering mistakes by non-Russian teams and sitting quietly when they do something well.

 

[mjb52]

You should have been at Minneapolis Worlds if you think “other countries” don’t engage in “that kind of behavior.”

 

[MAXSwagg]

I'm not sure what you are referring to with Gabby and home/national bias. Her PCS steadily climbed as her performances improved over the 2016 and 2017 seasons. Internationally she was pulling mid to high 7s going in and then mid to high 8s two years later. Her performances, packaging and presentation were generally improving in tandem so I'm unclear on what you saw as an inexplicable jump overnight as a result of a home skate.

Not sure where Chan fits. There was no inexplicable jump in scoring and his scores reflected what he did. He was penalized for his missed jumps in TES and there was no mandate at that time to deduct unduly on PCS for missed jumps. That rule came into effect in the next cycle so the scoring was correct and not inflated. People just had a conceptual problem that Chan could fall and win but the scoring system allowed for it.

People forget that Yuzu almost won his GP event in Asia when he had that awful collision with the Chinese skater, fell 3-4 times in his program and skated in a complete fog the entire time. Judging in this regard has been consistent and not favoring any skater from any nation. Can you show me your posts complaining about Yuzu hitting the podium in Asia for a skate that, all things considered, was beyond terrible and should have placed him out of the top 5? Likely not. See how bias works?

Sotnikova's skate wasn't much more special than anything she did up until that point. Even if you think it 'was' special in some regard can you argue her SS, TR and CH changed and improved in a month? OK give her a bit more for PE but in fact she removed TR and some of her CH between events leading up to Olympics to help her skate more cleanly I guess. She should have been ratcheted downwards if anything in most of her components.

My perspective is it was the home crowd that made a difference, roaring for every Russian performance and sitting almost quietly for everyone else. Go back and watch Yuna's free and focus on the crowd almost immediately after the music stopped. Only a handful of people clapping and cheering for her performance with a TON of people sitting motionless. If you were not on your feet for that performance (and Sotnikova's equally) then you are inherently biased or otherwise don't know figure skating. This to me is the perfect example of home bias where the crowd willingly played a factor in perception of performances. I have not seen this kind of behavior from any other country. Note even the Russian curling team stated they hate playing at home because the crowds are awful, cheering mistakes by non-Russian teams and sitting quietly when they do something well.

There was a big uproar about that Cup of China event. But of course you leave that out. The judges probably gave Hanyu enough points in order to get him into the GPF. They desperately needed and still need the incomparable amount of money he generates.

It’s a mix of blatant bias and incompetence. USFSA is certainly more aggressive with it as of late.