Biased judging game

[gkelly]

So you think you can rig the results as a judge under IJS?

Let's see how you do.

At the end of this post are links to five ladies' short programs that were originally skated under 6.0 judging at different competitions. So there's no "right" answer. I chose these five because I think they're comparable enough in enough ways that I have no preconceptions as to how they woulda shoulda ranked if skated in the same competition, with some diversity of style and nationality.

I'll play technical specialist and call the programs according to 2011 rules. If more rule changes come in the next few days before I get around to posting, I'll use the new ones. We'll consider the spirals as transitions.

You play judge and assign GOEs and PCS for each of the skaters in real time. You don't have to judge all five if you just want to focus on a subset.

You may study the current rules and the scale of values before you start judging as much as you like.

See here (and here for ISU communication 1672) or here.

You might want to remember that the component scores in the ladies' short program are factored times 0.8.

But once you start watching the first program, don't consult any outside notes except the following two documents:

GOE guidelines

Components guidelines

Then post your GOEs and component scores for each skater, and also tell us how you think you ranked them. If you like, you can give 6.0 style Required Elements and Presentation scores, with deductions as necessary.

Feel free to choose a favorite to inflate the marks of, and a least favorite to lowball on purpose. But don't make it so obvious you'll get all your marks dropped and/or get flagged for bias. The rest of us shouldn't be able to tell whether you're cheating or really just think one skater is that much better than the other.

After a few days I'll post the levels and the base values for each of the elements I call. Anyone else who wants to play technical controller or ATS is welcome to consult by PM.

Should I post rotation or edge calls in advance, since judges would see those before they turn in their scores?

Then you can add in your factored components and GOEs to see how you really ranked the skaters, if you were the only judge. Did it match the ranking you thought you were producing?

If we get enough participants scoring all five skaters, and if someone with a scoring program wants to crunch the numbers, we could also combine the scores into overall results for a full panel.

And now, the skaters:

Butyrskaya 1997

Hubert 1998

Gusmeroli 1999

Nikodinov 2001

Suguri 2002

Have fun!

[Lanna]

You are awesome and this is awesome.

[Macassar88]

This seems fun. Maybe next time we do this we could do it with more recent skaters from the same comp for example the Olympics or worlds or something so we can see how much we can distort the results that way. My judging will be here soon.

[Jenna]

I have the software on my computer. GKelly, if you want to PM the scores/calls I would be willing to input them!

Also, I could serve as ATS if you need one.

[casken]

Oh, these are fun! I'm going to do this.

[aftershocks]

Sounds like a really fun game for numbers crunchers and skating fans. Have fun playing pretend figure skating judge for the fun of it. This kind of fun FSU game doesn't actually prove a thing, as it is not reality-based. Even despite my suspicion of ISU judges and judging system, I believe they do have to undergo some kind of training (albeit kind of worthless when some of them didn't even realize they were supposed to take a deduction from Amodio's rap lyric lp at 2011 Worlds). What the judges do seem to realize and enjoy the benefits of, is the fact that internationally they are anonymous and therefore protected from any public challenge regarding how they happen to score skaters. The system of judging is and has always been politically-based, and the scoring now under CoP is even more manipulable -- lots of times PCS scores make no sense, and that is one area where the judges manipulate the hell out of the scoring, believe it or not.

Sure, a lot of the time, the results seem fair, but in every major competition, there are placements that are questionable. At Olympics 2010, e.g., come on, re U.S. pairs Evora and Ladwig being so low-balled and placed behind two Canadian pairs teams that put out crap programs (both teams are no longer skating). Now at least when the latest two Canadian pairs teams are in front of Evora/ Ladwig, it can be viewed as a fair result.

I'll say it on behalf of CoP, it certainly has gotten the recent generation of fs fans/ viewers excited and engaged about the technical aspects of the sport, which can be good to a degree. But to believe that the scoring and that figure skating in and of itself is inherently all above board and not based on politics, gossip, and feds making deals with each other and with the ISU... oh well. Perhaps, skating fans are so used to the politics and cheating and denial of cheating that is so ingrained as a part of fs, that it is like a gimpy left leg, or an annoying Great Aunt, we put up with it, are used to it, and therefore accept it as a given, and sometimes do all we can to defend what is, and what we know will never change.

[aliceanne]

Tony Wheeler had an interview with a recently retired Belgian judge. The judge claimed that he gave average scores to a skater based on how he ranked them compared to the other skaters. For example if the maximum component score was 8, and he thought skater B might outskate skater A, he would give skater A no more than 7.5 no matter how well skater A did. Furthermore, this judge saw nothing wrong with this and saw himself among the few (10 percent in his estimation) honest judges.

[gkelly]

OK, judges, here are the calls Jenna and I agreed on for the elements we're using, that the judges would see during the competition. No level calls shown.

If you've already sent me your marks and want to make any changes based on these calls, you can do so before the marks are finalized.


Butyrskaya
3Lz+2T
3Lo
LSp
2A
FCSp
SlSt
CCoSp

Hubert
3T+3T
3F
FCSp
LSp
SlSt
CCoSp
2A

Gusmeroli
3Lz+2T
3Lo
LSp
CCoSp
2A
CiSt
FCSp

Nikodinov
3Lz+2T e
FCSp
3F
CCoSp
2A
CiSt
LSp

Suguri
3Lz+2T
3F
LSp
FSSp*
2A
CiSt
CCoSp

[casken]

Hubert
3T+3T
3F
FCSp
LSp
SlSt
CCoSp

You forgot Hubert's axel. I'll send my marks in soon!

[skateboy]

Hope we haven't forgotten about this game!

[gkelly]

Three people have sent me scores privately. Should we wait for any more?

Then I'll have Jenna calculate the panel's results.

And we, or the invidividual judges, should also calculate what their rankings would have been for these skaters if they were the only judges, but first they should let us know how they were trying to rank the skaters. Then we can see how much influence the base marks determined by the tech panel have compared with the judges' scores.

[Jammers]

The bias in skating today is nothing compared to the days of the cold war when the Soviet judges and their stooges would always vote for a Russian or East German skater or gymnastist. I'm almost glad the US boycotted the Summer Olympics in Moscow. I can just imagine the biased judging for the eastern bloc nations.