Biased judging game

Discussion in 'The Trash Can' started by gkelly, May 8, 2011.

  1. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,669
    So you think you can rig the results as a judge under IJS?

    Let's see how you do.

    At the end of this post are links to five ladies' short programs that were originally skated under 6.0 judging at different competitions. So there's no "right" answer. I chose these five because I think they're comparable enough in enough ways that I have no preconceptions as to how they woulda shoulda ranked if skated in the same competition, with some diversity of style and nationality.

    I'll play technical specialist and call the programs according to 2011 rules. If more rule changes come in the next few days before I get around to posting, I'll use the new ones. We'll consider the spirals as transitions.

    You play judge and assign GOEs and PCS for each of the skaters in real time. You don't have to judge all five if you just want to focus on a subset.

    You may study the current rules and the scale of values before you start judging as much as you like.

    See here (and here for ISU communication 1672) or here.

    You might want to remember that the component scores in the ladies' short program are factored times 0.8.

    But once you start watching the first program, don't consult any outside notes except the following two documents:

    GOE guidelines

    Components guidelines

    Then post your GOEs and component scores for each skater, and also tell us how you think you ranked them. If you like, you can give 6.0 style Required Elements and Presentation scores, with deductions as necessary.

    Feel free to choose a favorite to inflate the marks of, and a least favorite to lowball on purpose. But don't make it so obvious you'll get all your marks dropped and/or get flagged for bias. The rest of us shouldn't be able to tell whether you're cheating or really just think one skater is that much better than the other.

    After a few days I'll post the levels and the base values for each of the elements I call. Anyone else who wants to play technical controller or ATS is welcome to consult by PM.

    Should I post rotation or edge calls in advance, since judges would see those before they turn in their scores?

    Then you can add in your factored components and GOEs to see how you really ranked the skaters, if you were the only judge. Did it match the ranking you thought you were producing?

    If we get enough participants scoring all five skaters, and if someone with a scoring program wants to crunch the numbers, we could also combine the scores into overall results for a full panel.

    And now, the skaters:

    Butyrskaya 1997

    Hubert 1998

    Gusmeroli 1999

    Nikodinov 2001

    Suguri 2002

    Have fun!
     
    Lanna and (deleted member) like this.
  2. Lanna

    Lanna Well-Known Member

    Joined:
    Mar 21, 2006
    Messages:
    8,655
    :rofl: You are awesome and this is awesome.
     
  3. Macassar88

    Macassar88 Well-Known Member

    Joined:
    Dec 24, 2010
    Messages:
    1,836
    This seems fun. Maybe next time we do this we could do it with more recent skaters from the same comp for example the Olympics or worlds or something so we can see how much we can distort the results that way. My judging will be here soon.
     
    gkelly and (deleted member) like this.
  4. Jenna

    Jenna Well-Known Member

    Joined:
    Jun 20, 2009
    Messages:
    7,139
    I have the software on my computer. GKelly, if you want to PM the scores/calls I would be willing to input them!

    Also, I could serve as ATS if you need one.
     
    gkelly and (deleted member) like this.
  5. casken

    casken Well-Known Member

    Joined:
    Oct 17, 2004
    Messages:
    6,478
    Oh, these are fun! I'm going to do this. :fan:
     
    gkelly and (deleted member) like this.
  6. aftershocks

    aftershocks Well-Known Member

    Joined:
    Dec 8, 2009
    Messages:
    4,682
    Sounds like a really fun game for numbers crunchers and skating fans. Have fun playing pretend figure skating judge for the fun of it. This kind of fun FSU game doesn't actually prove a thing, as it is not reality-based. Even despite my suspicion of ISU judges and judging system, I believe they do have to undergo some kind of training (albeit kind of worthless when some of them didn't even realize they were supposed to take a deduction from Amodio's rap lyric lp at 2011 Worlds). What the judges do seem to realize and enjoy the benefits of, is the fact that internationally they are anonymous and therefore protected from any public challenge regarding how they happen to score skaters. The system of judging is and has always been politically-based, and the scoring now under CoP is even more manipulable -- lots of times PCS scores make no sense, and that is one area where the judges manipulate the hell out of the scoring, believe it or not.

    Sure, a lot of the time, the results seem fair, but in every major competition, there are placements that are questionable. At Olympics 2010, e.g., come on, re U.S. pairs Evora and Ladwig being so low-balled and placed behind two Canadian pairs teams that put out crap programs (both teams are no longer skating). Now at least when the latest two Canadian pairs teams are in front of Evora/ Ladwig, it can be viewed as a fair result.

    I'll say it on behalf of CoP, it certainly has gotten the recent generation of fs fans/ viewers excited and engaged about the technical aspects of the sport, which can be good to a degree. But to believe that the scoring and that figure skating in and of itself is inherently all above board and not based on politics, gossip, and feds making deals with each other and with the ISU... oh well. Perhaps, skating fans are so used to the politics and cheating and denial of cheating that is so ingrained as a part of fs, that it is like a gimpy left leg, or an annoying Great Aunt, we put up with it, are used to it, and therefore accept it as a given, and sometimes do all we can to defend what is, and what we know will never change.
     
  7. aliceanne

    aliceanne Well-Known Member

    Joined:
    Oct 12, 2010
    Messages:
    2,251
    Tony Wheeler had an interview with a recently retired Belgian judge. The judge claimed that he gave average scores to a skater based on how he ranked them compared to the other skaters. For example if the maximum component score was 8, and he thought skater B might outskate skater A, he would give skater A no more than 7.5 no matter how well skater A did. Furthermore, this judge saw nothing wrong with this and saw himself among the few (10 percent in his estimation) honest judges.
     
  8. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,669
    OK, judges, here are the calls Jenna and I agreed on for the elements we're using, that the judges would see during the competition. No level calls shown.

    If you've already sent me your marks and want to make any changes based on these calls, you can do so before the marks are finalized. :)


    Butyrskaya
    3Lz+2T
    3Lo
    LSp
    2A
    FCSp
    SlSt
    CCoSp

    Hubert
    3T+3T
    3F
    FCSp
    LSp
    SlSt
    CCoSp
    2A

    Gusmeroli
    3Lz+2T
    3Lo
    LSp
    CCoSp
    2A
    CiSt
    FCSp

    Nikodinov
    3Lz+2T e
    FCSp
    3F
    CCoSp
    2A
    CiSt
    LSp

    Suguri
    3Lz+2T
    3F
    LSp
    FSSp*
    2A
    CiSt
    CCoSp
     
    Last edited: May 11, 2011
    LynnW and (deleted member) like this.
  9. casken

    casken Well-Known Member

    Joined:
    Oct 17, 2004
    Messages:
    6,478
    You forgot Hubert's axel. I'll send my marks in soon!
     
  10. skateboy

    skateboy Well-Known Member

    Joined:
    Jun 12, 2006
    Messages:
    4,303
    Hope we haven't forgotten about this game!
     
  11. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,669
    Three people have sent me scores privately. Should we wait for any more?

    Then I'll have Jenna calculate the panel's results.

    And we, or the invidividual judges, should also calculate what their rankings would have been for these skaters if they were the only judges, but first they should let us know how they were trying to rank the skaters. Then we can see how much influence the base marks determined by the tech panel have compared with the judges' scores.
     
  12. Jammers

    Jammers Well-Known Member

    Joined:
    Oct 24, 2010
    Messages:
    3,878
    The bias in skating today is nothing compared to the days of the cold war when the Soviet judges and their stooges would always vote for a Russian or East German skater or gymnastist. I'm almost glad the US boycotted the Summer Olympics in Moscow. I can just imagine the biased judging for the eastern bloc nations.