GOE factoring, what do you think?

Discussion in 'The Trash Can' started by l'etoile, Oct 21, 2011.

  1. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    ISU Communication 1611
    Single & Pair Skating - Scale of Values, Levels of Difficulty and Guidelines for marking Grade of Execution
    http://isu.sportcentric.net/db//files/serve.php?id=1862

    When ISU released their revised rules one of which was that GOEs would be factored down to 70% of original grade which I still think ridiculous much, I assumed that points scored in general would be hardly higher than previous seasons except when skaters deliver something really extraordinary on ice.

    But as it's turned out obviously, my assumption was wrong and every time I opened the protocol sheets after competitions I was perplexed at what I was looking at. It was like judges seemed very determined that they would give as much GOEs as that it feels nothing's changed in factoring for them.

    Like, "Well, they got goe of 1 last season, so if we were to make it 2 from now on they would still get the same mark! Or better yet, let's just give them 2s and 3s. It's already too complicated to check every time."

    We were seeing more 2s or 3s than ever before, some of them raising the question mark if the skaters really did deserve those marks. When there are clear, detailed guideline as for what kind of execution should receive how much. Take it from ladies field last season, there were no apparent/aggressive per se technical change or advance especially in top ladies and their technical scores were not lowered, even in more occasions, they were higher.

    Is it really reasonable to assume that those ladies have a significant "improvement" in their technique, skating skills in just six months? Or is it that there are clear irregularities in judges calls? Or am I just being a paranoid nuts here?


    I have other concerns, more of as complaints to why judges would give 0s or even positive GOE when "some" skaters have falls, down-grade calls, under-rotation calls and wrong-edge calls on jumping elements from technical panels or obvious visible error in their execution of elements like step sequences, spins etc. Are they botching them so beautifully that judges somehow don't get willing to give negative goes which they're supposed to do-I had wrong information on this- to them? But it's already a big enough discussion I made so I'll take that aside for now.


    So, to sum the question, what are your opinions about current judging patterns for GOE? Do you think they're just fine or they have some concerns? Then what kind of concerns and what kind of suggestions would you make?

    I recently read a post (can't remember which thread) that a skater suggested that ISU should introduce an robotic/automatic system in scoring program which generates automatic -3 goes for falls. And I think it can be applied to other errors in elements. Kinda neat, wouldn't it be?


    -------------

    oh no. This was meant to be thrown in the trash can. Could admin please move this to Trash can? Sorry for the inconvenience:(
     
    Last edited: Oct 21, 2011
  2. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,750
    The clearer guidelines for positive GOEs are only a couple years old. I think there has been a push in the last couple years to encourage judges to use the +2s and +3s where warranted, so maybe it's finally sinking in and more judges are becoming brave enough to go out on a limb with those high scores for elements they consider to be exceptionally well done.

    I think the push to use the higher GOEs was probably strongest going into the 2010 Olympics, when the highest quality skating was expected (and in some cases delivered) and when there were more casual viewers watching who would be excited by high scores.

    I think you'd need to give some specific examples for us to look at. A general trend could result from many different causes -- if we're going to try to read the officials' minds to guess where the high scores come from, it would help to look at the actual scores and the actual elements they were give to.

    Are you talking about one isolated judge per element in a few isolated instances? Those are likely data entry mistakes, or possibly mistakes caused by the judge not seeing the element as clearly as the rest of the panel.

    If you're talking about a majority or large minority of judges on a panel giving good scores to elements that you think are bad, maybe the elements aren't really as bad as you think, or the rules aren't as strict as you would like.

    Look down the right column of the negative GOE guidelines on pp. 12 and 13 of the document you linked. There are some kinds of errors that are supposed to get -1 or -2 off from what the GOE would have been otherwise but according to the rules, like it says right there at the top of the column, are not restricted -- not required to be negative. GOE of 0 or even +1 is allowed with a < call or an e call now that the same symbol is used for both wrong edge and unclear edge.

    GOEs for elements with falls and jumps with << downgrade must be negative, but they are not required to be -3; -1 or -2 would be acceptable within the rules.

    If you can find elements with << and/or falls that received any GOEs of 0 or better, those would be incorrect scores on the part of the judges, quite likely inadvertent mistakes. Otherwise, look past the error and try to find what positive bullet points the element might have demonstrated to offset the reductions.

    Give specific examples and we can try to figure out which is which.
     
  3. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    thanks gkelly! I guess I should have put more thinking and clarification in my post before starting a thread. I'll look into accurate details and try to have some more input soon.
     
    Last edited: Oct 21, 2011
  4. Ziggy

    Ziggy Well-Known Member

    Joined:
    Oct 7, 2002
    Messages:
    20,569
    GOE scoring makes me sad. :p
     
  5. RumbleFish

    RumbleFish New Member

    Joined:
    Feb 11, 2010
    Messages:
    1,004
    IJS has lost all credibility. It's no more than a bunch of fuzzy numbers to hide politics going on backstage. It has added obscureness in addition to lack of accountability that figure skating judging already had in place.

    Personallly, I don't even bother to look at the protocols anymore.
     
  6. victoriaheidi

    victoriaheidi New Member

    Joined:
    Aug 17, 2010
    Messages:
    1,784
    Agreed, but I do look at protocols (especially if something seems way off, like someone got a much lower score than I expected. Or if I know I won't get to see the program for a while).
     
  7. RumbleFish

    RumbleFish New Member

    Joined:
    Feb 11, 2010
    Messages:
    1,004
    I guess I have even less respect for ISU establishment than you do. I really don't care about marks given to skaters anymore, nor placements determined by such. I just try watch performances and give my own versions of placements.
     
  8. bek

    bek Guest

    I'm beginning to think GOE needs to be done away with just in general. Its sad because in general it really is a great idea. But its soo freaking abused, that I'm just tired of it. T hey should perhaps have another mark where the judges could maybe put in general an overall score for the quality of all the jumps, spins etc.
     
  9. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    Just finished analysis on skater example A. Names and nations of skaters will be unknown. I'll keep on doing on other skaters. (It took me one hour to analyze one skater so I can't to this on every others. I have two or three more male/female skaters planned.)

    skater A.
    Protocols just for technical components in SP from 09-10 to 10-11
    Protocols for technical components in FS from 09-10 to 10-11

    Same jumping contents throughout 09-10 to 10-11.

    Total GoE received in short programs in the order of competition he/she appeared.
    *Points scored at the Olympics where GoEs were higher than ever before then was bolded.
    *Points scored with clean performance were italicized.

    09-10 :
    0.58(downgraded combo 3loop, downgraded 3flip) ,
    -0.8(fall on 3flip, landing mishap on combo, got level 2 in layback),
    4.7,
    2.7(downgraded combo 3loop),
    2.2(failed to complete combination, 3 lz was downgraded)

    10-11 :
    0.49(stepped out 2a, under-rotated combo 3loop),
    0. 93(under-rotated 3flip, level 1 layback),
    -0.83(downgraded combo 3 loop, downgraded 3fil, clear mistake in flying sit spin got level 2),
    7.29,
    6.5


    Total GoE received in free skates in the order of competition he/she appeared.
    *Points scored at the Olympics where GoEs were higher than ever before then was bolded.
    *Points scored with clean performance were italicized.

    09-10 :
    4.18(downgraded combo 3toe),
    3.5(downgraded 3 lz, downgraded combo 2loop) ,
    5.84(downgraded 3 salcow),
    5.3,
    8.24(mistake on flying sit spin got level 2, downgraded combo 2loop)

    10-11 :
    5.81,
    6.46,
    7.68,
    13.43,
    9.06
    (stepped out combo 2toe)

    The skater's overall GoEs were higher than previous season's even under the same circumstances of performances both clean-wise and number-of-mistakes-wise. Note the skater's scores between olympics and competitions in 10-11. (the former will be considered as the competition with highest GoEs of skaters before ISU's revised rule came out.) A weird "growth spurt" in GOEs at fourth competition in 10-11 season is something I'd rather ignore. If one to argue that maybe this skater's technical skills have enormously, obviously improved, I've got no idea what to argue against that:p

    Call me anything, I'm just trying to specify examples of current errors I see in GOE system.
     
    Last edited: Oct 22, 2011
  10. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    I know that as a judge there has definately been a push to really use the positive GOE guidelines. At the last couple of seminars I have attended we have certainly been encouraged to apply positive GOEs when warranted. And I find it makes things a lot easier in looking for the positive in elements.

    So you are correct in that you are seeing more positive GOEs in the last couple of years.

    However the question I would ask - is that such a bad thing?

    It is not a static system. Every year the ISU do try to fine tune and make changes to improve it.
     
    Poggi and (deleted member) like this.
  11. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    Well let's just go back to the 6.0 system then and keep you happy. Doesn't matter that it really was pretty crap way of judging because it basically is just a placement system. And then that will really give fans something to complain about because there will be no more transparency in the judges results.
     
  12. bek

    bek Guest

    In someways its transparent but in someways its not. What judge gave what score again? Not to mention how easy it is to hide behind numbers. Instead of owning that you put someone above the other.

    I don't want to go back to 6.0 and I like a lot about IJS, but the one thing I miss about 6.0 is that the skaters pretty much had to deliver, because the judges couldn't justify putting Michelle Kwan above Tara at 1997 Nationals, when Michelle was all over the place. Now under IJS, we could have a very different result. Thanks to much higher PCS for Michelle for skating skills, choregraphy etc. Now sure one could argue its deserved but is it really? Shouldn't the fact that Michelle was a mess matter???

    What I miss the most about 6.0 is the concept of a whole program being evaluated and that includes how well said skater executed their program. Now everything is in peace meal, and I don't think that's right. The whole of the program technically should be evaluated and scored. The fact that the person Zambonied all over the ice should significantly affect their results (And Chan's results say it doesn't!)

    The thing is I don't know how anyone can say reputation doesn't affect PCS/GOES. Or politics plays no role, this is a judged sport it does, which is exactly why the fact that PCS/GOES can be so easily inflated, that should be concerned, and exactly why multiple mistakes need to be significantly penalized more.

    Even if you want to argue that the top skaters earn all of their PCS/GOES once could point out that it takes time to get top skater PCS/GOES for the most part. And that PCS can give some such a significant advantage that multiple errors are allowed.

    And I'm sorry your falling all over the place it affects the quality of your performance to the average viewer.

    There needs to be room for upsets in this sport, there needs to be room for movement. And the more IJS involved it seems like there's less room for these things, its a problem. Especially since room for movement was one of IJS selling points before.

    I'd just like to see the system have a balance where jumps, choregraphy/transitions/skating skills/artistry/execution are all rewarded. All of it. Not convinced IJS does that one, at all. And it would be easy to institute more mandatory penalities for messy programs, and it would lead to less controversial results..
     
    Last edited by a moderator: Oct 22, 2011
  13. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    It's not a bad thing at all when skaters deserve those marks and overall scoring line is equally augmented.

    But is it okay that now those "guidelines" are evoked, GoEs now play more politically open role. Skaters who had definite limited line PCS-wise because of their abilities are now receiving insanely high goes to make that gap which used to be there more vague. Hello, it's already done by PCS, which most of ppl are tired of it being abused. Did they really deserve them? Did they really have "that" excellent execution on those elements?

    It's about judges being allowed to use GOE as push in an unfair way.
     
    Last edited: Oct 22, 2011
  14. bek

    bek Guest

    Exactly, its the fact that PCS/GOE can be so easily manipulated. And the fact that penalities for messy programs aren't really as punished in relation. The fact that they are making rules like we dont' have to give full minus 3 for a fall!

    GOES/PCS would be fine if they started instituted harder rules for poor execution. I mean thats the main thing I mess not only when it comes to jumps but also footwork, spins etc. Well executed elements.. Not piling on transitions that they can't do because that gives them GOE and PCS points, etc etc.

    I mean Aussie your acting like we are asking for the moon, but asking that gasp Performance/Execution marks get hit hard when someone falls four times...Is it to much to throw the casual viewers that bone, so that the sport is a little understandable. Its one of the few major errors that anyone and their brother can see. But the system acts now like its no big deal. You fell four times and 8s for you. And I don't care how well you acted, smiled afterawards, for a lot of us, the performance IS affected.

    Is it to much to say that falls/major errors in this system of multipe GOE points/PCS aren't penalized enough given some of the results we've seen...
     
    Last edited by a moderator: Oct 22, 2011
  15. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    ^^this.
     
  16. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    Not is not. It is about educating judges and trying to actually get the full range of GOEs used.

    6.0 judging was all about deductions and looking for the negatives in what skaters were doing. It has actually taken a long time for judges to change their way of thinking about looking for positives in elements. That is why they came up the guidelines for positive GOEs and that is why you are seeing them used more. Because if I remember correctly when the system came in there were quite a few complaints on this forum about skaters not being rewarded for doing great elements. If anything I think what you are seeing now is what it should be.

    But I suppose that being involved in the sport and having seen and applied the changes, I am not really qualified to comment on how things have developed, am I?
     
  17. bek

    bek Guest

    Aussie, I can see how this is beneficial and I don't mind that great elements shouldn't be rewarded. But are you involved in the international level, though? Because at the international level, I'd imagine there's a whole new ball of wax involved, and a whole level of poltiics involved.

    And GOE makes it so easy, I give this person a plus 1 here, etc etc. You can't educate people if they want to cheat/play politics. There's no educating there. There's no way to stop politics, collusion. So to just give the judges complete wiggle room... Just makes the sport questionable.

    I do like the idea of great elements being rewarded. But why shouldn't a skater who went out there and perhaps didn't have the highest jumps but still had solid jumps with every kind of jump in the arsenal, also not get rewarded over a skater who perhaps had some really quality jumps, but not the highest arsenal?

    And well yes maybe someone maybe landed 3 really quality jumps and then had 2 falls. Should that person really be making up all kinds of points over the person who went out there and landed 5 solid, clean jumps. While yes the quality jumps should be rewarded, shouldn't the fact that the person with the five solid jumps delivered a solid, clean performance also matter in the technical mark?

    You can say all you want well said skater lost 12 points because they fell, but its hard to see that they are really getting penalized for their big errors when one sees their TES versus others.

    Great elements SHOULD be rewarded, but if your going to give judges leadway to reward great elements, than you need to also force some much harder penalities for bad elements. Thats all people really want, which is a balance.

    Why should only pieces be rewarded but not the entirety? It seems to me both should be rewarded. The quality of each element, and then the total technical package all together. Wouldn't doing both lead to a better/more fair result?

    Allowing the judges to look at programs as a whole technically would be helpful too because they could say okay yes Laura Lepisto had no falls on the jumps, but she did mainly doubles. And yes Miki Ando had a fall on a triple, but she did triples not doubles. Piece meal doesn't do these things.
     
    Last edited by a moderator: Oct 22, 2011
  18. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    Aussie you're totally qualified to comment. It's also great to read ppl's opinions who are involved in the matter. It helps to figure how they see it and what they're focused on and people outside can assess/make suggestions more validly.
     
  19. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    Thanks. I understand that people do get frustrated with what they see in the sport. But sometime I feel that the criticisms are unjustified. And this case there has definately been a history of development with how GOEs have been marked. In the last 12 months particularly it has been heavily talked about giving more positive GOEs for elements and we are constantly reminded about it. But it has to be within the guidelines.

    Figure skating is such a technical sport and there is so much to learn. Whether it be those who are developing the rules or those applying them.
     
  20. RumbleFish

    RumbleFish New Member

    Joined:
    Feb 11, 2010
    Messages:
    1,004
    No offense to you in particular, but this is what I find so despicable about the ISU establishment and the wretched judging system. All they do is hiding behind fuzzy numbers trying to rationalize their corruptive decisions.

    You have said judges have been instructed by the ISU to give more +2s and +3s in marking GOEs.

    There should be no problems if same principles are applied to skater A and skater B. But the reality is, only skater A gets truck loads of +2 and +3 GOEs, and skater B's marks stays the same.

    Now, I'm not so naive to believe the world should be fair and everyone should be treated the same. All I want to say is that before anonymous judging got started, judges were at least accountable for marks they were giving, and received booing when it was deserved.
     
  21. RumbleFish

    RumbleFish New Member

    Joined:
    Feb 11, 2010
    Messages:
    1,004
    BTW, please don't start saying stuff like how judges are volunteers who run on thin budget.

    I personally don't sympathize so-called volunteers who can't claim jobs they have done.
     
  22. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    It is not about being instructed to give more +2 and +3 GOEs. It is about applying the bullet points as per the guidelines.

    So if the skaters are doing elements that fulfill the requiremnts of those bullet points, then they are going to get the better GOEs. And it results in more +2s and +3s.

    For example - if a skater does a jump that:

    - has good height and distance
    - is effortless
    - is matched to the musical structure
    - has a difficult entry

    there are 4 bullet points which then means the skater should get a +2.

    To even get a +1 these days, all a skater has to do has good height and distance and do it on the music. I see skaters are lower levels who can do a single axel who can get a +1 just by doing those things.

    However if people want to believe the system is totally corrupt and unfair, nothing I can say here is going to change that belief.
     
  23. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    Actually if it wasn't for the volunteers (and not just the judges but everyone else) you wouldn't have a sport to bitch about. :D
     
    Last edited: Oct 22, 2011
  24. l'etoile

    l'etoile New Member

    Joined:
    Nov 30, 2010
    Messages:
    4,891
    The reality is Aussie, that those examples you mentioned are hardly what we've seen last season.
     
  25. RumbleFish

    RumbleFish New Member

    Joined:
    Feb 11, 2010
    Messages:
    1,004
    Oh, I deeply appreciate volunteers, only those who account for what they have done. ;)
     
  26. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,106
    Well that is nice to hear. I am sure all the volunteers will be so thrilled to know that. I will make sure I let them all know.
     
  27. RumbleFish

    RumbleFish New Member

    Joined:
    Feb 11, 2010
    Messages:
    1,004
    Whatever. :shuffle:
     
  28. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,750
    I suspect you would hate that even more and find abuse there because judges would still reward overall quality more than they would penalize a few errors that stand out to you.

    I'm going to give a hypothetical example of a short program. Let's assume the jump content and the levels are the same.

    Skater P
    axel jump very good (+2)
    solo jump hand down (-1)
    jump combo good first jump with difficult entry, fall on second jump (-2)
    combo spin very good (+2)
    solo spin outstanding (+3)
    flying spin fall no credit (-3 but only dashes will show on the protocol)
    step sequence very good (+2)

    That's two serious mistakes and one smaller one in seven elements, two fall deductions, complete loss of base mark for one element, but the overall GOEs under today's system, from this judge, would add up to +2, not accounting for factoring. So let's say that judge gives an "overall score for the quality of all the jumps, spins etc." equivalent to +2.

    Skater Q
    axel jump fine (0)
    solo jump underrotated (-1)
    jump combo edge call on first jump, downgrade on second jump (-3)
    combo spin recenters with awkward positions (-2)
    solo spin OK (0)
    flying spin weak air position, slow, short of revolutions (-2)
    step sequence OK (0)

    No disruptive mistakes -- you don't even notice some of the technical errors. Looks like a clean program to you. And Q skates with lots of energy and you prefer Q's personality and style over P's. So when Q ends up with an overall score for the quality of all the jumps, spins etc. equivalent to -8 for a clean program, compared to P's +2 for a program with two falls, all you see is politics. What the judges saw was weak technique.

    At least with individual scores for individual elements you can see where P's points are coming from and that P is indeed being penalized for the errors. With one overall score it's a black box with practically no information given, so you're free to make up your own stories that have nothing to do with the judges' actual thought processes.

    Examples? Can we look at specific elements to apply the bullet points?

    Quite likely, when the culture of judging GOEs was more cautious, many judges gave both skaters +1 for their good elements, even though skater A's elements were clearly better than skater B's. The real difference in quality wasn't being reflected in the scores. Once the culture changed to encourage +2s and +3s, those judges felt free to give skater A the scores s/he really deserved, whereas skater B continued to receive the +1s that s/he always deserved.
     
  29. Ziggy

    Ziggy Well-Known Member

    Joined:
    Oct 7, 2002
    Messages:
    20,569
    How about training the judges instead so that the quality of judging improves, instead? :p

    There really isn't the need to throw the baby out with the bathwater.

    Oh and when I say train the judges, what I mean first and foremost is psychological training.

    So that judges are aware of all the psychological mechanisms that are in place and are able to make rational and well throught through decisions.

    Because at the moment, judging is the equivalent of a group of lemmings jumping off a cliff. :scream:

    (And the idiotic corridor rule is a part of this problem)
     
    Last edited: Oct 22, 2011
  30. bek

    bek Guest

    But I"m fine with someone getting hit on weak jumps scores if they have underrotated jumps. I still think individual elements should be scored and judged.

    But what about the skater who perhaps may not jump like an Oda or a Yu-na Kim. But said skater has jumps that are fully rotated, and executed correctly. And has a complete arsenal. Shouldn't said skater win the jump mark over a skater who may have some quality jumps in there, but also has falls in jumps, and not a complete arsenal of jumps? At the end of the day the person with the high jumps may normally be the better jumper, but their jumps were not there that day. Now IJS just hands huge points for quad falls etc. And GOE can hand back points. But the fact of the matter is one skaters jumps were all over the place and the other skaters jumps were not.

    If the system were to reward the individual elements and than have a part where the program as a whole could be judged technically, I think we could have both and even the explanation for why said skater got a high mark for jumps/why some skater did not.

    And I still think huge falls though in their own way are a bigger disruption than downgrades. I want downgrades punished but the disruption multiple falls makes needs to be factored into things.