PJ Kwong: What is wrong is the failure of some to learn how the sport is scored

Discussion in 'The Trash Can' started by Maofan7, May 4, 2013.

?

Is PJ Kwong right?

Poll closed Jun 4, 2013.
  1. Yes

    78 vote(s)
    56.5%
  2. No

    60 vote(s)
    43.5%
  3. Don't Know

    0 vote(s)
    0.0%
  1. The Accordion

    The Accordion Well-Known Member

    Joined:
    Sep 21, 2005
    Messages:
    3,416
    And when that happens - I think everyone is happy! But how do you mark when none of the skaters have a perfect program? Also I think there are a few programs that most of us can agree upon as being flawless and perfect. But those are rarities. What makes up a perfect skater? I don't think there has ever been agreement on that!
    Maofan7 and (deleted member) like this.
  2. karlon

    karlon New Member

    Joined:
    Jan 7, 2013
    Messages:
    163
    ITA.
    And I want a champion who can win in any system...
  3. aftershocks

    aftershocks Well-Known Member

    Joined:
    Dec 8, 2009
    Messages:
    4,482
    Well, at least we have you gkelly to bust through all the generalized proclamations, and to get at the heart of our individualized self-important idiotic ways of thinking, if not exactly to precisely fix what's wrong with figure skating. :p

    Have you ever thought of becoming a trivia question writer on Jeopardy?

    My above comment and question are meant with all due respect to your knowledge and positive intentions and desire to educate us and to make us truly reflect on the underlying implications of generalizing too much about the intricacies of the sport's symbolic and very real problematics. Perhaps you can forgive me and others for being so sullen and disrespectful and riding our own pretentious high horses, much less having the effrontery to offer thoughtless uselessly idealistic escapist throwaway solutions, or not providing any solutions at all. This whole conundrum is such a maze really, even that intelligent mice can't hope to escape. ;)


    Seriously, once anyone realizes that something isn't working, why keep forcing it on yourself and other people? Your questions gkelly, seem designed to prop-up the importance of maintaining the status quo.

    It's really way too late to try being reasonable and logical about the difficulty/ impossibility of fixing the mess, gkelly. The baby has long since been thrown out with the bath water after all. Where were you when Cinquanta and the ISU were f'ing things up in the first place?

    As usual, figure skating will change for the better or it won't. How, when, why, and if that ever happens or not won't stop fans from making what you feel are generalized proclamations.
  4. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    No, I'm not arguing in favor of blindly maintaining the status quo.

    What I'm arguing against is blindly throwing out the status quo without suggesting a viable alternative.

    I think we can all critique specific rules and specific results and try to brainstorm solutions within the existing system. I think that if any people with actual decision-making powers are reading, practical suggestions that could be implemented next year would have more chance of being adopted than starting from scratch.

    But if we do think the current system is irredeemable and it's necessary to throw it out and start from scratch, I think we need a big-picture understanding of what a good system for evaluating skating should aim to achieve and practicable suggestions for how to achieve that.

    Over the years I have posted many suggested changes to the existing system -- and some of the smaller ones have actually been adopted. Nice to know that some decisionmakers were actually thinking along the same lines.

    I've also posted proposals for hypothetical alternative systems. I would be much more surprised to see any of them actually adopted because throwing out a whole system and starting from scratch is such a huge step. And look how resistant people have been to the last time that happened.

    Even if you and I and the ISU agreed that a completely new and different system would be better than 6.0 or IJS, and it was put into practice, I guarantee that it would not work perfectly every time, especially at first, and that there would be complaints and calls from some to go back to IJS, from others to go back to 6.0.

    But I can come up with many different alternative approaches, some that I like better than IJS or 6.0 and some that I like worse. Which ones I'd recommend depend on what the goal of the system is.

    A system that best serves the needs of casual fans and would therefore bring in the most outside interest and outside money would likely be different from a system that serves the needs of knowledgeable fans, which might be different from what best serves the needs of elite skaters, or all competitive skaters, or athletically inclined skaters, or technically inclined skaters, or artistically inclined skaters, or large federations, or small federations, etc. So before we design a new system, first we'd need to decide which needs we're trying to serve.

    Just declaring that the current system doesn't work, without defining what "working" would entail, isn't going to get anyone any closer to a better system.

    So I'd rather see more analysis and brainstorming about productive ways to change the status quo, and less generalized complaining.
  5. aftershocks

    aftershocks Well-Known Member

    Joined:
    Dec 8, 2009
    Messages:
    4,482
    Well, gkelly, I doubt ISU is seriously looking to FSU for suggestions/ solutions to implement. ;)

    In any case, I don't see most people here saying "throw everything out" -- ISU is the one who already did that lock stock and barrel, and the scoring system and the sport are suffering as a direct result. I think TPTB need to think differently and to seriously examine the mess they've gotten the sport into, and also to reach out more to fans with more goodwill and with genuine proclamations that they are interested in effecting positive change instead of hiding behind the status quo. If they were actually interested in productive change that would benefit the skaters, if they actually did give a rat's a** about figure skating fans, there might be fewer fans haranguing and complaining and creating worthless polls about IJS.

    Lots of posters offer genuine ideas for solutions and thoughts about what is wrong. Sometimes people in the sport may get ideas from reading productive comments in forums, but IMHO, the ISU does not care what fans think. In any case, we've been told more than once here that the ISU isn't responsible for effecting change. Certainly with the same person at the helm of ISU for this many years, there's no room for fresh ideas and progressive movement. Change if it comes will be from the bottom up.

    Jeez re your wanting to see "less generalized complaining." Are you running a figure skating fan board meeting? :lol:

    Why not go back through all the many threads that have been on FSU recently and in the past regarding the scoring system. I'm sure you will find lots of informed analysis and constructive ideas and suggestions being imparted amidst the generalized snarking, b**ching and complaining.
  6. lala

    lala Well-Known Member

    Joined:
    Jul 12, 2012
    Messages:
    1,959
    I was wondering the viewers want to see that winner, who skated the most perfect programs in the current competition. For example that gymnast who wasn't flawless, won't win.
  7. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    17,988
    I don't think it is a matter of how many numbers are read out when the scores announced that would actually lead to any greater understanding of the system. They are numbers and the skater with the highest score at the end of the event wins. I think that is how most casual fans view it.

    For myself, I watch other sports that I don't have much knowledge of about how it is scored and at the end of the day I particularly don't care how the result is obtained or who is the champion. I watch it because I enjoy it. This includes judged sports such as diving, gymnastics, rythmic gymnastics. I only have knowledge about figure skating because I am very involved in the sport. But I think even if I was a casual fan of skating, at the end of the day I probably wouldn't give a rats about the result at the end of the day. The thing that is going to stick out in my mind are the programs I like. Even before I started skating myself and got myself very deep in judging and administration, that is how I viewed it.
  8. VarBar

    VarBar Well-Known Member

    Joined:
    Dec 8, 2008
    Messages:
    1,187
    This man gymnast wasn't flawless here and did lose the vault but won overall and was the Olympic champion in the individual all-around event.:) I could give you yet other such examples but I don't think they would change your mind.;)

    https://www.youtube.com/watch?v=sNnzqx7SXUk
    spikydurian and (deleted member) like this.
  9. The Accordion

    The Accordion Well-Known Member

    Joined:
    Sep 21, 2005
    Messages:
    3,416
    But my point is what is the definition of "flawless"?
    For many this just means no falling. For others a fall-less program can still have flaws.
    Does it mean no doubling or leaving out a 2nd jump?
    Does it mean no deducted levels or negative GOEs?
    Does it mean a program with good transitions - because some would see a program with less than adequate transitions to be flawed.
    If someone has a slight bobble - does that take away the flawlessness?
    If the program is far superior to another skate in that competition but the skater touches their hand down after a jump - should the "flawless" inferior program win?
    Should less than inspired choreography be considered flawed?
    Can someone with a bad leg wrap skate a flawless program?
    If someone lands all of their jumps and completes all their elements and makes no "mistakes" but does not connect in any way with the audience - is their program flawless?
    What about someone who skates their very best and nails all the elements but skates very slowly with shallow edges?

    These are just some of the questions that have to be considered. It is not so black and white as you would suggest.
  10. Susan M

    Susan M Well-Known Member

    Joined:
    Mar 21, 2004
    Messages:
    1,448
    I voted no. For starters, P J Kwong is pretty much never right, so that was easy.

    I think it is fair to say that the judges (and certainly the skater) should not be blamed for a particular result when all they did was follow and apply the rules as written. That does not mean that there is nothing wrong with the rules.

    All that BS about unwillingness to accept change is just a smoke screen to avoid the hard questions. As mentioned upthread, the conversation doesn't need to be about dumping the new judging system, it can also be about making changes within its framework to make the sport more appealing and more comprehensible to audiences. The ISU technical committee itself has seen this deterioration of the artistic aspects of the sport and has already made some changes to try to steer choreography back in a better balanced direction. (For example, elimination of levels for the ladies spiral seq and second men's fw seq was a response to the visual clutter and ugly positions skaters were using in quest for higher levels.)

    IMO, I think it highly likely we will also see them move on the question of Program Components and find a way to tell judges to make sure they are giving marks based on the totality of the program and not looking at components in isolation. Sandra Bezic recently used the "missing the forest for the trees" analogy and I think that is pretty apt.

    So, no Kwong is not right to think the only problem is people not understanding the rules. There are still problems in the rules themselves.
  11. Zemgirl

    Zemgirl Well-Known Member

    Joined:
    Jul 13, 2009
    Messages:
    4,833
    I think a case can be made that the sort of things the IJS emphasizes are less likely to produce programs that would appeal to casual fans. The advantage of the system is that skaters know what they need to do to score well and where they can improve, so they can plan their elements and structure their programs accordingly. The problem is that (IMO) this has led to less variety in the programs than we saw in the past, because there is only so much one can do and still get good marks; and I feel that there is now more of an emphasis on the technical side than on certain aspects of the presentation. I'll happily sacrifice some transitions for perhaps easier but more aesthetically appealing choreographic content.

    That said, I don't think a return to 6.0 or dumbing down the requirements is the answer, but I do feel that the system has become too constraining, and perhaps some things should be reconsidered. Maybe dropping spin and step levels, for instance - choreographic step sequences don't seem to have led to a drop in quality, and scoring other elements based on quality and execution rather than level features might allow for more variety and originality.
  12. lala

    lala Well-Known Member

    Joined:
    Jul 12, 2012
    Messages:
    1,959
    Wrong example, the other men did more mistakes. And I remember Nemov! http://www.youtube.com/watch?v=izuJEIS5P68 :mad::angryfire
    In London 2012 my favorite was Mustafina, but she did mistakes and Gaby Douglas won. Great!!!
  13. lala

    lala Well-Known Member

    Joined:
    Jul 12, 2012
    Messages:
    1,959
    I understand you, but I can't answer your questions. But I'm sure the audience don't want to see such a champion who did falls and some major mistakes. The judges must be punished for making mistakes! When the choreographers made ​​the programs, they need to know that the skaters likely able to skate flawlessly the program in race conditions. And I don't think that the falls don't spoil the artistic impression...sorry, or the interpretation and the choreography, etc..
    Last edited: May 6, 2013
  14. Eyre

    Eyre New Member

    Joined:
    Jul 11, 2007
    Messages:
    321
    Actually, it's Patrick detractors in general focus their attentions on falls as if falls should determine the outcomes. Also each category name under PCS has given people excuses (maybe that's one of IJS's faults) for not studying and going into the contents of those categories. People fail to relate each category closely to figure skating. Instead, they consider these categories in general non-skating terms which they know long ago.

    I'm not saying that IJS is perfect. It does need some changes and tweaks. But changing IJS has been used by many to try to make the kind of skaters they would like to win.

    Especially the old skating fans because they are stuck into the past and cannot accept the new system.

    Wait till after Sochi when Plushenko retires, big change might come.;)
    spikydurian and (deleted member) like this.
  15. leafygreens

    leafygreens Well-Known Member

    Joined:
    May 26, 2009
    Messages:
    1,648
    The problem is that the general public is not going to see or understand edges and skating quality, which count significantly towards the score (and should.) This is anecdotal, but I can be at a public session and a beginner skater will beg me to do a "trick", so maybe I will do a power pull into a rocker-rocker and they ask me to do something "harder" like a "triple axel" or "iron lotus." The non-skating public wants to see triple jumps landed perfectly with no falls and I honestly think nothing will ever change that. It's all about the visuals for them and the same reason figures were cancelled. I can't even see edges on TV and I'm a skater myself. Is this the fault of the commentators or the skating federations to not educate the public? Not sure. That being said I think IJS is a huge improvement over 6.0 even if I don't understand everything.
  16. Eyre

    Eyre New Member

    Joined:
    Jul 11, 2007
    Messages:
    321
    Exactly! IJS is so much better than 6.0.

    People favor more on visual effects, such as cleanness and better upper body movements. Therefore, conflicts between casual fans and the experts are sometimes inevitable. It seems to me, the only resolution is that if casual fans cannot bring themselves closer to the expert level of understanding the system, they should just learn to accept it and ignore the results. In the meantime, enjoy the skaters you love regardless the placements.
    Last edited: May 6, 2013
  17. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Ha.
    Even at a more realistic level, a layperson might be more impressed by a mediocre single axel than by a rocker-rocker, but the latter is certainly a higher level skill.

    Which means it would be helpful if there were some means -- especially on television -- to help viewers recognize and appreciate those skating skills that happen primarily from the blade down but that are fundamental to what makes figure skating figure skating (as opposed to, say, gymnastics or dance).

    Both, I would say. The ISU and the national federations could make written materials and videos available directly to the public -- some free online or free handouts at competitions, others available for sale at reasonable prices. And make information directly available to journalists and to broadcast networks.

    Then it would be up to the networks to pass along that information in a format that enhances their broadcast and engages the viewers. They could also produce educational materials of their own, initiated by their expert commentators and consultants.

    I think the most important thing is for the federations to invite the fans into the process rather than holding them at arms length and assuming they all want to remain ignorant and just buy expensive tickets to see pretty skating.

    And for the media to frame their coverage with a basic respect for the evaluation process and respect for fans' ability to understand it, rather than setting them up as in opposition to each other, e.g., a skaters-vs.-judges conflict with judges as the bad guys and fans on the side of the skaters who "should" win but often don't.
  18. wonderlen

    wonderlen New Member

    Joined:
    Sep 12, 2007
    Messages:
    600
    The MAIN PROBLEM with COP is that the judging is inconsistent on things like <, <<, levels on spins and footwork and deduction on fall and/or what constitute a 'fall'. Bigger problem is the PCS part of COP. Sure i mean the judges give score from 1 to 10, but the rules doesn't really say what is 1, what is 2, etc. Its pretty much on all relative and comparing a skater A against skater B. Meaning skater A will always score higher PCS mark regardless of how he/she performs in actual competition, and skater B will be always marked less.
  19. karlon

    karlon New Member

    Joined:
    Jan 7, 2013
    Messages:
    163
    deleted
    Last edited: May 6, 2013
  20. lala

    lala Well-Known Member

    Joined:
    Jul 12, 2012
    Messages:
    1,959
    No, you are wrong. I don't like the new system, because the programs are so similar, I don't like the meaningless TRs, because the programs become too chaotic, annoy me. I adore Plush, but I also adored the FS, since.....very very long time ago. You believe me, Plush isn't the reason, why I don't like the new system. Even more so, because he won some competitons and won a silver medal not too long time ago, under the new system. I think, you perfectly know this, but you wanted to be a little hostile with me.
  21. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Here's what the ISU has to say about what is 1, what is 2, etc.:
    http://www.isu.org/vsite/vfile/page/fileurl/0,11040,4844-152077-169293-64120-0-file,00.pdf

    I figure 5 = average is in the context of international-level competition. I.e., 5 is clearly senior level but nothing special beyond that. For younger skaters or those at lower skill levels, 5.00 could be an excellent score.

    This document also relates the numbers to the percentage of the program time that the skater demonstrates each of the criteria.

    I don't find it as useful to think of in those terms. One skater may be pretty good at performance or at interpretation for 100% of the program with no real breaks and no real highlights. Another might be excellent for 80% of the program and only average for the other 20%. How do you translate that difference into actual scores?

    In theory, no. In practice, if skater A is a stronger all-around skater than B, then even when A has a bad day and B has a good day, A's score will and should still be higher. That's the system working correctly.

    If there is not much difference between them to begin with, then when A has a bad day and B a good one, B should score higher, at least in the areas where B does well and A does poorly.

    Reputation effects might come in and prevent that from happening, which would be the system not working correctly.

    The trick is to recognize the difference between occasions when A is still better and occasions where B deserved to score higher. How much do A's struggles on this occasion cancel out superior basic skills, superior program construction, etc.? Often it's a judgment call and different judges, different fans, would disagree with good reasons on both sides. Under 6.0 in those situations, we'd probably see mixed ordinals.
  22. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    17,988
    I can do an upright spin in a public session and I get wows from those around me. Those who attend public sessions are generally impressed with anyone who can actually stand up.

    I want to see the video of your "iron lotus". :)
  23. Susan M

    Susan M Well-Known Member

    Joined:
    Mar 21, 2004
    Messages:
    1,448
    IJS is pretty much only "more popular" among some fans from countries whose skaters have enjoyed more success under it than they did previously and by some Canadians because they keep getting blamed for foisting it onto the world so they have to pretend its a good thing. (In other words, Canadians aside, it is, generally speaking, defended by people who haven't been watching skating for very long and seen as fairly flawed by folks who remember what skating used to look like and how beautiful it could be back in the 70s, 80s & 90s.)

    But as for which is simpler, I think you have it backwards. The 6.0 system was really pretty simple, especially during the period of majority ordinals. People could understand looking at two numbers and seeing who got more points and they can understand the idea of winning 6 judges to 3. (The vast majority of results did end up with pretty clear panel majorities.) It did get a little more obscure when they went to the OBO comparisons, but still most results came down to the winner getting the most points from more judges that anyone else.

    The current system is way more complicated and the proof is very simple. There is no way they ever would have entertained this system before computers. You have element values, then you have + or minus 1-3 GOE except 3 GOE isn't in fact always worth 3. Then sometimes the element is factored to reduce it by some fraction or to increase it by some fraction. Then you get to add up all that mess to 45 other numbers (some of which are also multiplied by two and 10 of which don't count anyway). How can you seriously argue this is simpler than 9 judges with two marks each? I think what you are really saying is that it is mathematically linear. That's not the same thing as simpler.

    I suppose for fans whose skating comprehension is satisfied by looking only at the skaters' total points, then it is pretty simple to see that one number is bigger than the other, but that isn't what audiences find confusing. Why the system seems complicated are exactly that ridiculously over-thought numerical minutia producing results that bear little relation to what they thought they just saw on the ice. Explaining the minutia is still not going to make people think the right skater won when that doesn't fit with their own subjective perception.
    Last edited: May 7, 2013
  24. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    17,988
    The marks given under 6.0 had nothing to do with a skaters getting more points. It was a placement and ranking system. It had absolutely no accountability nor transparency. And who knows how judges came up with their marks.

    Give me IJS over 6.0 any day. And I don't come from a successful skating country nor am Canadian. In fact Australia only had one competitor at Worlds this year because of the system. And I have been skating for 20 years and been a fan of the sport for longer than that.
    Last edited: May 7, 2013
    algonquin and (deleted member) like this.
  25. VIETgrlTerifa

    VIETgrlTerifa Well-Known Member

    Joined:
    Jul 12, 2003
    Messages:
    9,940
    One thing I miss about 6.0 is that the scores were fun to see. I used to imagine the nine judges were like the Justices of the Supreme Court and they were determining which way the competition should go. You'd see a unanimous decision, a majority-minority split, a plurality with splits with more than two skaters with a result coming out of that, etc.

    I can see how IJS will be fun for accounting majors and such. It is fun to try to calculate scores and add them up to see how skaters exploit the system to their benefit or how their failings to do so cost them, with sometimes the actual execution not mattering so much. It's an interesting observation to see how humans are trying to fit into a rigid, mechanical system that doesn't give much leeway outside GOE. Of course there's PCS, that many think is actually being used as a pseudo-ranking system by the judges. If you think about it, IJS is a more interesting study because it has a lot elements one can pick apart and critique or praise.
  26. Andrushka

    Andrushka New Member

    Joined:
    Nov 2, 2004
    Messages:
    3,114
    IMO the judging system and those who manipulate it to give medals to whom they favor even when they don't deserve it.
  27. spikydurian

    spikydurian New Member

    Joined:
    Jan 15, 2012
    Messages:
    2,684
    When we design a policy, we think about the reasons why we need it in the first place. I think professional skating competitions will serve some casual fans better since some fans prefer to look at skating as an art form rather than technical form. Any chance of bringing professional skating competitions back to serve the needs of of some fans?

    If that is so easy, there won't be biitching and whinging on skating boards. ;)

    Same here. But then looks like not all viewers are the same. I just think when one is too emotionally attached whether as a fan or for other reasons, these emotions inevitably marr one's judgement or conclusion on what they see.
  28. lala

    lala Well-Known Member

    Joined:
    Jul 12, 2012
    Messages:
    1,959
    OK. Look at the figure skating's popularity...
  29. rayhaneh

    rayhaneh New Member

    Joined:
    Oct 8, 2012
    Messages:
    452
    This ^

    This isn't about what's best for my countrymen/women. Being Swiss there is no one competing at the top at the moment from my country, and even if it were, I've never really cared about nationalities anyway, only about the skating they produced. And I don't feel some of my favorite skaters being, on occasion or more systematically for some, fleeced in PCs is the fault of the current system: this was also true under 6.0, in fact maybe even more so, and at least IJS gives you a better, more detailed insight into the marks increasing the transparency in that respect

    I do believe however that there's this aspect of the 6.0 system, the thrill of seeing athletes getting the perfect 6.0 mark that appealed to audiences and that we have no lost and let's be honest, the larger audience doesn't care about knowing the details and the reasoning behind the marks and won't check the protocols on the isuresults website. But as a long-time fan of the sport, and someone who cares about its development, I think overall the IJS is a positive development, although it certainly needs tweeking and some fundamental work with judges as they seem, at least at high-level competitions, to get their CH and IN mixed up with their SS and their TR, or having no clue what performance/ execution is really about
  30. os168

    os168 Active Member

    Joined:
    Mar 25, 2010
    Messages:
    868
    What ISU should really do is adapting the Hobbit Gollum/Avatar Na'vi film SFX plotting technology and keep the COP as it is.

    They should have a screen blinding the judges and the rink, and a projector playing infront of them with multiple camera views calibrated to bring the most accurate representation of what went on the ice to the skater's avatar.

    Each skater are plotted before hand with a random character so the judges don't know who's who. As the skater skated, their version of Avatar is being replayed to the screen infront of the live judging panel complete with mistake, falls, edge issues, body movement, musical interpretation and all.

    Audience will react to performances as they normally would. Judges wouldn't have the luxury to see if it is Patrick Chan skating or Dennis Ten, whether the skate is from Canada or Kazakhstan, reputation, history and all.

    Then they can judge with total unbiased view purely on the skating. The audience then judge the judges like watching a blind date program. Thumbs up or thumbs down complete with canned laughter, boos, jeers and cheers buttons, and a bucket over judges head with mysterious substance in the case of jeers. Oh yeah I'd pay 1 month salary to see that even if I have to travel to Latvia. I really think this will attract more viewing and grow the sport.
    Last edited: May 7, 2013
    alilou and (deleted member) like this.
  31. dorianhotel

    dorianhotel Member

    Joined:
    Jan 11, 2013
    Messages:
    239
    The judges would score many of Chan's performances up to 50 points lower if the same skate was done by someone they thought was a junior skater from Australia or Hungary. Even if they thought it was say Denis Ten it would be 30 points lower in many cases.
  32. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    I doubt that. If they can see the speed and edge depth and complexity of the steps, they'll reward those no matter who they think is skating.

    And if the technical panels can see which jumps were executed and rotated, which turns and steps and positions were achieved, then the technical content would get full credit, and that's accounts for about half the total scores.

    In reality, there is undoubtedly some reputation effect, but I can't imagine it would amount to more than 10 points even in the most extreme cases.

    Now, if you intend this avatar technology to hide the quality of the skating as well as the identities, and only give a vague representation of what the skater did, then there would be no possible way to judge accurately no matter who the judges thought they were watching. In which case there would be no point in holding competitions to begin with.
  33. os168

    os168 Active Member

    Joined:
    Mar 25, 2010
    Messages:
    868
    Not really. The technology must be used to accurately depict the exact quality of the skating AND hide the identity of the skater. e.g camera zoom in on the actual skating details similar to what tech panel sees. The advantage of getting everything digitalised is the movements are entirely trackable measurable, tracesable, and make useful data analysis for things like coverage distance, speed, height, power, degree of rotation and UR etc.

    The point is, the judges should have no preconceived notion of who the skater is and what they are capable of, but focus entirely on what they actually did on the day. The panel should still able to see the quality and details in which he/she have skated and make judgement purely on that. The technology need to be sophisticated enough so the emotional aspect of the performance must be able to be translated on to the Avatar while still hide the skater's ID. Such technology might not exactly exist yet, but it sounds way better than putting a paper bag over the skater's head and just cut out 2 holes. (that is perhaps for the B events :D)
  34. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    This would be interesting as a test. Digitize several performances by skaters at different general skill levels but with the same jump content, and see whether the scores for the higher level/more successful skaters are consistently higher without the judges having a clue who is who.

    Another way to test the theory would be to look at JGP results, especially the first-time competitors, because most of those skaters have no international reputation yet -- the only real confounding factor is federation affiliation and any lobbying that may go with it.
  35. os168

    os168 Active Member

    Joined:
    Mar 25, 2010
    Messages:
    868

    This would certainly makes an interesting academic study and probably will validate many of the problems of COP many have already identified. However, it does not resolve the matter at hand which is how to prevent biased judging happen even if judges themselves are probably conscious of it and try their bests not to.

    Bad decisions can easily happen due to things like peer pressure and preconceived knowledge. Little time allowed to think clearly, logically, carefully in detail with time needed to process and make good analysis. Lack of visual aid, data, facts does not help things. You add political, federation pressure, expectation, personal interests... suddenly there create all sort of variables on top of what the rule book allows. COP being a tick box system will create tick box results. A literal essay format like 6.0 that has the breadth of room for intellectual and emotional realization will created more well rounded views rather than merely recite facts, yes and no answers. How will the judging change if the score is only marked AFTER every skater has skated, then the result are finally announced at once. Or how about allow a secondary stage where the Judges can go back and correct the score if they feel like they made a mistake then? Any change in marking is noted, and openly declared. This may add a secondary round of thriller to the sport, and gave the judges the room to update their marks.

    The biggest negligence of the COP is that it treats the judges like some mathematical robots with no emotions or feelings. It assumes they just do as they are told and are 100% accurate all the time, despite this is not humanly possible. It failed to taking in account of the emotional, psychological aspect of human judging, or how social, political, cultural and environmental effects can all affect decision making during pressurized environment. Let alone things like artistry are really unquantifiable and that there should not be a ceiling on it. Or that some how all PCS categories should be awarded on the same scale out of 10 (are all triples the same?) A more evolved model of COP would take all these things into account and find ways to deal with inconsistency in marking between different competitions, between judges, between skaters, by using measurable, quantifiable hard data in support. Carolina is fast, but how fast is she really compare to Yuna Kim in the first half, compare to her past, how about acceleration, speed, height, power, coverage, distance traveled etc.

    I don't think this will ever be resolved unless ISU really want to tackle 'true' fairness. And actually I don't think ISU really want the sport to be truly fair without at least able to dictate its direction as it see fit, since its has its own economic interests to take care of despite the sport itself already seems rather broken economically these days. It sucks, but CPR is completely useless unless the patient him/herself want to live.
  36. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    My hypothesis, which we can't validate unless we actually perform the study and replicate it several times is that

    1) such a study would validate that there are some systematic effects of reputation, politics, and skate order, which means that anyone who supports the objectivity of the IJS needs to qualify such claims with appropriate caveats
    but also
    2) the study would also validate that the majority of variance in scores between different skaters executing the same content is in fact attributable to real, visible differences in the skating that trained observers can see irrespective of the skaters' identity. In other words, that

    If, say, the results showed that 90% of the scoring was attributable to factors inherent in the skating itself (signal) and 10% attributable to outside factors such as reputation and skate order (noise), would that satisfy critics who guess the noise percentage is higher because they haven't themselves trained to perceive as much of the signal as the officials?

    As to whether that makes IJS better or worse than holistic ordinal judging, for comparative purposes we'd have to conduct similar experiments using traditional 6.0 scoring or some similar system, with and without judges knowing the skaters' reputations.

    But just anecdotally, and logically, it seems to me that although code-of-points style scoring is somewhat subject to effects of reputation and skate order, ordinal judging is much more so.

    Ordinal judging is, however, easier for lay observers to identify with because it doesn't require detailed technical knowledge.

    So deciding between the two approaches comes down to a decision between fairness for the athletes and pleasure for the spectators (and the subset of competitors who consider the technique merely a means to artistic ends rather than an end in itself).

    As I say, I think this is even more true of 6.0 than IJS. So solving the problems in the existing system would do better to move forward to even more objective -- probably technology-dependent -- approaches.

    The appropriate technology might not exist at all yet, or not yet in affordable form. But it's fun to imagine what such a more evolved model might look like.

    How can the objective details of the performance best be quantified? And how could judges evaluate the various global, subjectively perceived aspects that make up most of the PCS criteria in ways that measure the various aspects appropriately to their importance (which may differ from one skater to the next)? How should those assessments be combined with the objective data to produce results? How can they best be communicated to the skaters and spectators?
  37. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    17,988
    Sorry but as a judge who uses the system I cannot even put into words how I feel about this comment (and it is not positive). A judging system should provide a set of tools that helps judges apply the rules as written and it should aim to provide an objective outcome. It should also aim to provide a consistent approach. 6.0 was meant to do the same but it just had no transparency. So it gave an impression that it was based on judges feelings and emotions. Is that such a bad thing that a system does take out emotion and tries to provide a more objective criteria? That is actually why the system works so much better than what came before.

    Try judging a field of 30 young skaters at a Preliminary level. These skaters are all about the same technical level, some will be worse and some better. You need something as you watch skater after skater that is going to help you through getting to get a result for that competition. I am not sure how judges managed to judge that many skaters under 6.0 because I became a judge as IJS was being introduced, but as we still use 6.0 for beginner level competitions, even trying to rank a field of 10 skaters can be a pain in the arse. Because as you go along you are not giving skaters the credit for what they do but rather pigeonholing them into a place.

    However you talk to any judge and they will talk about performances in terms of how it made them feel and what they liked about the performances pretty much as any skating fan.
    Last edited: May 8, 2013
  38. alilou

    alilou Crazy Stalker Lady

    Joined:
    Oct 28, 2005
    Messages:
    3,972
    This is just nonsense to me. I've been watching skating since the 70's. What I saw then is what I see now - a lot of fairly pedestrian programs, and a few outstanding ones. The judging system doesn't make much difference. I'm frankly puzzled that anyone thinks skating was so beautiful back in the 70's, 80's and 90's. Much of it was literally nothing more than crossovers and elements. And also some of it was extremely beautiful. Just as much of it now is fridge-break material (except I love to watch everyone because I'm thrilled when I see someone's improvements) and some of it is extremely beautiful (Takahashi's best, Abbott's best, heck even Chan at his best, Kozuka, Czinsy - exquisite skater, Ice dance I could create a list a mile long of beautiful programs, S&S's Pina, and the list goes on. To imply IJS has taken the beauty out of skating I find bewildering. What are you watching?
    I'm aware that the system has some flaws that could be addressed, but I much prefer it to 6.0
  39. Zemgirl

    Zemgirl Well-Known Member

    Joined:
    Jul 13, 2009
    Messages:
    4,833
    A few months ago I ran a few searches for skating articles on some academic databases and Google Scholar. As it turns out, Shirra Kenworthy (I imagine it's the same one who competed for Canada in the 1960s) wrote a dissertation (2009) on decision-making and cognitive processes of skating judges. I also found this 2004 article, which examined reputation bias in judging (under 6.0, though), and this 2010 study found what the authors termed a "difficulty bias" in gymnastics.

    I haven't read any of these in full - just saved them in my future reading file, so I can't comment about the rationale, methodology etc.

    Replication studies are very difficult to publish ;)
  40. vodkashot

    vodkashot New Member

    Joined:
    Aug 4, 2009
    Messages:
    197
    Well, that's one way to frame it. Looking at it another way, 6.0 was away more transparent and accountable than IJS. You knew exactly who was giving what mark. You may not know how judges came up with their marks, but it's not like we know why or how judges come up with their marks under IJS either. You can see where the marks are distributed, but that's about it. Who knows why one judge gives an element a 0 in GOE, but another gives the exact same element a +2? Ditto for wide point spreads in PCS as well. And in some ways IJS is even worse than 6.0 because of the anonymous judging, which really is the epitome of zero accountability and transparency