Opining on IJS scoring system: 5 calls to action by Jackie Wong - agree/disagree/your suggestions?

[Rock2]

Further to my last post, my frustration with this sport that lies above everything else is the routinely incorrect and inconsistent application of GoE and PCS marks. What's worse, the ISU seems to have little mechanism or interest in fostering more reliable scoring.

As a result, the panel has in my mind too much latitude to select winners without repercussion.

Inasmuch as we can argue IJS is killing the sport, the rampant subjectivity that continues to be unchallenged plays just as big of a role in the failure to attract new audiences, much of whom do not see skating as a credible sport.

The ISU is trying to straddle a fence between athletics and performance art and I don't see it as a long term solution.
Pick a side:
Become more like 'artistic' gymnastics where tech drives almost all of the scoring...or find a way to get more transparency and consistency in the subjective elements of the scoring
OR
go the other way and allow judges (and audiences) to vote with their hearts or personal agendas based on how they respond to what they see....but insodoing take it out of the Olympics.

As me gran'pappy used to say: you can't suck and blow at the same time. So, stop trying to

 

[gkelly]

The ISU is trying to straddle a fence between athletics and performance art and I don't see it as a long term solution.
Pick a side:
Become more like 'artistic' gymnastics where tech drives almost all of the scoring...or find a way to get more transparency and consistency in the subjective elements of the scoring
OR
go the other way and allow judges (and audiences) to vote with their hearts or personal agendas based on how they respond to what they see....but insodoing take it out of the Olympics.

Or have technical experts award medals based on athletic and technical concerns, with "artistry" devalued or removed from the official scoring.

And also have a means for audiences and/or artistic experts to score the same performances and award separate medals/trophies to the skaters that win the hearts of the most fans or the approval of arts experts apart from the technical decisions.

 

[Rock2]

Or have technical experts award medals based on athletic and technical concerns, with "artistry" devalued or removed from the official scoring.

And also have a means for audiences and/or artistic experts to score the same performances and award separate medals/trophies to the skaters that win the hearts of the most fans or the approval of arts experts apart from the technical decisions.

Fine. But where I get hung up is this is an OLYMPIC sport. The second part of your recommendation has no place there IMHO.

I'm not an expert in gymnastics but look at ladies floor routine, set to music. Using IJS as a comparison the ENTIRE mark is built off of levels and GOE. What PCS there is, is built into the GOE as far as musical interpretation and other aesthetic elements goes.

I'm not saying skating should go to that level, but having a subjectively-marked PCS score driving 50% of the final grade is fine from an entertainment standpoint but not from a sporting one.

 

[Tahuu]

Maybe going back to pre-2008 and having 12 judges, have 6 judges score GOE and 6 judges score PCS. It has two advantages. First, it would be easier for the judges to focus on their respective tasks and arrive at more accurate assessment and scores. Secondly, it will provide fairer judging and scoring to the skaters as it'll be harder for the judges to manipulate the results. Essentially the tech panel determines the BV, the GOE panel decides the GOE and the PCS panel decides PCS. This three three panel separation would further reduce manipulations.

 

[MsZem]

Note, this is about OLYMPIC sports. And the Olympic slogan is 'faster, higher, stronger', if memory serves.

Figure skating predates the modern Olympic movement, and its inclusion in the games predates the adoption of "faster, higher, stronger" as an official motto. The sport's approach to combining athleticism and artistry was well-established many years ago. It doesn't diminish the idea of "faster, higher, stronger"; it means that those thing are not sufficient to be a successful figure skater. Just as they are not sufficient in teams sports, or in sports requiring strategy or precision, or in judged sports that have other criteria.

 

[vesperholly]

If all judges start with the same idea that “This is what a 7, 8, and 9 in Composition looks like” and “This is what a 7, 8, and 9 in Interpretation looks like,” they might be better positioned to say “That was a 9 in Interpretation but somewhere between 7 and 8 in Composition.”

I think this is the whole problem. IJS is trying to have skaters compared to "the book", and judges need to rank skaters. It's an inherent dichotomy.

 

[MsZem]

I think this is the whole problem. IJS is trying to have skaters compared to "the book", and judges need to rank skaters. It's an inherent dichotomy.

Not necessarily. Ideally what you want is for judges to have a standard that they can refer to when evaluating performances and tells them "this is what an 8 means". Judges need to evaluate specific performances, so a concrete example, rather than abstract definitions, would be very helpful. And to keep things fair, use performances by retired skaters to illustrate what constitutes decent vs. good vs. great - e.g. Kozuka for skating skills, Lambiel for IN, V/T for P&E and so forth.

 

[BittyBug]

....Kozuka for skating skills....

There's an example of the judges no marking the way the audience would have expected. Kozuka rarely got the cred he deserved for SS.

 

[Vagabond]

Not necessarily. Ideally what you want is for judges to have a standard that they can refer to when evaluating performances and tells them "this is what an 8 means". Judges need to evaluate specific performances, so a concrete example, rather than abstract definitions, would be very helpful. And to keep things fair, use performances by retired skaters to illustrate what constitutes decent vs. good vs. great - e.g. Kozuka for skating skills, Lambiel for IN, V/T for P&E and so forth.

The ISU does have videos with concrete examples of each aspect of each program component, though the videos don't refer to the actual scoring.

Here, for example, is a video using Sarah Meier and Tomáš Verner's Short Programs from 2008 Worlds as positive examples "effortless movement," which is part of the Interpretation component.

Of course, effortless movement isn't all there is to Interpretation, but it's worth noting that Sarah Meier's IN score for that program was 6.79 and that Verner's was 7.79.

 

[MsZem]

There's an example of the judges no marking the way the audience would have expected. Kozuka rarely got the cred he deserved for SS.

All the more reason to finally give him the recognition he deserves!

The ISU does have videos with concrete examples of each aspect of each program component, though the videos don't refer to the actual scoring.

I know. They could just take the extra step of using certain performances as standards. But of course, the ISU has to complicate everything...

Verner at 2008 Worlds

 

[Vagabond]

Verner at 2008 Worlds

 

[MsZem]

No, it was just sad. No camp factor.

This was true greatness, though.

 

[MAXSwagg]

All the more reason to finally give him the recognition he deserves!

The ISU won’t do that because they would have to admit that they screwed him and many skaters over for reasons that have no base other than games.

Sorry. For me, figure skating is inherently comparative and should be judged as such within the scope of a competition. Was this skater’s 3A better than, worse than, or similar to the skater I just saw/have the highest mark to? I also don’t buy the “But they see 30+ skaters over the course of a short program!!” excuse as to why that’s not possible. As a classically trained clarinetist, I’ve judged performances by high school students at the state level where you could see almost a hundred over the entire day and had no issue doing this, and still would have no issue doing it if they all played Mozart his Clarinet Concerto. Of course, for me it’s a hybrid. How well did they do in the scope of their own merits plus how did their performance compare to other students and my ratings for them.

 

[Jot the Dot Dot]

Anyone for Rock - Scissors - Paper ?

 

[DreamSkates]

No, they are not. They are written in very accessible and knowledgeable language.

All of them are excellent ideas. I would add #6 - if a judge or official is found to have cheated, minimum of one year suspension. If a judge or official cheats a second time, they're banned for life. As I've said before, even the best judging system won't be effective or fair if the system's users are corrupt.

Cheating at all - banned for life. A judge must be held to the highest standard.

 

[DreamSkates]

The ISU won’t do that because they would have to admit that they screwed him and many skaters over for reasons that have no base other than games.

Sorry. For me, figure skating is inherently comparative and should be judged as such within the scope of a competition. Was this skater’s 3A better than, worse than, or similar to the skater I just saw/have the highest mark to? I also don’t buy the “But they see 30+ skaters over the course of a short program!!” excuse as to why that’s not possible. As a classically trained clarinetist, I’ve judged performances by high school students at the state level where you could see almost a hundred over the entire day and had no issue doing this, and still would have no issue doing it if they all played Mozart his Clarinet Concerto. Of course, for me it’s a hybrid. How well did they do in the scope of their own merits plus how did their performance compare to other students and my ratings for them.

How would a competition be different if you were judging that many professionals? That is what I don't see as a comparison to figure skating at the senior-international level.

 

[Rock2]

Here's an idea I've been sitting on for a bit:

Give the judges rubrics to help differentiate the full range of component scores.

Could be helpful, but you're assuming the problem with PCS judging is a lack of understanding of the components and what a 5, 6, or 9 means.
I would argue that is a bit of a problem with judges from smaller countries who get less training.

IMHO the core issue is that panels are using PCS and GOE to slot skaters which was happening in 6.0. The increasingly-narrower band of PCS scoring from each judge adds fuel to my argument and reflects a clearer intent to slot a skater.

There is no mechanism within ISU to correct this. There is no protest feature and no centralized officiating admin in the ISU to correct panels on their mis-application of the scoring system. All the bad judging goes unchecked, effectively. Even those caught cheating get a little slap on the wrist.

I like the components, but they either need to be weighted less or scored properly. I'm good with either, but it needs to be one.

 

[overedge]

Cheating at all - banned for life. A judge must be held to the highest standard.

A cheating judge is an issue of discipline, and a suspension would be appropriate as a warning that if it happens again, they're gone.

 

[DreamSkates]

A cheating judge is an issue of discipline, and a suspension would be appropriate as a warning that if it happens again, they're gone.

In principle, at least in the rest of the world, cheating is an issue of honesty.

 

[Rock2]

Figure skating predates the modern Olympic movement, and its inclusion in the games predates the adoption of "faster, higher, stronger" as an official motto. The sport's approach to combining athleticism and artistry was well-established many years ago. It doesn't diminish the idea of "faster, higher, stronger"; it means that those thing are not sufficient to be a successful figure skater. Just as they are not sufficient in teams sports, or in sports requiring strategy or precision, or in judged sports that have other criteria.

That argument would fly until, oh about 2002 and the infamous pairs competition at SLC.
From that moment on skating's heritage became irrelevant because the sport's credibility as being olympic-worthy was in serious danger.

IJS was the direct result of the IOC telling the ISU to turn skating into a more credible olympic sport or you're gone. The specific directives I'm not clear on but a big part of it was to have a scoring system that was much more transparent and objective that could overcome any federation attempts to negotiate with each other for medals.

The more influence GoE and PCS have on the scoring...and the less the ISU does about it, the more we're back to 6.0 effectively...and the greater the chance of another SLC scandal. If that happens again, I will truly fear for the sport.

 

[MsZem]

That argument would fly until, oh about 2002 and the infamous pairs competition at SLC.
From that moment on skating's heritage became irrelevant because the sport's credibility as being olympic-worthy was in serious danger.
...
The more influence GoE and PCS have on the scoring...and the less the ISU does about it, the more we're back to 6.0 effectively...and the greater the chance of another SLC scandal. If that happens again, I will truly fear for the sport.

Skating's status was not in danger because it had an artistic component; it was in danger because of dodgy judging. This may shock you, but other sports - including Olympic ones - have also dealt with judging issues. Technical panels have as much effect on the scoring as judges GoE's and PCS, if not more.

Also, your original point was that skating must adhere to "faster, higher, stronger". It does; it's just not all that matters. If you don't like it, watch the skiing instead.

 

[Vagabond]

The more influence GoE and PCS have on the scoring...and the less the ISU does about it, the more we're back to 6.0 effectively...and the greater the chance of another SLC scandal. If that happens again, I will truly fear for the sport.

What, exactly, do you mean? Almost the only things the judges ever do is mark for GOE and PCS. They aren't the ones identifying the elements executed or the levels attained.

As it is, the relative weight of the second mark has diminished significantly under COP. This is particularly true in the Men's discipline, where some skaters are now receiving TES in excess of 50 in the SP and 100 in the FS, which is the most they could possibly receive in PCS. Also, from what I can tell, without doing a detailed analysis, the overall ratio of skaters receiving higher TES than PCS in top-level international competition is higher than ever now.

 

[overedge]

In principle, at least in the rest of the world, cheating is an issue of honesty.

As I pointed out in my previous response, when there are multiple and conflicting stories as to what happened - which is often the case on the rare occasions that the ISU does anything about cheating - it's not always clear who was dishonest or why.

It would be lovely if, say, there was video and audio of a judge accepting and saying, "Thanks, Suzie Snowflake is going to get a 10 on skating skills from me". That would be a reason for a ban for life right there. But it's not usually that clearcut.

In the rest of the world, progressive discipline suggests that someone who is dishonest needs to be clearly informed that dishonesty is unacceptable, why it's unacceptable, and what the consequences are if it continues. A suspension is a punishment - it's not saying, "you just keep being you". A suspension also gives the dishonest person a chance to become a honest person, and the opportunity to correct one's behaviour is also a basic principle of discipline and justice.

 

[Vagabond]

hey could just take the extra step of using certain performances as standards.

So, which one of these performance should set the standard for PCS of approximately 26.25?

Sarah Meier, 2008 Europeans SP, 26.28 pts.
Alaine Chartrand, 2017 Skate Canada, 26.17 pts.

Or could these be used to explain what justifies a difference of 0.11 pts.?

 

[Rock2]

What, exactly, do you mean? Almost the only things the judges ever do is mark for GOE and PCS. They aren't the ones identifying the elements executed or the levels attained.

This is the best thing about IJS. It took a scoring system that was 100% subjective and made it 50-60% subjective. Major step in the right direction.

As the years go by we're seeing more pronounced efforts to use GoE and PCS as a more powerful tool to drive outcome. Top marks used to be hard to come by 10 years ago but now we're seeing +3s, 9s and 10s for close to the same level of skating. That's a warning flag.

Going back to the original point of the post there is a LOT of discussion about scrolling back TES values. As I have said already I'm not in favor. Gives the overall scoring an increasing bias toward subjectivity (again) and more greatly empowers panels to effectively 'pick' a winner. Slippery slope, so I'm not in favor of this recommendation until we get more reliable and accurate GoE and PCS scoring.

 

[Rock2]

Skating's status was not in danger because it had an artistic component; it was in danger because of dodgy judging. This may shock you, but other sports - including Olympic ones - have also dealt with judging issues. Technical panels have as much effect on the scoring as judges GoE's and PCS, if not more.

Also, your original point was that skating must adhere to "faster, higher, stronger". It does; it's just not all that matters. If you don't like it, watch the skiing instead.

Artistry was/is irrelevant for this discussion. Dodgy judging was the symptom of a bigger problem -- a subjective scoring system with no transparency or accountability. Conditions ripe for back door deals and picking the winners you wanted.

See my prior post just a min ago. TES does limit the influence panels have vs 6.0 but I'm not in favor of swinging more bias toward PCS relative to TES. Creating these same conditions all over again.

The skiing comment was unnecessary. I'm sorry you feel you had to type that. I love skating and want to see it do well. I personally love the artistic side but I'm putting that aside to force myself to be objective, thinking about what does skating need to be as an Olympic sport, which is important context for this whole discussion going back to the subject line. With that in mind, my belief is that 'faster, higher, stronger' should dominate the result...or skating will evolve in a way that will challenge its Olympic standing. Luckily that won't happen since it's such a ratings driver so we should be fairly safe in the short to medium term.

Other sports have judging but the 'opinion' comes on to what extent the athlete met a black and white technical requirement. Those debates do make the news on occasion. Sadly, the debates in skating are never about this...or rarely. The general public has to constantly hear about debates over subjective/artistic criteria such as who had the performance of the night. This mystifies general audiences and completely turns off the bulk of sporting enthusiasts. This erosion in sporting credibility is hurting the sport more than IJS ever will. Outside of the stars that transcend the sport to draw audiences in waves that come and go, skating's appeal will indefinitely be limited to hard core fanatics and the passive viewers who see skating as light entertainment worthy of a look at the Olympics or the occasional world champs.

 

[starrynight]

I'm not saying skating should go to that level, but having a subjectively-marked PCS score driving 50% of the final grade is fine from an entertainment standpoint but not from a sporting one.

Or maybe in the case of ice dance where really the TES mark is more or less just a disguised version of PCS....

 

[Rock2]

Or maybe in the case of ice dance where really the TES mark is more or less just a disguised version of PCS....

Ya sort of.

Dance is a tough one, now. TES is all bunched up which means callers can bump someone off the podium with a level change. The panel has just as much power now to slot the skaters through PCS with all the narrow TES margins.
I wouldn't be surprised if level 5 and 6 elements were introduced to the rulebook soon enough....

 

[skatingguy]

Sarah Meier, 2008 Europeans SP, 26.28 pts.

I don't think this links to what you intended - though maybe I've missed something and the joke went over my head.
Sybil Personalities
https://www.youtube.com/watch?v=mYMTyN_BZ8Y

 

[starrynight]

Ya sort of.

Dance is a tough one, now. TES is all bunched up which means callers can bump someone off the podium with a level change. The panel has just as much power now to slot the skaters through PCS with all the narrow TES margins.
I wouldn't be surprised if level 5 and 6 elements were introduced to the rulebook soon enough....

I think that's a problem that a lot of people have with ice dance. Ever since Montreal figured out how to *hack* the levels on a number of elements (or at least everyone realised Marina Zueva just choreographed show-off elements that weren't actually needed for TES) it has been demonstrated how relatively straight forward it is to obtain a level 4 with reasonably basic elements. Then it all comes down to GOE which is/can be adjusted according to reputation and isn't linked to difficulty, just execution.

The result is that there is no way for a team to escalate difficulty to overtake if they cannot obtain the reputation based GOE or PCS. There's nowhere left to go with TES. Plenty of teams do lifts which should probably get a level 5 or 6 but it doesn't exist and GOE is not there to assess difficulty. It certainly is frustrating to watch teams do spectacularly difficult lifts but know that there is no points benefit to it, as they can't score higher than a level 4, will probably take a GOE hit due to the difficulty and may as well have done something easier.

The result is that a fraction of a mark on a level 3 step sequence is what will probably decide the Olympic podium.

Ice dance isn't like pairs where for example, a team could increase their scores by adding a big 'ta-dahh' element such as a quad twist or a quad throw. In singles and pairs there's always a *next step* - add a triple axel, a quadruple axel! - but no such thing exists in ice dance.

They are going to change things to -5 to +5 GOE next season. I wonder if in ice dance that might have the effect of discouraging difficult elements? Would it be more beneficial to get +4 GOE on a level 3 lift than a +2 on a level 4 lift? The GOE changes I think are designed to try to slow the quad rampage in mens but I wonder what the impact on dance will be.