New way to score PCS....?

[fsfann]

For the past number of years it has become evident that PCS scoring has veered away from its original intent and now seems to be more a way to slot skaters in a specific position. It is quite rare to see a variance in the PCS marks for each skater, when in reality, there "should" be a good variance for many skaters depending on their strengths and weaknesses... It seems now that a "5.8" is now how many judges are scoring skaters with one mark for all of the PCS.....

I'm wondering... Do you think we might see an improvement if each component was judged separately by one or two judges? (So instead of having all judges judge all components, we would switch to a system where two judges only look at skating skills... another two would just mark transitions and so on... ?)

I'm thinking that it might actually see more accurate marking with this method??

 

[gkelly]

That's an interesting proposal.

Effects of national bias would probably be no worse than is currently the case, since judges from different countries could cancel out the effects of each others' scores on the different components.

However, if only one or two judges are judging each component, then the scores for that component would end up being heavily dependent on those individual judges' personal preferences, pet peeves, and weighting of the different criteria for that component. And also on whether each judge tends to score high or low in general, or to use a particularly wide or narrow range between skaters of similar but not identical ability on those criteria.

There would be no dropping of outlier scores -- or even averaging if only one judge is assigned -- to mitigate any of the "noise" introduced by how any individual judge interprets particular criteria or tends to use numbers.

So I don't think we could say that the scores would be "more accurate" as a result.

What they would be, most likely, is less similar to the scores for the other components. But how much of that dissimilarity is due to separating the evaluation of the different components from each other, and how much to the fact that different human beings are doing the scoring for each, would be difficult to identify.

What might be an interesting experiment:

Get a large group of expert judges together and divide them into three or more groups of at least 9 judges each.

Assemble a group of diverse performances (preferably performed live by skaters of different skill levels and emphases).

Assign one group to judge a these performances normally, GOEs (with input from a technical panel) and all components.

Have the remaining groups each score a different component and only that one component. If there are only two groups, I'd suggest assigning them to Skating Skills and Interpretation, since these have the least to do with each other.

Then look at the results.

If the averaged component scores of the normally judging panel are all close to each other, as is usually the case, and if the averaged scores of the single-component panels are significantly more different from the other components, but fairly similar between judges on the same component, that would argue that separating the scoring of each components produces more accurate component scores. I.e., different experts will approximate the same score for each component if that component is all they need to focus on.

However, if there is still a wide range between the top and bottom scores for each component for each skater (the judges for that component disagreed with each other about what number to assign, for any of a variety of reasons), and a narrower range between the average scores for one component and a completely different component scored by a different panel, that would suggest that differences between individual judges' thought processes would have more effect allowing them to avoid thinking about GOEs and other components.

In the latter case, assigning different judges to different components would lead to less rather than more accuracy.

I'd be curious what the results would be.

 

[AxelAnnie]

Y'all didn't think the new system would be fair?

 

[Amantide]

They already experimented with separate judges for PCS & Tech. The result was a big MEH...

Seriously though, I don't know who was that delusional to think that the new system would change that? Let's be real, it's not like the old system didn't have them. The problem wasn't with the system, the problem was the people not using it properly. I've rarely seen a judge use them independently, as they should, except perhaps Vanessa Reilly? Ghad, I miss that woman.

On the other hand B&S would have won handily, in spite of the mistakes in the LP, with the new system, which I find it ironic. So there is that.

 

[VGThuy]

On the other hand B&S would have won handily, in spite of the mistakes in the LP, with the new system, which I find it ironic. So there is that.

You’re assuming they would have applied the new system “correctly” or whatever you think is correctly in that case. You never know.

 

[dinakt]

They already experimented with separate judges for PCS & Tech. The result was a big MEH...

You think the result was a bit Meh? From what I remember, it significantly improved the separation between PCS and TES and judged the skaters much more fairly. And that is without any special "only PCS" training for judges. I was quite bereft that it did not get traction as I saw a marked improvement...

 

[KaoriFan]

Yes I agree with your post. PCS is just ordinal judging with fudgy numbers to confuse people who don't understand.

I don't know the solution, but there probably isn't one in IJS that won't be manipulated in same way again.

 

[MacMadame]

They already experimented with separate judges for PCS & Tech. The result was a big MEH...

That isn't the OP's proposal though. The proposal is that each judge who is judging tech, be assigned just 1 or 2 PCS to also judge instead of judging all 5.

I am not sure what that would accomplish because the reason PCS are scored the way they are isn't because the judges aren't paying attention and judging less PCS would solve that.

 

[AxelAnnie]

Instead of all the judges and their finding a way to finegale the results they want to say nothing of bribery and secret deals I see a perfectly marvelous alternate. FSUers should be in charge of all judging!

 

[Amantide]

You think the result was a bit Meh? From what I remember, it significantly improved the separation between PCS and TES and judged the skaters much more fairly. And that is without any special "only PCS" training for judges. I was quite bereft that it did not get traction as I saw a marked improvement...

No, I didn't thought it did much good tbh. But if I remember correctly, one of the reasons it didn't get much attraction was also for practical reasons? Meaning the number of judges and budget etc. I'm not sure because it was a long time ago but I seem to remember the Italian commentators talking about this.

You’re assuming they would have applied the new system “correctly” or whatever you think is correctly in that case. You never know.

Right. Which brings us back to point 1, people. And you make a good point with "whatever you think is correctly" because the same goes for judges. You can certainly improve things but in the end you cannot take away the subjectivity factor from this sport. There are elements in this sport that very much depends on individual taste and/or culture also.

 

[zigzig]

The PCS aren't the only problem. The TES are crazily manipulated with inconsistent, unfair, biased and selective use of q's, <, <<, e, ! and even questionable calling of footwork levels and sometimes spin levels.

I really don't know what the solution there would be but it drives me insane how the objective component of scoring can sometimes be as subjective as the presentation scoring based on how the panel wants the results to come out.

 

[screech]

There's unfair use of GOE as well. There are skaters who have some not so great jumps that are given +2 because of who they are. Meanwhile another skater can do a beautiful jump that theoretically ticks all the boxes but gets maybe +1, because they aren't one of the big names.

I really hope there's a complete overhaul of the scoring system (though I doubt it). The theoretical idea behind the system is decent, but it's how it's being used that's complete bull poop. If skating wants to be taken seriously after all the controversy at these Games, this next quad should be used to seriously fix things.

 

[marbri]

I haven't read every post here so if it's already been mentioned apologies in advance.
Meagan Duhamel had an instagram live with an ISU judge in the past week. In the course of the conversation this judge mentioned that there had been discussion about dropping PCS down to three categories. In was said when they were discussing and explaining PCS categories, specifically PE and IN and the overlap in between them. IIRC it was a serious consideration that kind of got out on the shelf during the pandemic. So that might be something to look out for in the future.

 

[AxelAnnie]

There's unfair use of GOE as well. There are skaters who have some not so great jumps that are given +2 because of who they are. Meanwhile another skater can do a beautiful jump that theoretically ticks all the boxes but gets maybe +1, because they aren't one of the big names.

I really hope there's a complete overhaul of the scoring system (though I doubt it). The theoretical idea behind the system is decent, but it's how it's being used that's complete bull poop. If skating wants to be taken seriously after all the controversy at these Games, this next quad should be used to seriously fix things.

What an optimist you are! The last time they overhauled the scoring system it took a matter of a few months before judges figured out how to manipulate it.

I say it again: They should have FSU do the judging.

 

[Amantide]

I haven't read every post here so if it's already been mentioned apologies in advance.
Meagan Duhamel had an instagram live with an ISU judge in the past week. In the course of the conversation this judge mentioned that there had been discussion about dropping PCS down to three categories. In was said when they were discussing and explaining PCS categories, specifically PE and IN and the overlap in between them. IIRC it was a serious consideration that kind of got out on the shelf during the pandemic. So that might be something to look out for in the future.

Not that it's going to change much but I actually think PE and IN should be one. I also think Choreo&Composition can be in one category. As Is aid, it's not going to revolutionize anything but the simple you make things (when/if you can) the better. Also, I don't understand the q thing they added. I mean, yes I understand what it means but it just makes things unnecessary complicated for all the people involved.

Truth be told, I would go back to 6.0 myself but I'm alone in that. lol

 

[Allskate]

If skating wants to be taken seriously after all the controversy at these Games, this next quad should be used to seriously fix things.

Unfortunately, I think the controversy focused on the doping and the Russians, not the scoring. Looking at the PCS from the women's competition just has me shaking my head. IMO, there is not just a PCS bump from landing quads and triple axels but also from merely attempting them.

The PCS problem seems to have become worse for the men. Especially with the broader range of GOE, the ceiling for TES is higher than the ceiling for PCS. And there is a much bigger range in TES. When you look at the actual scores, even putting aside the impact of jumps on the PCS score, the TES score has a much bigger impact on the score than the PCS score does.

Tara Lipinski repeatedly has complained that a quad is not allowed in the women's short program. But, instead of allowing greater difficulty, maybe it should be the opposite. At one point, skaters had to try to master the fundamentals and do figures in competition. Perhaps limiting the amount of jump difficulty that can be put into a program - or at least the short program - could emphasize quality. But, I suspect that the scores still would be distorted based on who is landing the most difficult jumps. I don't think this is just national bias or cultural bias.

 

[VALuvsMKwan]

Not that it's going to change much but I actually think PE and IN should be one. I also think Choreo&Composition can be in one category. As Is aid, it's not going to revolutionize anything but the simple you make things (when/if you can) the better. Also, I don't understand the q thing they added. I mean, yes I understand what it means but it just makes things unnecessary complicated for all the people involved.

Truth be told, I would go back to 6.0 myself but I'm alone in that. lol

No you aren't.

 

[gotoschool]

I almost think the doping was a diversionary tactic to allow the blatant manipulation in scoring to receive a free pass. My first point about tech is the extreme degree of pre-rotations that many skaters performing the biggest jumps do, which is completely ignored though this is essential in establishing the actual airborne rotation and many land hunched over to conceal the hooked landings which is again selectively ignored for favorites while others with more erect posture and a weaker fed connection are nailed incessantly for tech calls with deflated GOE to rub salt in the wound. The same thing is done when favored skaters have their incorrect and suspect edges ignored by turning through them to get the torque they need.

Furthermore while the doping is certainly an important health and fairness concern, so is the fact that skaters are forced to contort their backs and bodies into these abnormal shapes that stretch the joints and ligaments to the breaking point not to mention how unsightly it is compared to a more natural takeoff and landing. This increases the risk of injury astronomically and most likely is one of the reasons women's careers have been curtailed.

The PCS is often not only inaccurate but a contradiction and obviously only used as a placeholder for favorites in the old 6.0 style. I would argue that COP is worse than 6.0 because there are more nooks and crannies to conceal bonuses and deductions which add up to a big jackpot for the selected or an empty cupboard for the unchosen. To have any real meaning there must be a way for those with truly great skating skills, rink coverage, posture, intricacy and crispness in edge work and great and engaging artistic expression including reaction from the audience to receive far higher scores than the ones receiving simply big tech scores, especially since PCS is now merely tied to the biggest tech (which is often manipulated) and powerful fed connection.

I am for dividing skating up into at least two different leagues or federations kind of like the NFC and AFC in football likely based on geographical location with separate champions and systems that can possibly come together for an overall winner at the end of the season. The championship can be decided by judges on a 50 / 50 basis with each system represented equally perhaps one program under one system and another under another system. In any case, currently I basically am more interested in only watching the skaters I wish to see and don't tune in for any alleged "competitive" aspect since the scoring is so blatantly biased and so often gives no value to the elements I wish to see in a performance.

In tech, the thing that aggravates me the most is the near complete exclusion of anything but where the blade appears or is judged to be on landing or take off (for edge jumps) in deciding its value. This excludes most all the elements that make watching a jump worthwhile to my mind such as distance, beautiful symmetrical rotation, height, distance, erect landing position, flow in and out, intriguing and well executed transitions with musicality, no extra stroking of the free blade to aid in jumping in the air, and most importantly no teflon tech immunity giving favored position under the system.