PCS used correctly

Discussion in 'The Trash Can' started by gkelly, May 11, 2013.

What would it mean to score PCS correctly?

Poll closed Jun 10, 2013.
  1. Some judges do it right, too many do it wrong

    38 vote(s)
    43.2%
  2. No one official does it right, but it can be done.

    9 vote(s)
    10.2%
  3. The rules need to be written better.

    23 vote(s)
    26.1%
  4. Can't be done right, so don't do it at all.

    2 vote(s)
    2.3%
  5. Can't be done right, so just use one number.

    3 vote(s)
    3.4%
  6. Right vs. wrong is meaningless -- but there is better vs. worse

    13 vote(s)
    14.8%
  1. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    So often we see posts stating the program component scores are not being used correctly. What does this mean? What would correct scoring look like?

    *There is a correct way to use these scores under the current rules. Some judges do it right (examples, please?), but too much of the time too many judges do not use this correct approach.

    *There is a correct way to use these scores under the current rules, but I have never seen any official judge do it right. Let me show you how it should be done.

    *The problem is more with the way the rules are written, and the judges are just doing what the rules tell them to do. We need better rules. (Example, please?)

    *Everything that's scored under the program components is subjective anyway, so there can't be any "right" way to score them. Skating should just make all these things extraneous to the results and score objectively on technical merit only.

    *Everything that's scored under the program components is subjective anyway, so why not just wrap it all up in one mark like the Presentation score under 6.0 and stop pretending that it can really be quantified?

    *Everything that's scored under the program components is at least somewhat subjective, so it doesn't make sense to talk about correct vs. incorrect scoring of these qualities. At best, it only makes sense to talk about better vs. worse. Here's an example of better.


    Looking forward to discussion and examples...
     
    LynnW and (deleted member) like this.
  2. Maofan7

    Maofan7 Member

    6,365
    3,471
    113
    This sums it up nicely
     
  3. munow

    munow New Member

    99
    3
    0
    Wrapping it up and basically resurrecting "artistic mark" does not make sense in the new judging system. Program components integrate two very distinct categories, namely technical qualities aka SS/TR and "artistic (or presentation)" qualities aka CH/PE/IN.
     
  4. Maofan7

    Maofan7 Member

    6,365
    3,471
    113
    The answer is to scrap COP/IJS altogether and revert back to the 6.O system.
     
  5. munow

    munow New Member

    99
    3
    0
    No thank you.
     
    TheIronLady and (deleted member) like this.
  6. sk8ingcoach

    sk8ingcoach Active Member

    460
    36
    28
    I agree that i think judges have 2 component marks instead of 5. They combine SS/TR and they combine PE/CH/IN together. I think the rules need to be written clearer so that judges can understand the differences in the 5 components.
     
    TheIronLady and (deleted member) like this.
  7. olympic

    olympic Well-Known Member

    7,133
    613
    113
    Aren't judges forced to keep their marks for PE, SS, CH, IN, TR with little variation? If so, I think it would be simple to keep the current breakdown, but allow judges to fluctuate as wildly as they see fit between PCS marks.

    Some skaters perform robotically but do great elements and skate cleanly. In this case the PE, SS and maybe TR (if there all there) could be around 7-8, but the CH and IN marks could be dropped to like 4-5. I also think telegraphing for jumps and spins should obviously deflate the TR mark independently of everything else. On the flip side, skaters have messy landings but all the in-betweens are there, the audience can tell the skater understands the music, so the flip would be true - low PE and SS scores, high CH and IN scores. I think there are 'fuzzy areas', like when a skater has great natural edging and speed along with good posture, yet is messy on the elements at a particular competition (Kostner on a bad day). What do you do with the SS mark, split the difference and give them a 5-6? Being able to land jumps cleanly to me would be part of superior SS (along with PE)
     
  8. query5

    query5 New Member

    672
    64
    0
    My take is this. They have had over 10 years to do it correctly. They havent done it.
    So why believe they would now and more important, why would they want to.
    They dont own up to past mistakes under figures, 6.0 and now ijs.
    They would have to admit they , was, are wrong.
    They cant do it.
     
  9. shady82

    shady82 Active Member

    473
    31
    28
    PCS isn't done correctly at all. They are generally a good reflection, but the five categories are meaningless because judges give similar scores across the board anyway. Skaters who would get high PCS also got high presentation scores under 6.0, or would have. It seems very similar.

    A glaring example of PCS done wrong is the 2006 U.S. Nationals Ladies SP. For 2-5, the PCS ranking should have been Liang, Hughes, Czisny, then Meissner based on what is accurate. However, the PCS scores were given solely on reputation - it was Czisny, Meissner, Hughes, then Liang. For those of you who watched all these skates, does it make any sense?
     
  10. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    How does one define "what is accurate"? I.e., what are the correct criteria?
     
  11. Jun Y

    Jun Y Well-Known Member

    1,141
    291
    83
    1. The way program components are described now is not well written. Too vague and basically impossible to quantify. The way they are written practically invites large interpersonal variations due to individual perception and training in music/dance/art/whatever and cultural differences. I think we all get the vague ideas they want to convey but they are not explicit and clear enough to be consistently applied and to allow for accountability (ie, when a judge makes a mistake in the TR or IN score he cannot be caught and can explain away his decision). If it doesn't allow for accountability or measurability, I think it is close to useless.

    2. Meanwhile, assuming that the program components CAN be judged consistently and "accurately" if only they are better written (which is an unproven assumption), it remains unknown whether these standards are the right way to go to measure good skating and good performance. As you said in one of the items, can these subjective issues --- skating skills, expression of music, etc. --- be quantified, remains a philosophical black hole. Is Patrick Chan's skating skills 2 points better than Takahashi, interpretation of music 2 points better than Denis Ten? Really?

    3. Do we all agree on what makes for good skating and good performance? I am not a coach or an athlete or a judge or a tech specialist. I am a pure fan/outsider. But I have heard enough from insiders to be convinced that they do not agree on these fundamental questions. I am not saying this is necessarily a bad thing. It means that this is a philosophically diverse sport and perhaps fundamentally not the most suitable for competition. I'm fine with that.

    And then there is the issue of judge training and quality, but I have no idea how international judges are held accountable for their performances and/or corrected for mistakes.

    This is not to say that I prefer to revert back to the 6.0 system, which does not resolve any of the above problems. If I remember correctly, the only accountability then was to reprimand the judge whose scores are significantly different from other judges. The same system of appealing to the lowest common denominator (lowest as in the judging community, of course, which is certainly higher than casual fans). There was even less accountability for judges' quality and less clarity on what is good skating, which could swing from one year to another or from one quadrennial to another. The ONLY quality in the 6.0 system is the ordinal structure, which admits openly that we don't know whether Chan is 1 point or 2 points better than Ten. We just know Chan's better in SP and Ten's better in FS. OK sometimes the distance between two skaters can indeed vary but is not reflected in placements, but I consider it a lesser evil than pretending to know the exact and objective difference between two skaters in, say, music interpretation.
     
    Last edited: May 12, 2013
  12. briancoogaert

    briancoogaert Well-Known Member

    12,544
    1,330
    113
    I think judges understand the difference between 5 components.
    From what I've heard from judges at a lower level, they have no choice, as long as the first mark is given, the others have to be at the same level. That's the problem.
     
  13. olympic

    olympic Well-Known Member

    7,133
    613
    113
    And thus, the weird PCS marks we see. See my post above. I think judges should be given the freedom to fluctuate as wildly as they see fit (provided they have a good reason).
     
  14. VarBar

    VarBar Well-Known Member

    1,253
    211
    63
    This shouldn't be a problem for the judges because I assume just about any reason would be a good reason in a subjective sport like figure skating when you don't have to follow any preset rules and stay within a certain corridor. Yeah, it would be interesting to see what PC marks the judges would come up with.
     
  15. Aussie Willy

    Aussie Willy Well-Known Member

    18,137
    2,165
    113
    Hmmm and I have been told the total opposite. So I try to differentiate each component when I am judging when the skater presents one component better than the other.
     
    alilou and (deleted member) like this.
  16. npavel

    npavel Well-Known Member

    1,488
    187
    63
    I really hope it is so it works, as it should. Otherwise the different entries don't make sense
     
  17. numbers123

    numbers123 Well-Known Member

    30,765
    5,026
    113
    It isn't going to happen, the system is in place with the CoP. Get over it. It's been 11 years. very few skaters (Plushy and Joubert are some who have) skated at the 6.0 scale. The kids know what they are to skate for. They understand it. They have choreographers who understand it. The fans are the ones who have difficulty in letting go of the past.

    What I understood from listening to the tech panel at US Nationals was that they had to score the same across the board. So if they determined something as a level or a completed jump or whatever, they needed to be consistent in calling.

    Not that they were locked in to scoring the same - the ordinal system seemed to be at the same level scoring - with more manipulation of the ordinals. I heard it said over and over again in the 6.0 system - the judges had to leave room for the last skaters.
     
  18. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    We're not talking about the tech panel calls, though, or even the judges' GOEs, which should be more objective.

    The program components are more subjective, so we would expect some more variation between judges. How much the system allows seems to be an issue. Are some judges taught that the most important thing is to stay within the corridor? Most judges I've heard from are more interested in reflecting what they see on the ice.

    Which comes back to my question, what do folks here think is the "correct" way to use program components? Are you using "correct" as shorthand for "wide variations between one component and another for the same program is always more appropriate than scores that are all in the same range"?

    Surely there must be some real reasons to assign each of the individual scores. Sometimes the different reasons would end up with similar numbers and sometimes not, depending how well-balanced the skaters skills are. So what should those reasons be based on?
     
  19. numbers123

    numbers123 Well-Known Member

    30,765
    5,026
    113
    I think that the program components scores are more subjective. I would liken the scoring variations similar to what the policeman running a driver safety course told me (us the class). There are some things that are technically defined, but each police officer, or in this case of skating judge, strikes a certain chord. For example, he said some officers are triggered to write a ticket, or in skating a downgrade, with a specific item/line that is crossed.

    So maybe for one judge a certain level of blade noise is acceptable whereas another judge not so much. How do you regulate one's observations and/or what their biggest pet peeve is? That the judge themselves should be reflective of consistency across the season or years. Just my opinion
     
  20. kwanatic

    kwanatic Well-Known Member

    2,039
    468
    83
    I agree that there are too many categories. There needs to be 2 or maybe 3 categories...not 5.

    SS & TR should be combined into one.

    IN & CH should be combined into one.

    PE stays the same but I'd be much happier if the mark actually reflected what was done on the ice. To me it rarely ever does. I see skaters deliver emotionally flat/blank performances (going through the motions) and yet they receive high marks. I don't understand it. PE should really reflect the impact performance as a whole...the rest of the bullet points can be calculated with SS/TR and IN/CH.
     
  21. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    duplicate
     
  22. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    Just from a mathematical point of view...

    Right now, using five components on a scale of 0-10, with factors for each discipline, competition level, and program, the total of the PCS available is supposed to be roughly equivalent to the TES earned at that skill level. (Clearly there are exceptions for skaters who are much better at elements that at performance, or vice versa, as well as good skaters who lose credit for many of their elements in ways that don't have significant effect on the component criteria, e.g., by doing the wrong type of spin or by violating the Zayak rule.)

    Currently the smallest increment that judges can use to distinguish between one skater and another who is slightly better or worse in that area is 0.25.

    For the senior men's short program, the factor is exactly 1.0 so that translates directly to 0.25 difference. For other types of programs, the factors may be smaller or larger, up to 2.0 for the senior men's freeskate. So for the men's free, already a difference of 0.25 in each judge's score for one component translates to a difference of 0.5 in the total score.

    If two components were combined into one, presumably the factor for that combined component would be doubled. (Or tripled if three components were combined.) So, e.g., in a senior men's LP, the factor for combined SS+TR would probably be 4.0, and ditto for IN+CH.

    Unless there were a decision to change the PCS weightings more in favor of technical skills between the elements, or more in favor of artistic skills.

    Using kwanatic's breakdown of three components equivalent to SS+TR, PE, and IN+CH, should each of those three groups be equal to each other -- i.e., PE now worth twice as much as each of the other four original components?

    With a factor of 4.0, that means that every time the judges on average agree that Peter was just enough better than Paul in that area to deserve a score one increment higher on that component, the effect on the score would be 1.0 points, not the 0.25 that the judges are thinking in terms of. There would be no way for any individual judge to indicate smaller differences.

    Allowing the judges to use 0.1 instead of 0.25 would mitigate that problem.

    Even if judges almost always seem to give practically identical SS and TR scores, or CH and IN scores, I think there are a couple of arguments in favor of keeping them separate.

    For the mathematical reasons I tried to lay out above, keeping the factors smaller allows judges to indicate finer distinctions than lumped-together scores.

    Also, the option to score TR higher or lower than SS, or IN higher or lower than CH, even if only by one increment, conveys more information. E.g., Peter was faster and smoother than Paul so he gets higher SS, but Paul had lots of good transitions and Peter didn't, so Paul gets the higher TR mark tells us more than averaging both areas into the same combined score for both skaters.

    Or Percy has good skills all around, everything pretty much at the same level in terms of basic skating, content between elements, carriage and style and energy and projection, construction of the program in time and space and general timing to the musical phrases. But the one area that he fell short was in reflecting the details and style of the music. So giving him the exact same scores for the first four components and somewhat lower for IN lets him know where he could use the most improvement.

    And, of course, sometimes skaters are significantly unbalanced in their skills and judges do reflect that in the PCS -- though not always to the extent everyone would like.

    With that in mind, the loss of detail and the loss of information that would result from combining two or more components, what would be the advantages to combining them?
     
  23. briancoogaert

    briancoogaert Well-Known Member

    12,544
    1,330
    113
    Is PE really needed ? It's already reflected in other marks. If you don't perform well, judges can give -GOE on technical elements, and lower the SS mark. IMO
     
  24. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    That's not everything that PE covers. In fact, that's not the main thing that it covers.

    It's entirely possible to have successful and even well-executed technical elements while doing a poor job on the actual, written Performance/Execution criteria. Or vice versa.

    Just as a reminder, the actual criteria for Performance/Execution are:

    There's actually nothing in the written criteria that refers to the success of the elements at all.

    It's likely that a program full of mistakes will have some disruptions of carriage and of clarity of movement; possibly in the physical, emotional, and intellectual involvement; and very likely in the unison between partners.

    So some judges may intentionally choose to penalize those disruptions in this mark for those reasons. But that's not what the P/E mark is designed to focus on.
     
  25. kwanatic

    kwanatic Well-Known Member

    2,039
    468
    83
    I think a lot of people don't realize that PE has nothing to do with what they think it has to do with.

    Performance and Execution seems like it should reflect the program as a whole...including the success/failure of the elements b/c if you're zamboni-ing the ice, your performance is obviously suffering and your execution is clearly off. When we look at clarity, variety, contrast and blah dee blah it all sounds superfluous. Shouldn't nearly all of those have been covered by one of the other components?

    Physical, emotional, and intellectual involvement - Interpretation

    Carriage - Skating Skills

    Style and individuality/personality - Interpretation

    Clarity of movement - Choreography

    Variety and contrast - Choreography

    I think Projection is the one bullet point in all of those criteria that should be evaluated as Performance/Execution. The rest of those seem redundant b/c they easily fit into and should be evaluated in one of the other categories. I still say they should go with 3 components: Transitions should be included in Skating Skills (1), Choreography and Interpretation (2) go hand in hand as one pretty much sucks without the other and Performance/Execution (3) should be about the performance and execution...not a variety of bullet points that should have been addressed in the other marks. They can find a way to make it all work out mathematically...
     
  26. lauravvv

    lauravvv Well-Known Member

    1,477
    127
    63
    Firstly, while skating skills can help to have good carriage, and vice versa, skating skills definitely are not all that carriage consists of. Secondly, clarity of movement is not the same as choreography. It is possible to execute good choreography with weak clarity of movement, as well as to execute weak choreography with great clarity of movement. Choreography is what the choreographer has intended, but clarity of movement is a part of skater's execution of this choreography, which is not one and the same. Of course clarity of movement/good execution can make weak choreography look better, and the other way round. But it's still not one and the same. Thirdly, skating skills and transitions is not one and the same either. Yes, most often good skating skills are needed to execute complicated transitions, so it's very likely that choreographers and coaches are not going to give complicated transitions to skaters who don't have good skating skills, and can't execute them. But sometimes skaters with good skating skills get programs with little transitions which doesn't mean that they are not showing their great basic skating skills in those programs. Finally, I have to repeat that choreography is what the choreographer (or choreographers) has intended, including how the movements should accentuate the music, while interpretation is a part of what the skater is doing (or not doing) with it. Different skaters can do the movements and nuances very differently - starting with robotic and barely there, and ending with an interpretation that exceeds the choreographer's idea/intention infusing the movement/choreography with skater's own personality. So, it's all connected, but not so completely that it could be judged as one.
     
    gkelly and (deleted member) like this.
  27. gkelly

    gkelly Well-Known Member

    10,804
    2,392
    113
    Just for fun, here are three lower-ranked ladies' short programs from the 1994 Olympics. I chose them for contrasts in strengths and weaknesses, by skaters who don't have much of a reputation among skating fans to interfere with honest assessments. Because these are from the 6.0 era, there's no right or wrong answer or preconception about how these performances "should" be scored on program components.

    Tsvetelina Abresheva

    Liudmila Ivanova

    Guona Zhao

    So what do you think? Care to choose one or two or all three and explain how you would score them on PCS?

    Use real-life 2013 rules and expectations (acknowledging that a spiral sequence was required in the 1994 technical program) and demonstrate the approach judges should be taking to scoring PCS.

    Or rewrite the rules to what you think they should be, and then show us how you would score under your rules.
     
  28. ciocio

    ciocio Active Member

    578
    38
    28
    SS-as it is, SS are very important in FS.
    TR-important, but I don't think we need a separate component, skaters are also rewarded in GOE for transitions.
    CH- stop rewarding the choreographers, please, I really don't care about Wilson, Nichol, etc.
    IN, PE- Could be scored together with CH and they'll reflect how the skater performed and executed the program, etc.
     
  29. kwanatic

    kwanatic Well-Known Member

    2,039
    468
    83
    It all sounds like splitting hairs to me. The fact that one is often directly related to or influenced by the other (skating skills and carriage, skating skills and transitions, choreography and clarity of movement, etc.) means they could simplify it by combining them together.

    Just my two cents.
     
  30. rayhaneh

    rayhaneh New Member

    452
    102
    0
    Except if you lump CH, IN and PE together, you'd diminish the impact the more "artistic" and creative aspects of figure skating have on the global mark, since SS and TR are more technical-related (even if they do influence the impact and quality of a program without a doubt). It's not exactly as if the current system really gives much leeway in terms of creativity anyway, so if on top of that you give less weight to the more artistic/choreographic/-related marks, there will be even less insentive for skaters to try and make an effort since it wouldn't make much of a difference on their total mark anyway

    Unless of course you'd put them together, but give them a more important coefficient to reflect the fact that they regroup a rather vast number of aspects of a program in order to keep a balance between the technical and the presentation/ "artistical" (this term being used loosely here). But I don't see how putting them together would be an improvement if I'm honest