PJ Kwong: What is wrong is the failure of some to learn how the sport is scored

Discussion in 'The Trash Can' started by Maofan7, May 4, 2013.

?

Is PJ Kwong right?

Poll closed Jun 4, 2013.
  1. Yes

    78 vote(s)
    56.5%
  2. No

    60 vote(s)
    43.5%
  3. Don't Know

    0 vote(s)
    0.0%
  1. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    Sorry but I thought not identifying judges came in before IJS and was a result of previous controversies. So how can that be blamed on IJS?

    Actually at most competitions, judges are identified, just not at the top international events. You actually have to go into the software and adjust the parameters to set up so judges are shown randomly on the protocols. So it is not a problem with IJS but rather a decision that has been made about it's application. So don't blame IJS, blame those who made the decision to not identify the judges at that level of competition.

    So for the majority of events in the world, the judges are identified and everyone knows who has given what marks.
    Last edited: May 8, 2013
  2. Marco

    Marco Missing Ziggy

    Joined:
    May 9, 2002
    Messages:
    11,250
    Corrupt / irresponsible / reputational / plain unreasonable judging isn't just a IJS problem. It's the same under 6.0s. But at least under IJS there are more published and even quantified criteria for everything and any clear deviation becomes obvious (not that they are going to do anything about it).

    After a seemingly clean performance, you used to get a 5.6/5.8 afterwards. You left the building not knowing exactly what you need to work on or why you didn't score higher. Now you get a TES and a PCS and then a protocol full of details afterwards. You know you received less scores because your jumps were downgraded or your spin did not count or you did not have sufficient skating skills or transitions. It's definitely more transparent now.

    Values and rules can always be tweaked (and judging will always be a problem in a subjective, judged sport - which isn't as black and white as who crossed the line first) but the concept of IJS is much better if you ask me.

    Back to the original question, I think PJ's point is valid but not full. The failure to learn the system could be due to:

    - its complexity (I personally think the system is simple enough),
    - the details being overly mathematically focused (except if you only focus on the final numbers, a 5.6/5.8 or 70/80 isn't all that different),
    - mass media not properly explaining things (how hard is it to explain that every element is scored based on difficulty and quality, and the skater's qualities and programs are also scored based on quality and complexity - at the end, the person with the most points win) or
    - a simple reluctance to learn (the thrill of the 6.0, reluctance to change, 6.0 being more predictable etc).
    gkelly and (deleted member) like this.
  3. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    I think the thing I see so much of here is people just apply their criticism to the international events. They really don't understand that this system goes right down the grass roots of the sport. So have no knowledge of the day to day application of it for the little club competitions that happen every week.
  4. umronnie

    umronnie Active Member

    Joined:
    Jan 3, 2005
    Messages:
    828
    Transparent? All the compexity of IJS, the rows after rows of numbers that make up the final score, make it so much easier to cheat.

    Two judges working together can dramatically change the result and not only can you not tell who gave which mark, you cannot even see that they were doing it. All they have to do is give GoEs 1 degree higher than is "warranted" and PCS scores only 0.25 more. Since there is a "marks corridor" anyway, and judges' individual scores can vary from -1 to +2, thus is undetectable, but what is the resulst?

    Suppose those two judges give the highest score on everything. One of their scores will be thrown out but the other will count (and since two judges "agreed" on the high score they certainly don't look out of place). With average GoE factor of 0.5 per element the additional +1s will result is 3.5 points in the SP, 6-6.5 points in the FS (12/13 elements). An additional 0.25 in PCS is worth 1/1.25 points in the SP and 2/2.5 in the FS. A total point difference of 12.5-13.75.
    Divide the additional points by the 7 remaining judges and you get nearly 2 points per skater. 2 points may not seem like much but the men's gold in London was decided by 1.3, the ladies silver/bronze by 0.93 and the pairs silver/bronze by 1.0...

    Now, to be more effective all our dou needs to do is do the same for their favorite's closest rival, but in reverse - 1 less GoE degree, 0.25 less PCS. The net difference is 3.5-4 points and that is a considerable margin, decided actually by one judge's marks (the other's marks get thrown away).

    There can be many arguments for IJS, but transparence and reliability is not one of them.
    Cherub721 and (deleted member) like this.
  5. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,512
    Well, I guess it depends what you mean by "transparent."

    Do we want to know the reasons for the results -- both what the judges thought about the elements and the program as a whole, and how much each part of the program was worth? In that case, a system that breaks down separate scores for each element and separate scores for each global aspect of the program is more transparent than one that gives a single number for technical merit and a single score for presentation.

    Do we want to know which individual official gave which set of marks? In that case, we want columns of scores to line up with the judges' names.

    These are two separate questions. It's possible to have one of these kinds of transparency but not the other, or both at the same time, or neither. In the past decade, we've seen all four possible permutations.

    With neither approach do we know exactly what judges (or technical panels) were thinking. But we have a lot more information with IJS than we do with 6.0.

    Neither of those kinds of transparency has anything to do with how easy it is to cheat. Officials who actively want to cheat will find a way to do so regardless of the system. It does make sense to try to build in safeguards to discourage people from trying, to catch them if they do try, and to minimize the effects on the final results if they succeed.

    However, when designing a system in the first place and deciding whether to implement it, the first question shouldn't be "How hard will it be to cheat, or to catch the cheaters?"

    The first questions should be "How well will this system work to evaluate the skating effectively when used honestly and to communicate those evaluations to the competitors and to the public?"

    As Aussie Willy notes, the vast majority of competitions below the senior international level use both forms of transparency: detailed marks AND all judges identified.

    If the detailed marks tell us more about the reasons behind the results, then it's worth using detailed marks.

    OK, so then, after we have a system that works well and communicates well when used honestly, we need to address the question of how to deal with the cases when people actively try to manipulate the system dishonestly. These cases are a minority, the exception rather than the rule, but they do occur more often at the highest levels where the stakes are highest, and so they gain more attention.

    Would minimizing the instances or effect of active cheating be more effective by identifying the judges or by obscuring their identities? The source of the dishonesty (individual judges' initiative vs. pressure from federations) may make a difference. Most of us here think identifying individuals would be better. Some decisionmakers within the ISU think otherwise.

    If we were all to agree on naming the judges and making responsibility for each mark transparent to anyone who reads the protocols, then we come back to the question of which approach to scoring is more effective for evaluating the skating and for communicating the reasons for the evaluations to the skaters and other observers when used honestly.

    Then we can get into questions of reliability in terms of honest evaluations. But "transparency" is a separate issue.
  6. leafygreens

    leafygreens Well-Known Member

    Joined:
    May 26, 2009
    Messages:
    1,648
    My iron lotus consists of :rolleyes: and skating off.
  7. Seerek

    Seerek Well-Known Member

    Joined:
    Nov 11, 2002
    Messages:
    3,441
    Interesting point about the regional levels, though there is the fact that the total raw scores are generally lower at the club level and therefore, the point gaps between the skaters is less, making any potential fluctuations in applying GOE, under-rotations and component scores more influential to the final placements (even with lower level spins and jumps).
  8. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    Cheating comes down to an individuals actions. So that is based on human decision making.

    Those who are arguing that IJS is the problem, they are still using arguments for problems that are based on human decision making to say the system is flawed.

    It is not the system, it is the application and decisions that are made by people about it's application that are the issue.

    Sorry but people need to stop confusing the actual system with the application of it.

    If anyone wants a copy of the software PM me and I will tell you how to get.
  9. Susan M

    Susan M Well-Known Member

    Joined:
    Mar 21, 2004
    Messages:
    1,457
    Doh. And how do you think those placements were decided? By adding up the points on that judge's score card. 5.8 +5.9 beats 5.8 +5.8. What they did not do is sum points across the whole panel, but yes points decided the placements for each judge. As I said originally, you won the event by winning the majority of the judges.

    There actually was a time when placements were summed, but they dumped that system when judges from places like East Germany kept giving inappropriately low placements to their skaters' rivals to bring down their totals. Then they went to the majority ordinals system where one or two out of line judges could not alter the result as easily.

    That said, I am not one who pines for the 6.0 system and thinks the sport will wander in the wilderness until they return to it. I think any system will have its advantages and disadvantages and that pretty much any system could be made to work. In the case of the current judging system, it needs to be fixed to work better.

    They need to start by taking a hard look at what passes for choreography these days and figure out what in the current point values and Program Component scoring has led us to these messes.
    Last edited: May 9, 2013
  10. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    It might be points but at the end of the day you are still trying to put skaters into places. And before IJS if you had a field of 30-40 skaters, judges had to remember all the skaters that had come before them to work out where that last skater might fit into the placings. Not exactly fare system is it because it was relying on memory of what had come before, not what the skater did on the ice.
  11. skateboy

    skateboy Well-Known Member

    Joined:
    Jun 12, 2006
    Messages:
    4,245
    Why? I judged high-level roller skating competitions with that system (well, basically 10.0). There were often 30-40 skaters. I simply wrote down my scores based on the quality of the performance, regardless of whether or not it was the first skater, one in the middle, or the last skater. Except in the case of maybe the top three skaters, I did not need to try and calculate how each skater was going to be ranked. The scores were the important thing, the rankings were just what they were. It was never up to me to figure out the ordinal placements. They were simply determined by my scores.

    How is that so different?
  12. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    Under 6.0 you are not allowed to tie a skater. So you constantly have to consider where you slot skaters into their placements. And allow room with the early skaters to slot in the later skaters. So no the scores do not determine the results. It is specifically placements. And when you are considering your marks you constantly have that in mind.
    Maofan7 and (deleted member) like this.
  13. skateboy

    skateboy Well-Known Member

    Joined:
    Jun 12, 2006
    Messages:
    4,245
    Not that I don't believe you, but I could swear I've seen a number of ties in 6.0 (from individual judges)...

    That wasn't an issue with roller competitions and, with 10.0, there was much more wiggle room.
  14. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    Well when I have used it for judging, I have been told you can't tie skaters. Ah but maybe if you have seen ties that is because it is very easy to give skaters the same marks if you have a large field. Just another of the many problems with the 6.0 system.
  15. VIETgrlTerifa

    VIETgrlTerifa Well-Known Member

    Joined:
    Jul 12, 2003
    Messages:
    9,977
    I wonder what would happen if the ISU would ever go back to the 6.0 (slim chance I know), and people complained about the results the way they are now. Will we see PJ Kwong blaming fans for simply not knowing the scoring system? Would we have apologists talking about the virtues of 6.0 over COP? Would we have detractors critiquing all the short-falls of 6.0? Basically, will we have the same arguments we're having now?
  16. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,512
    More or less, I expect.

    Some people who loved the 6.0 system and hate IJS might stop complaining and go back to enjoying everything they enjoyed about the 6.0 system, keeping any quibbles to themselves to show support for their favored system.

    People who love IJS and would be sorry to lose it might complain more about 6.0 results and processes than they would if they didn't feel that their favored system had been taken from them.

    People who don't care so much about the system but just love to complain would continue to complain. That includes people who understand and enjoy nitpicking about details, and also people who don't know much about skating technique and will probably be bewildered by results regardless of the scoring system.
  17. bek

    bek Guest

    Well it depends Aussie. You have many saying that the system insists that the words Performance/Execution don't mean that and that its perfectly acceptable according to the system for some to get enormously high P/e mark and other PCS for programs that are messy with glaring errors.

    If this is true than I think its perfectly fair for people to say the system is flawed. That the actual execution of the entire program should matter. And that skaters with high reputations shouldn't be guaranteed huge PCS gap no matter how they skate.

    If PCS like that are perfectly ridiculous and skaters should be marked down harder for bigger glaring issues. Than fine its the judging.

    Either I think what many are saying is they don't like the results the system is producing.

    I find it really insulting that the folks not understanding the system argument. As if the system is more important than skating itself.

    There is a problem and blaming fans isn't enough. Not to mention there are plenty of very knowledgable skaters complaining about the results. You can say they don't understand the system, but they certainly know skating. And they are perfectly entitled to say the results aren't right. Your not going to convince those people that somehow Patrick skated better overall even if the math as done by the system argues it.
    Last edited by a moderator: May 11, 2013
  18. PDilemma

    PDilemma Well-Known Member

    Joined:
    Apr 8, 2008
    Messages:
    4,763
    Exactly.
  19. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,017
    I would suggest stop trying to turn my posts into something they are not. I commented on people confusing human decision making and the system. And then trying to use arguments about human decision making as justification for why the system doesn't work. What you have said is totally irrelevant to my post.