View Poll Results: Is PJ Kwong right?

Voters
138. You may not vote on this poll
  • Yes

    78 56.52%
  • No

    60 43.48%
  • Don't Know

    0 0%
Page 5 of 5 FirstFirst ... 345
Results 81 to 99 of 99
  1. #81

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    Quote Originally Posted by vodkashot View Post
    Well, that's one way to frame it. Looking at it another way, 6.0 was away more transparent and accountable than IJS. You knew exactly who was giving what mark. You may not know how judges came up with their marks, but it's not like we know why or how judges come up with their marks under IJS either. You can see where the marks are distributed, but that's about it. Who knows why one judge gives an element a 0 in GOE, but another gives the exact same element a +2? Ditto for wide point spreads in PCS as well. And in some ways IJS is even worse than 6.0 because of the anonymous judging, which really is the epitome of zero accountability and transparency
    Sorry but I thought not identifying judges came in before IJS and was a result of previous controversies. So how can that be blamed on IJS?

    Actually at most competitions, judges are identified, just not at the top international events. You actually have to go into the software and adjust the parameters to set up so judges are shown randomly on the protocols. So it is not a problem with IJS but rather a decision that has been made about it's application. So don't blame IJS, blame those who made the decision to not identify the judges at that level of competition.

    So for the majority of events in the world, the judges are identified and everyone knows who has given what marks.
    Last edited by Aussie Willy; 05-08-2013 at 10:07 AM.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

  2. #82
    Registered User
    Join Date
    May 2002
    Location
    Hong Kong
    Posts
    11,193
    vCash
    500
    Rep Power
    0
    Corrupt / irresponsible / reputational / plain unreasonable judging isn't just a IJS problem. It's the same under 6.0s. But at least under IJS there are more published and even quantified criteria for everything and any clear deviation becomes obvious (not that they are going to do anything about it).

    After a seemingly clean performance, you used to get a 5.6/5.8 afterwards. You left the building not knowing exactly what you need to work on or why you didn't score higher. Now you get a TES and a PCS and then a protocol full of details afterwards. You know you received less scores because your jumps were downgraded or your spin did not count or you did not have sufficient skating skills or transitions. It's definitely more transparent now.

    Values and rules can always be tweaked (and judging will always be a problem in a subjective, judged sport - which isn't as black and white as who crossed the line first) but the concept of IJS is much better if you ask me.

    Back to the original question, I think PJ's point is valid but not full. The failure to learn the system could be due to:

    - its complexity (I personally think the system is simple enough),
    - the details being overly mathematically focused (except if you only focus on the final numbers, a 5.6/5.8 or 70/80 isn't all that different),
    - mass media not properly explaining things (how hard is it to explain that every element is scored based on difficulty and quality, and the skater's qualities and programs are also scored based on quality and complexity - at the end, the person with the most points win) or
    - a simple reluctance to learn (the thrill of the 6.0, reluctance to change, 6.0 being more predictable etc).

  3. #83

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    I think the thing I see so much of here is people just apply their criticism to the international events. They really don't understand that this system goes right down the grass roots of the sport. So have no knowledge of the day to day application of it for the little club competitions that happen every week.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

  4. #84
    Registered User
    Join Date
    Jan 2005
    Posts
    828
    vCash
    500
    Rep Power
    0
    Transparent? All the compexity of IJS, the rows after rows of numbers that make up the final score, make it so much easier to cheat.

    Two judges working together can dramatically change the result and not only can you not tell who gave which mark, you cannot even see that they were doing it. All they have to do is give GoEs 1 degree higher than is "warranted" and PCS scores only 0.25 more. Since there is a "marks corridor" anyway, and judges' individual scores can vary from -1 to +2, thus is undetectable, but what is the resulst?

    Suppose those two judges give the highest score on everything. One of their scores will be thrown out but the other will count (and since two judges "agreed" on the high score they certainly don't look out of place). With average GoE factor of 0.5 per element the additional +1s will result is 3.5 points in the SP, 6-6.5 points in the FS (12/13 elements). An additional 0.25 in PCS is worth 1/1.25 points in the SP and 2/2.5 in the FS. A total point difference of 12.5-13.75.
    Divide the additional points by the 7 remaining judges and you get nearly 2 points per skater. 2 points may not seem like much but the men's gold in London was decided by 1.3, the ladies silver/bronze by 0.93 and the pairs silver/bronze by 1.0...

    Now, to be more effective all our dou needs to do is do the same for their favorite's closest rival, but in reverse - 1 less GoE degree, 0.25 less PCS. The net difference is 3.5-4 points and that is a considerable margin, decided actually by one judge's marks (the other's marks get thrown away).

    There can be many arguments for IJS, but transparence and reliability is not one of them.

  5. #85

    Join Date
    Sep 2003
    Age
    53
    Posts
    10,472
    vCash
    500
    Rep Power
    21476
    Quote Originally Posted by umronnie View Post
    Transparent? All the compexity of IJS, the rows after rows of numbers that make up the final score, make it so much easier to cheat.
    Well, I guess it depends what you mean by "transparent."

    Do we want to know the reasons for the results -- both what the judges thought about the elements and the program as a whole, and how much each part of the program was worth? In that case, a system that breaks down separate scores for each element and separate scores for each global aspect of the program is more transparent than one that gives a single number for technical merit and a single score for presentation.

    Do we want to know which individual official gave which set of marks? In that case, we want columns of scores to line up with the judges' names.

    These are two separate questions. It's possible to have one of these kinds of transparency but not the other, or both at the same time, or neither. In the past decade, we've seen all four possible permutations.

    With neither approach do we know exactly what judges (or technical panels) were thinking. But we have a lot more information with IJS than we do with 6.0.

    Neither of those kinds of transparency has anything to do with how easy it is to cheat. Officials who actively want to cheat will find a way to do so regardless of the system. It does make sense to try to build in safeguards to discourage people from trying, to catch them if they do try, and to minimize the effects on the final results if they succeed.

    However, when designing a system in the first place and deciding whether to implement it, the first question shouldn't be "How hard will it be to cheat, or to catch the cheaters?"

    The first questions should be "How well will this system work to evaluate the skating effectively when used honestly and to communicate those evaluations to the competitors and to the public?"

    As Aussie Willy notes, the vast majority of competitions below the senior international level use both forms of transparency: detailed marks AND all judges identified.

    If the detailed marks tell us more about the reasons behind the results, then it's worth using detailed marks.

    OK, so then, after we have a system that works well and communicates well when used honestly, we need to address the question of how to deal with the cases when people actively try to manipulate the system dishonestly. These cases are a minority, the exception rather than the rule, but they do occur more often at the highest levels where the stakes are highest, and so they gain more attention.

    Would minimizing the instances or effect of active cheating be more effective by identifying the judges or by obscuring their identities? The source of the dishonesty (individual judges' initiative vs. pressure from federations) may make a difference. Most of us here think identifying individuals would be better. Some decisionmakers within the ISU think otherwise.

    If we were all to agree on naming the judges and making responsibility for each mark transparent to anyone who reads the protocols, then we come back to the question of which approach to scoring is more effective for evaluating the skating and for communicating the reasons for the evaluations to the skaters and other observers when used honestly.

    Then we can get into questions of reliability in terms of honest evaluations. But "transparency" is a separate issue.

  6. #86
    Registered User
    Join Date
    May 2009
    Location
    “It is far more important to have a good judge than a possible conflict of interest." - Ottavio Cinquanta
    Posts
    1,635
    vCash
    500
    Rep Power
    0
    Quote Originally Posted by Aussie Willy View Post
    I can do an upright spin in a public session and I get wows from those around me. Those who attend public sessions are generally impressed with anyone who can actually stand up.

    I want to see the video of your "iron lotus".
    My iron lotus consists of and skating off.

  7. #87
    Registered User
    Join Date
    Nov 2002
    Posts
    3,427
    vCash
    500
    Rep Power
    0
    Interesting point about the regional levels, though there is the fact that the total raw scores are generally lower at the club level and therefore, the point gaps between the skaters is less, making any potential fluctuations in applying GOE, under-rotations and component scores more influential to the final placements (even with lower level spins and jumps).

  8. #88

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    Cheating comes down to an individuals actions. So that is based on human decision making.

    Those who are arguing that IJS is the problem, they are still using arguments for problems that are based on human decision making to say the system is flawed.

    It is not the system, it is the application and decisions that are made by people about it's application that are the issue.

    Sorry but people need to stop confusing the actual system with the application of it.

    If anyone wants a copy of the software PM me and I will tell you how to get.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

  9. #89
    Registered User
    Join Date
    Mar 2004
    Location
    Northern Virginia
    Posts
    1,435
    vCash
    500
    Rep Power
    0
    The marks given under 6.0 had nothing to do with a skaters getting more points. It was a placement and ranking system
    Doh. And how do you think those placements were decided? By adding up the points on that judge's score card. 5.8 +5.9 beats 5.8 +5.8. What they did not do is sum points across the whole panel, but yes points decided the placements for each judge. As I said originally, you won the event by winning the majority of the judges.

    There actually was a time when placements were summed, but they dumped that system when judges from places like East Germany kept giving inappropriately low placements to their skaters' rivals to bring down their totals. Then they went to the majority ordinals system where one or two out of line judges could not alter the result as easily.

    That said, I am not one who pines for the 6.0 system and thinks the sport will wander in the wilderness until they return to it. I think any system will have its advantages and disadvantages and that pretty much any system could be made to work. In the case of the current judging system, it needs to be fixed to work better.

    They need to start by taking a hard look at what passes for choreography these days and figure out what in the current point values and Program Component scoring has led us to these messes.
    Last edited by Susan M; 05-09-2013 at 12:20 PM.

  10. #90

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    Quote Originally Posted by Susan M View Post
    Doh. And how do you think those placements were decided? By adding up the points on that judge's score card. 5.8 +5.9 beats 5.8 +5.8. What they did not do is sum points across the whole panel, but yes points decided the placements for each judge. As I said originally, you won the event by winning the majority of the judges.
    It might be points but at the end of the day you are still trying to put skaters into places. And before IJS if you had a field of 30-40 skaters, judges had to remember all the skaters that had come before them to work out where that last skater might fit into the placings. Not exactly fare system is it because it was relying on memory of what had come before, not what the skater did on the ice.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

  11. #91

    Join Date
    Jun 2006
    Location
    Partying with Oda
    Posts
    4,226
    vCash
    1799
    Rep Power
    34903
    Quote Originally Posted by Aussie Willy View Post
    It might be points but at the end of the day you are still trying to put skaters into places. And before IJS if you had a field of 30-40 skaters, judges had to remember all the skaters that had come before them to work out where that last skater might fit into the placings. Not exactly fare system is it because it was relying on memory of what had come before, not what the skater did on the ice.
    Why? I judged high-level roller skating competitions with that system (well, basically 10.0). There were often 30-40 skaters. I simply wrote down my scores based on the quality of the performance, regardless of whether or not it was the first skater, one in the middle, or the last skater. Except in the case of maybe the top three skaters, I did not need to try and calculate how each skater was going to be ranked. The scores were the important thing, the rankings were just what they were. It was never up to me to figure out the ordinal placements. They were simply determined by my scores.

    How is that so different?
    "I hit him with my shoes... if he had given me the medal like I told him to, I wouldn't have had to hit him!" -- 8-year-old Rhoda Penmark in "The Bad Seed"

  12. #92

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    Quote Originally Posted by skateboy View Post
    Why? I judged high-level roller skating competitions with that system (well, basically 10.0). There were often 30-40 skaters. I simply wrote down my scores. Except in the case of maybe the top three skaters, I did not need to try and calculate how each skater was going to be ranked. The scores were the important thing, the rankings were just what they were. It was never up to me to figure out the ordinal placements. They were simply determined by my scores.

    How is that so different?
    Under 6.0 you are not allowed to tie a skater. So you constantly have to consider where you slot skaters into their placements. And allow room with the early skaters to slot in the later skaters. So no the scores do not determine the results. It is specifically placements. And when you are considering your marks you constantly have that in mind.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

  13. #93

    Join Date
    Jun 2006
    Location
    Partying with Oda
    Posts
    4,226
    vCash
    1799
    Rep Power
    34903
    Quote Originally Posted by Aussie Willy View Post
    Under 6.0 you are not allowed to tie a skater. So you constantly have to consider where you slot skaters into their placements. And allow room with the early skaters to slot in the later skaters. So no the scores do not determine the results. It is specifically placements. And when you are considering your marks you constantly have that in mind.
    Not that I don't believe you, but I could swear I've seen a number of ties in 6.0 (from individual judges)...

    That wasn't an issue with roller competitions and, with 10.0, there was much more wiggle room.
    "I hit him with my shoes... if he had given me the medal like I told him to, I wouldn't have had to hit him!" -- 8-year-old Rhoda Penmark in "The Bad Seed"

  14. #94

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    Quote Originally Posted by skateboy View Post
    Not that I don't believe you, but I could swear I've seen a number of ties in 6.0 (from individual judges)...
    Well when I have used it for judging, I have been told you can't tie skaters. Ah but maybe if you have seen ties that is because it is very easy to give skaters the same marks if you have a large field. Just another of the many problems with the 6.0 system.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

  15. #95

    Join Date
    Jul 2003
    Location
    I Want to Go to There
    Posts
    9,863
    vCash
    500
    Rep Power
    40900
    I wonder what would happen if the ISU would ever go back to the 6.0 (slim chance I know), and people complained about the results the way they are now. Will we see PJ Kwong blaming fans for simply not knowing the scoring system? Would we have apologists talking about the virtues of 6.0 over COP? Would we have detractors critiquing all the short-falls of 6.0? Basically, will we have the same arguments we're having now?

  16. #96

    Join Date
    Sep 2003
    Age
    53
    Posts
    10,472
    vCash
    500
    Rep Power
    21476
    More or less, I expect.

    Some people who loved the 6.0 system and hate IJS might stop complaining and go back to enjoying everything they enjoyed about the 6.0 system, keeping any quibbles to themselves to show support for their favored system.

    People who love IJS and would be sorry to lose it might complain more about 6.0 results and processes than they would if they didn't feel that their favored system had been taken from them.

    People who don't care so much about the system but just love to complain would continue to complain. That includes people who understand and enjoy nitpicking about details, and also people who don't know much about skating technique and will probably be bewildered by results regardless of the scoring system.

  17. #97

    Join Date
    Nov 2005
    Posts
    17,020
    vCash
    1561
    Rep Power
    4990
    Quote Originally Posted by Aussie Willy View Post
    Cheating comes down to an individuals actions. So that is based on human decision making.

    Those who are arguing that IJS is the problem, they are still using arguments for problems that are based on human decision making to say the system is flawed.

    It is not the system, it is the application and decisions that are made by people about it's application that are the issue.

    Sorry but people need to stop confusing the actual system with the application of it.

    If anyone wants a copy of the software PM me and I will tell you how to get.
    Well it depends Aussie. You have many saying that the system insists that the words Performance/Execution don't mean that and that its perfectly acceptable according to the system for some to get enormously high P/e mark and other PCS for programs that are messy with glaring errors.

    If this is true than I think its perfectly fair for people to say the system is flawed. That the actual execution of the entire program should matter. And that skaters with high reputations shouldn't be guaranteed huge PCS gap no matter how they skate.

    If PCS like that are perfectly ridiculous and skaters should be marked down harder for bigger glaring issues. Than fine its the judging.

    Either I think what many are saying is they don't like the results the system is producing.

    I find it really insulting that the folks not understanding the system argument. As if the system is more important than skating itself.

    There is a problem and blaming fans isn't enough. Not to mention there are plenty of very knowledgable skaters complaining about the results. You can say they don't understand the system, but they certainly know skating. And they are perfectly entitled to say the results aren't right. Your not going to convince those people that somehow Patrick skated better overall even if the math as done by the system argues it.
    Last edited by bek; 05-11-2013 at 01:47 AM.

  18. #98
    Registered User
    Join Date
    Apr 2008
    Location
    Midwest
    Posts
    4,750
    vCash
    500
    Rep Power
    0
    Quote Originally Posted by PrincessLeppard View Post
    I voted no, because the main problem with the scoring system (imho, of course ) is that components aren't scored correctly.
    Exactly.

  19. #99

    Join Date
    Feb 2005
    Location
    Melbourne, Australia
    Age
    48
    Posts
    17,941
    vCash
    500
    Rep Power
    36617
    Quote Originally Posted by bek View Post
    Well it depends Aussie. You have many saying that the system insists that the words Performance/Execution don't mean that and that its perfectly acceptable according to the system for some to get enormously high P/e mark and other PCS for programs that are messy with glaring errors.

    If this is true than I think its perfectly fair for people to say the system is flawed. That the actual execution of the entire program should matter. And that skaters with high reputations shouldn't be guaranteed huge PCS gap no matter how they skate.

    If PCS like that are perfectly ridiculous and skaters should be marked down harder for bigger glaring issues. Than fine its the judging.

    Either I think what many are saying is they don't like the results the system is producing.

    I find it really insulting that the folks not understanding the system argument. As if the system is more important than skating itself.

    There is a problem and blaming fans isn't enough. Not to mention there are plenty of very knowledgable skaters complaining about the results. You can say they don't understand the system, but they certainly know skating. And they are perfectly entitled to say the results aren't right. Your not going to convince those people that somehow Patrick skated better overall even if the math as done by the system argues it.
    I would suggest stop trying to turn my posts into something they are not. I commented on people confusing human decision making and the system. And then trying to use arguments about human decision making as justification for why the system doesn't work. What you have said is totally irrelevant to my post.
    When you are up to your arse in alligators it is difficult to remember you were only meant to be draining the swamp.

Page 5 of 5 FirstFirst ... 345

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •