ISU Posts "Calculation Error" in GPF FD

Discussion in 'Great Skate Debate' started by soxxy, Dec 29, 2011.

  1. soxxy

    soxxy Guest

  2. Louise

    Louise Banned Member

    Joined:
    Sep 24, 2003
    Messages:
    560
    Oh boy, can you imagine if this happened a couple weeks after the Olympics and the results were different? Not good. Reminds me somewhat of how Hamm won his all around Oly title by mathematical error.
  3. peibeck

    peibeck Counting down the days 'til Skate America

    Joined:
    Sep 3, 2004
    Messages:
    16,121
    Hmmm, I wonder how could the calculation error have been in the program for only one comp?

    I foresee lots of :drama: already.
  4. Lacey

    Lacey Well-Known Member

    Joined:
    Jul 11, 2007
    Messages:
    8,559
    Wow, just wow.
  5. Japanfan

    Japanfan Well-Known Member

    Joined:
    Mar 1, 2002
    Messages:
    12,779
    Scott Moir must be feeling very vindicated and having a very good day.

    What does 'previous' mean? It would seem it refers to a rule change that was implemented this season? In that case, might there not be calculation errors in other programs as well?
    PeterG and (deleted member) like this.
  6. The Accordion

    The Accordion Well-Known Member

    Joined:
    Sep 21, 2005
    Messages:
    3,417
    Aha! A pissoff!
  7. aka_gerbil

    aka_gerbil Rooting for the Underdogs

    Joined:
    Jan 25, 2008
    Messages:
    1,589
    It seems that the GOE for combination lifts was changed in the rules between last season and this season, but that change somehow didn't make it into the sekret computer.

    They said that they went back through other results from this season and didn't find another instance where the miscalculation affected the placements.
  8. shuilee

    shuilee New Member

    Joined:
    May 25, 2002
    Messages:
    3,166
    I wonder if Weaver & Poje should have won NHK gold medal over Shibs because of the computer bug.....they did lose by 0.1...they'll have to review all the competitions this year!

    Hopefully someone knows more about the details of this error. What element specifically was it, and did they use the wrong GoE multiplication factor, etc?
  9. caseyedwards

    caseyedwards Well-Known Member

    Joined:
    Dec 23, 2009
    Messages:
    6,356
    Interesting! So D/W has been beaten this season in something! VM is closer.
    Last edited: Dec 29, 2011
  10. modern_muslimah

    modern_muslimah Well-Known Member

    Joined:
    Apr 26, 2006
    Messages:
    3,908
    V/M and D/W were always very close this season. The difference would've been 0.5 points. Either way, these two teams are going to be neck and neck the entire season and probably all the way to Sochi.
    flutzilla1 and (deleted member) like this.
  11. Louise

    Louise Banned Member

    Joined:
    Sep 24, 2003
    Messages:
    560
    RIGHT!!! :rofl::rofl::rofl::rofl:
  12. danafan

    danafan Canadian ladies ├╝ber

    Joined:
    Sep 18, 2004
    Messages:
    3,415
    Placements, sure but what about accurate season and personal best scores? Will they be corrected?

    Is there any current avenue for skaters or coaches to protest such errors in relative real time?
  13. overedge

    overedge Well-Known Member

    Joined:
    Jan 21, 2005
    Messages:
    17,494
    And I believe that fairies live in the woods and that Donald Trump is going to give me all his money tomorrow.
  14. iggie

    iggie New Member

    Joined:
    Aug 20, 2007
    Messages:
    1,447
    they only did random checks of previous events, nothing thorough.
  15. Macassar88

    Macassar88 Well-Known Member

    Joined:
    Dec 24, 2010
    Messages:
    1,827
    Weaver and Poje did two simple lifts instead of a combo
  16. Jenna

    Jenna Well-Known Member

    Joined:
    Jun 20, 2009
    Messages:
    7,139
    Interesting...

    Imagine if it changed the overall result. I just can't believe things like this happen..
  17. aka_gerbil

    aka_gerbil Rooting for the Underdogs

    Joined:
    Jan 25, 2008
    Messages:
    1,589
    I should have read a little closer.

    As for the rest of it, I said that they said they checked. Not that they actually did. :p

    At any rate, I'm sure everyone, their dog, and their grandmother are re-checking their scores themselves if they think they might benefit.

    I wonder what else is wrong in the computer...
  18. Aussie Willy

    Aussie Willy Well-Known Member

    Joined:
    Feb 18, 2005
    Messages:
    18,029
    It would be a simple matter of once you have changed the particular value in the system that you then it would automatically recalculate the results for each event that this applied to. So it is quite feasible that they haven't found any other problems with other events.

    However many updates do get issued during the year for the software with various adjustments and corrections. So it is not unusual for this to happen.
  19. iggie

    iggie New Member

    Joined:
    Aug 20, 2007
    Messages:
    1,447
    feasible, but this is what the isu wrote:
  20. overedge

    overedge Well-Known Member

    Joined:
    Jan 21, 2005
    Messages:
    17,494
    So how are the ice dancers supposed to be reassured by a "random check"? What if a team's result is wrong and it wasn't one of the results that was randomly chosen? What if a team's placement is correct but the actual score isn't? Yet again, the ISU acts like everyone is supposed to be satisfied with what is really a completely inadequate response.

    The ISU needs to go back and recalculate the results of every program in every competition where the sekret computer was programmed with this error. It's the only fair way to resolve the situation.
  21. kwanfan1818

    kwanfan1818 I

    Joined:
    May 24, 2003
    Messages:
    19,438
    Any dance team who is concerned and to whom it is applicable will check their own protocols and raise the issue if there is one, especially if the scores could be borderline for making the GP minimum (not yet set for 2012) or top 24 SB.

    ETA: A random check is the usual test done for transactions that are calculated before a change was made, to confirm that the old calculations are valid.
    Last edited: Dec 29, 2011
  22. love_skate2011

    love_skate2011 Well-Known Member

    Joined:
    Oct 30, 2011
    Messages:
    1,476
    hmmm. interesting
    anybody by how many points
    maybe they will post a corrected result. lol
  23. The Accordion

    The Accordion Well-Known Member

    Joined:
    Sep 21, 2005
    Messages:
    3,417
    LUCKY! :)
  24. overedge

    overedge Well-Known Member

    Joined:
    Jan 21, 2005
    Messages:
    17,494
    Random sampling may be the usual method of testing, but random sampling is not appropriate in situations like this, because of the potential consequences associated with results that weren't chosen in the sample.
  25. Japanfan

    Japanfan Well-Known Member

    Joined:
    Mar 1, 2002
    Messages:
    12,779
    ITA.

    But didn't this check apply to programs skated after the change was made?

    It's true that any teams which were affected will let the ISU show and request an adjustment. However, the ISU is responsible to thoroughly investigate the issue as the ISU made the mistake. I would think it is very easy for them to search their records and identify the teams which did a combination lift, then examine the scores.
  26. kwanfan1818

    kwanfan1818 I

    Joined:
    May 24, 2003
    Messages:
    19,438
    Testing works on the same principal as an audit: a small sample checked is indicative of the whole.

    There are at least two possibilities for what they did, and since the ISU is inarticulate in general about what it is doing, it's hard to tell exactly what the circumstances are.

    For example, it's possible that the correct GOE (variable) was in the system before a change was made, reverting it to the old value. Part of regression testing after the variable is corrected is to do a random test among data that should not be changed, to be sure they haven't changed.

    Another possibility is that they changed a global reference used each time the element appeared in the records or, more likely, through an update script that recalculated either all entries in the database for that element using the new GOE variable.

    Before a script is run, they likely would do a query to see how many records should be updated.

    Then they'd run a script like: Where Element = [Lift Abbreviation] recalculate element score using new GOE info and then calculate new final score.

    The script returns stats such as the number of records updated with the matching criteria, which is compared to the original query. If 153 records matched the query, and 153 records were updated, the assumption is the update worked. They could also capture the old and new scores and compare them in a report.

    If there were hundreds of entries using that element, they would randomly check a subset of those entries, because if the script worked for one, and on the correct number of records, it should work the same way for them all.
  27. manhn

    manhn Well-Known Member

    Joined:
    Jan 18, 2002
    Messages:
    9,190
    I don't know. I can imagine the media/public reaction had the error occurred in a bigger event like Worlds or the Olympics--not that the GPF is exactly tiny--if Cinquata said "Well, this is how accountants do it!"
  28. kwanfan1818

    kwanfan1818 I

    Joined:
    May 24, 2003
    Messages:
    19,438
    Accountants are good enough for Miss America :) (And "Battle of the Blades".) But, of course, accountants aren't the only people who do internal audits. They'd have to say, "Google does it this way."
  29. casken

    casken Well-Known Member

    Joined:
    Oct 17, 2004
    Messages:
    6,325
    Well this explains the sudden reemergence of combo lifts in Shpilband FD's.
  30. DORISPULASKI

    DORISPULASKI Watching submarine races

    Joined:
    Mar 27, 2002
    Messages:
    9,884
    But it doesn't explain why no one noticed the scores were wrongt after the first competitions of the year and complain publicly.

    In each calculation, the high & low judge's score is dropped, so for counting scores at Skate America, D&W's rotational lift got 5.5, with all 3's, base level 4.0. So max GOE 1.5, and for two such lifts, 11.0.

    P&B's combo lift got 9.70 with all twos, on a base level of 8. That's .85 per GOE level, so you can extrapolate that for all 3's, a combo lift would have a GOE of 2.55, and the max score 10.55 rounded down to 10.5, for a perfect combo lift.

    So you could see the scoring was the same as last year at the very first event of the season.


    In fact, looking at any combo lift, you could see that the GOE given was less than the average of the scores given, meaning that the GOE had not been corrected to give +3 for a perfect lift (it still was 2.5). All 1's would give +.85, rather than 1.0, as it now should.

    And that it did not affect just V&M, but every skater at every level that included a combo lift in their program, at every event using the sportcentric software.

    (And in fact, I made such a check at that time, and thought, I guess they decided not to fix the lift scoring discrepancy, rather than thinking the program was not fixed)
    Last edited: Dec 29, 2011
  31. Japanfan

    Japanfan Well-Known Member

    Joined:
    Mar 1, 2002
    Messages:
    12,779
    What you've described is a comprehensive process as all the programs which had to be recalculated due to that element were updated. So the score of __ programs - your example being 153 - would have been possibly changed.

    The random check would affirm that the recalculations were correct, but all the teams involved would be be able to check if their score had changed.

    Perhaps this is what happened, but it wasn't clear in the ISU communication.
  32. RunnersHigh

    RunnersHigh Active Member

    Joined:
    Jun 29, 2009
    Messages:
    2,979
    Hmmm...
    PeterG and (deleted member) like this.
  33. soxxy

    soxxy Guest

    But the ranking of the Free Dance has changed. Will that be reflected in the ISU records, or will this notice suffice?
  34. TAHbKA

    TAHbKA Well-Known Member

    Joined:
    Mar 11, 2002
    Messages:
    8,836
    Wow... Just wow.
    But does it mean this particular competition the ISU computers were not updated with the latest software? Or this particular rule was not added to the software and then indeed all the competitions results should be overlooked?
  35. DORISPULASKI

    DORISPULASKI Watching submarine races

    Joined:
    Mar 27, 2002
    Messages:
    9,884
    At Skate America, the same scoring error exists. I would assume it would be so for all competitions between Skate America and the GPF as well, why not?

    And since the same scoring package is used by federations for Senior B's and nationals, probably there too, although I don't know.

    For a combo lift, one point of GOE is supposed to be 1.0 points. And it isn't. It's something like 0.85 or so (they seem to use an odd rounding scheme). So the scores should be off for every team doing a combo lift; about 1/2 the entrants per event. Only teams that got 0 GOE for the combo lift would be correct, and teams who didn't do one.
    geoskate and (deleted member) like this.
  36. kates8

    kates8 Member

    Joined:
    Oct 19, 2004
    Messages:
    214
    Does this problem exists all the way through the "Non-ISU" competitions --- all the way down to Sectionals, Regionals, Nationals etc?
    Last edited: Dec 29, 2011
  37. DORISPULASKI

    DORISPULASKI Watching submarine races

    Joined:
    Mar 27, 2002
    Messages:
    9,884
    The question is whether the software is the same exact package used by the ISU. In the US, there are 6 judges at Sectionals. I presume they use all the values, but don't know that for sure..



    Let's look at Midwestern Sectionals in the US.
    The Hubbells won.
    http://www.usfigureskating.org/leaderboard/results/2012/68255/results.html

    They did a combination lift, with a calculated GOE of 1.03.
    There are only 6 judges.
    1 0 1 1 2 2

    If you average all 6 and ask, did they use +1 per point of GOE, the average is 7/6=
    1.17
    If you drop the low & the high and average, you have 5/4 for an average of 1.2

    Either way it does not look like 1.03.

    If you assume about .85 points per GOE point (where +3 is V&M at 2.5 , you get either
    1.17*.84 = 1.0
    or 1.2*.84=1.02

    Note that 2.5/3 =.83333333, So there is some odd rouding going on in their program.

    Samuelson & Gilles at Pacifics
    1 1 1 1 2 2 also got GOE 1.03

    So it looks like in the US, they have the same issue.

    (I only did the case of a combo lift with two level 4 lifts. For lower levels, the tariff would be different.
    Last edited: Dec 29, 2011
  38. bmcg

    bmcg Well-Known Member

    Joined:
    Dec 28, 2011
    Messages:
    1,115
    I think it was just noticed now because no one looked that closely before.

    They said after GPF they were going to go over the numbers to see where they can make up points. They probably noticed the .5 point difference for doing a combo, questions were asked and the error detected.

    Good thing it was picked up now where it didn't affect overall standings.
  39. overedge

    overedge Well-Known Member

    Joined:
    Jan 21, 2005
    Messages:
    17,494
    Who are "they"?

    But if the wrong calculation has been used for most of the season to date, we don't know that it didn't affect overall standings.
  40. bmcg

    bmcg Well-Known Member

    Joined:
    Dec 28, 2011
    Messages:
    1,115
    They are Virtue and Moir, they said they were going to go over the numbers. Looks like they did.

    I also think any team that feels they might have been affected by this error will look into their own situation. I was referring to this specific situation where the two teams (V&M and D&W) are separated by 10's of a point.