ISU Posts "Calculation Error" in GPF FD

[kwanfan1818]

Any dance team who is concerned and to whom it is applicable will check their own protocols and raise the issue if there is one, especially if the scores could be borderline for making the GP minimum (not yet set for 2012) or top 24 SB.

ETA: A random check is the usual test done for transactions that are calculated before a change was made, to confirm that the old calculations are valid.

[love_skate2011]

hmmm. interesting
anybody by how many points
maybe they will post a corrected result. lol

[The Accordion]

And I believe that fairies live in the woods and that Donald Trump is going to give me all his money tomorrow.

LUCKY!

[overedge]

A random check is the usual test done for transactions that are calculated before a change was made, to confirm that the old calculations are valid.

Random sampling may be the usual method of testing, but random sampling is not appropriate in situations like this, because of the potential consequences associated with results that weren't chosen in the sample.

[Japanfan]

So how are the ice dancers supposed to be reassured by a "random check"? What if a team's result is wrong and it wasn't one of the results that was randomly chosen? What if a team's placement is correct but the actual score isn't? Yet again, the ISU acts like everyone is supposed to be satisfied with what is really a completely inadequate response.

The ISU needs to go back and recalculate the results of every program in every competition where the sekret computer was programmed with this error. It's the only fair way to resolve the situation.

ITA.

A random check is the usual test done for transactions that are calculated before a change was made, to confirm that the old calculations are valid.

But didn't this check apply to programs skated after the change was made?

It's true that any teams which were affected will let the ISU show and request an adjustment. However, the ISU is responsible to thoroughly investigate the issue as the ISU made the mistake. I would think it is very easy for them to search their records and identify the teams which did a combination lift, then examine the scores.

[kwanfan1818]

But didn't this check apply to programs skated after the change was made?

Testing works on the same principal as an audit: a small sample checked is indicative of the whole.

There are at least two possibilities for what they did, and since the ISU is inarticulate in general about what it is doing, it's hard to tell exactly what the circumstances are.

For example, it's possible that the correct GOE (variable) was in the system before a change was made, reverting it to the old value. Part of regression testing after the variable is corrected is to do a random test among data that should not be changed, to be sure they haven't changed.

Another possibility is that they changed a global reference used each time the element appeared in the records or, more likely, through an update script that recalculated either all entries in the database for that element using the new GOE variable.

Before a script is run, they likely would do a query to see how many records should be updated.

Then they'd run a script like: Where Element = [Lift Abbreviation] recalculate element score using new GOE info and then calculate new final score.

The script returns stats such as the number of records updated with the matching criteria, which is compared to the original query. If 153 records matched the query, and 153 records were updated, the assumption is the update worked. They could also capture the old and new scores and compare them in a report.

If there were hundreds of entries using that element, they would randomly check a subset of those entries, because if the script worked for one, and on the correct number of records, it should work the same way for them all.

[manhn]

I don't know. I can imagine the media/public reaction had the error occurred in a bigger event like Worlds or the Olympics--not that the GPF is exactly tiny--if Cinquata said "Well, this is how accountants do it!"

[kwanfan1818]

Accountants are good enough for Miss America (And "Battle of the Blades".) But, of course, accountants aren't the only people who do internal audits. They'd have to say, "Google does it this way."

[casken]

Well this explains the sudden reemergence of combo lifts in Shpilband FD's.

[DORISPULASKI]

But it doesn't explain why no one noticed the scores were wrongt after the first competitions of the year and complain publicly.

In each calculation, the high & low judge's score is dropped, so for counting scores at Skate America, D&W's rotational lift got 5.5, with all 3's, base level 4.0. So max GOE 1.5, and for two such lifts, 11.0.

P&B's combo lift got 9.70 with all twos, on a base level of 8. That's .85 per GOE level, so you can extrapolate that for all 3's, a combo lift would have a GOE of 2.55, and the max score 10.55 rounded down to 10.5, for a perfect combo lift.

So you could see the scoring was the same as last year at the very first event of the season.


In fact, looking at any combo lift, you could see that the GOE given was less than the average of the scores given, meaning that the GOE had not been corrected to give +3 for a perfect lift (it still was 2.5). All 1's would give +.85, rather than 1.0, as it now should.

And that it did not affect just V&M, but every skater at every level that included a combo lift in their program, at every event using the sportcentric software.

(And in fact, I made such a check at that time, and thought, I guess they decided not to fix the lift scoring discrepancy, rather than thinking the program was not fixed)

[Japanfan]

Another possibility is that they changed a global reference used each time the element appeared in the records or, more likely, through an update script that recalculated either all entries in the database for that element using the new GOE variable.

Before a script is run, they likely would do a query to see how many records should be updated.

Then they'd run a script like: Where Element = [Lift Abbreviation] recalculate element score using new GOE info and then calculate new final score.

The script returns stats such as the number of records updated with the matching criteria, which is compared to the original query. If 153 records matched the query, and 153 records were updated, the assumption is the update worked. They could also capture the old and new scores and compare them in a report.

If there were hundreds of entries using that element, they would randomly check a subset of those entries, because if the script worked for one, and on the correct number of records, it should work the same way for them all.

What you've described is a comprehensive process as all the programs which had to be recalculated due to that element were updated. So the score of __ programs - your example being 153 - would have been possibly changed.

The random check would affirm that the recalculations were correct, but all the teams involved would be be able to check if their score had changed.

Perhaps this is what happened, but it wasn't clear in the ISU communication.

[RunnersHigh]

It is important to point out however that the final ranking (Short Dance and Free Dance combined) has not changed.

Hmmm...

[soxxy]

But the ranking of the Free Dance has changed. Will that be reflected in the ISU records, or will this notice suffice?

[TAHbKA]

Wow... Just wow.
But does it mean this particular competition the ISU computers were not updated with the latest software? Or this particular rule was not added to the software and then indeed all the competitions results should be overlooked?

[DORISPULASKI]

At Skate America, the same scoring error exists. I would assume it would be so for all competitions between Skate America and the GPF as well, why not?

And since the same scoring package is used by federations for Senior B's and nationals, probably there too, although I don't know.

For a combo lift, one point of GOE is supposed to be 1.0 points. And it isn't. It's something like 0.85 or so (they seem to use an odd rounding scheme). So the scores should be off for every team doing a combo lift; about 1/2 the entrants per event. Only teams that got 0 GOE for the combo lift would be correct, and teams who didn't do one.

[kates8]

At Skate America, the same scoring error exists. I would assume it would be so for all competitions between Skate America and the GPF as well, why not?

And since the same scoring package is used by federations for Senior B's and nationals, probably there too, although I don't know.

For a combo lift, one point of GOE is supposed to be 1.0 points. And it isn't. It's something like 0.85 or so (they seem to use an odd rounding scheme). So the scores should be off for every team doing a combo lift; about 1/2 the entrants per event. Only teams that got 0 GOE for the combo lift would be correct, and teams who didn't do one.

Does this problem exists all the way through the "Non-ISU" competitions --- all the way down to Sectionals, Regionals, Nationals etc?

[DORISPULASKI]

The question is whether the software is the same exact package used by the ISU. In the US, there are 6 judges at Sectionals. I presume they use all the values, but don't know that for sure..



Let's look at Midwestern Sectionals in the US.
The Hubbells won.
http://www.usfigureskating.org/leade...5/results.html

They did a combination lift, with a calculated GOE of 1.03.
There are only 6 judges.
1 0 1 1 2 2

If you average all 6 and ask, did they use +1 per point of GOE, the average is 7/6=
1.17
If you drop the low & the high and average, you have 5/4 for an average of 1.2

Either way it does not look like 1.03.

If you assume about .85 points per GOE point (where +3 is V&M at 2.5 , you get either
1.17*.84 = 1.0
or 1.2*.84=1.02

Note that 2.5/3 =.83333333, So there is some odd rouding going on in their program.

Samuelson & Gilles at Pacifics
1 1 1 1 2 2 also got GOE 1.03

So it looks like in the US, they have the same issue.

(I only did the case of a combo lift with two level 4 lifts. For lower levels, the tariff would be different.

[bmcg]

I think it was just noticed now because no one looked that closely before.

They said after GPF they were going to go over the numbers to see where they can make up points. They probably noticed the .5 point difference for doing a combo, questions were asked and the error detected.

Good thing it was picked up now where it didn't affect overall standings.

[overedge]

They said after GPF they were going to go over the numbers to see where they can make up points. They probably noticed the .5 point difference for doing a combo, questions were asked and the error detected.

Who are "they"?

Good thing it was picked up now where it didn't affect overall standings.

But if the wrong calculation has been used for most of the season to date, we don't know that it didn't affect overall standings.

[bmcg]

They are Virtue and Moir, they said they were going to go over the numbers. Looks like they did.

I also think any team that feels they might have been affected by this error will look into their own situation. I was referring to this specific situation where the two teams (V&M and D&W) are separated by 10's of a point.