Rossano: Current Replay Systems Not Up To Task of Insuring Accurate Calls

[Japanfan]

Not to mention that the excessive emphasis on jumps have been a detriment to everything else that was great about skating. I rather see more of an focus on accurately scoring skating skills.

There's been an excessive emphasis on jumps ever since I first watched in '94 (though not regularly until '98). It was jumps that earned Stojko his titles/medals.

Of course there was period of time in the Kwan era when 3-2s were the standard in ladies, for too long. But then the judges could theoretically compare apples to apples. And of course, soon some ladies saw they could gain an advantage with 3-3s, and in a few cases, the quad.

I don't enjoy skating under CoP any less than I did under 6.0. And, I find the jumps exciting.

What does bother me about FS is that so many skaters use the same music. I would really really like to see more creativity in that regard, and more innovation in programs, insofar as that is possible.

 

[clairecloutier]

I don't think the timeline is quite that.

When the scandal happened there were a number of articles such as this quoting Rogge, demanding the ISU clean up the sport or it would risk losing Olympic status. Many of the articles pointed at Ice Dance as the first likely to go.

My understanding is that Ted Barton was part of the team that wrote the program for IJS and he wasn't working on this prior to Salt Lake. This ended up being a rushed initiative to get in place and sort of 'working' by Torino, because Speedy was afraid of Rogge and another scandal. Sonia Bianchetti lays this out in an interview here.

Now, what Speedy did like about the new system was the anonymity of judges, so he did rush the new system through because it would mask block judging. He ended up supporting what was created because it would do just enough to save face for him in the eyes of the IOC.

There is no indication this system was ready before Salt Lake City happened. I mean this sport took decades to approve vocals in music. I have a hard time believing they would move to revolutionize the scoring system with no crisis to inspire the work and expense.

This is what I’ve heard as well. The Salt Lake City scandal happened in February 2002. At Junior Worlds the next month in March 2002, three people were holed up in a back room to pull together the basics of IJS ....

 

[gkelly]

At the ice dance technical committee meeting with competitors and journalists at 1998 Worlds (which I attended as a journalist), Jean Senft mentioned that there beginnings of development of a new scoring system that would separate the technical and presentation sides of the scoring.

I don't remember the exact wording of what she said. At the time, I understood it to mean that one panel would award a 6.0-style Technical Merit mark and another panel would award a 6.0-style Presentation mark and somehow they would be combined as ordinal placements. I didn't think that would work.

If that is what they had in mind, then obviously the change to absolute add-'em-up scores was introduced into the thinking over the subsequent 5 years.

I'm also not sure whether she was referring to discussions within the ISU leadership or just within Canada.

But clearly some potential changes were in the works as of 1998, at which time the main concerns were bloc judging and (especially for Cinquanta) place changing in standings while an event was in process.

 

[Rhino]

I would have thought a 2nd camera angle would work wonders. It's like NFL reviews, one camera angle might be unclear, but another could give a definitive view. Plus in skating you wouldn't have the great pile up of bodies you have in the NFL.

Get one technical specialist to look at one angle, the other at the other (if really pushing the boat out have split screen technology with the cameras synced in time). If they agree brilliant, if not up to the technical controller for the final decision.

Something like that must get closer to the truth, and not be exorbitant in cost (I would have thought you must have motion sensitive cameras these days rather than needing a 2nd operator - you'd always need the first one just in case you hit 'technical difficulties').

 

[VGThuy]

There's been an excessive emphasis on jumps ever since I first watched in '94 (though not regularly until '98). It was jumps that earned Stojko his titles/medals.

Of course there was period of time in the Kwan era when 3-2s were the standard in ladies, for too long. But then the judges could theoretically compare apples to apples. And of course, soon some ladies saw they could gain an advantage with 3-3s, and in a few cases, the quad.

I don't enjoy skating under CoP any less than I did under 6.0. And, I find the jumps exciting.

What does bother me about FS is that so many skaters use the same music. I would really really like to see more creativity in that regard, and more innovation in programs, insofar as that is possible.

To be fair to the Kwan era, it was the first whole generation to be completely post-figures and before it, doubles were the norm as the individual jump in the short program. Oksana Baiul won gold with like three clean triples, Yamaguchi with five that included a hand-down/near fall and a double. By the time Kwan competed at her first Olympics, she lost the gold while doing seven clean triples and people’s expectations had gotten so high that some will argue she gave the gold away even with doing that. Kwan actually has a high track record with completing the triple toe/triple toe. Rome wasn’t built in a day and coaching methods had to find the best ways to best teach jumps while also being able to compete with Kwan with the what we now call the PCS aspects of the sport, which became her strength. During her 10 year tenure, we got more 7 triple programs attempted/completed with female skaters now expected to learn and master triples from toe to Lutz which was unprecedented before and introduced by the likes of Midori, Tonya, and Kristi but I think even Tonya didn’t attempt seven triples, especially by 1993 and 1994. Let’s look at some of Kwan’s competitors: Tara won gold with a 3/3 and a 3/1/3, Irina started attempting two 3/3s, Miki Ando did difficult 3/3s and a quad, Sarah Hughes won gold with two 3/3s, Arakawa played with 3/3/3s in practices and won her Worlds title with two 3/3s. In other words, the ladies had a lot of technical advancement during the Kwan era compared to what we received beforehand. Midori, Kristi (1990-1992), and Tonya’s technical peak (1991) were not the norms whereas it became more normalized in the Kwan era as ladies were adapting to having to do more triples than ever before.

 

[B.Cooper]

Sorry, I just have to laugh at this....the entire review system, much less, the one and only camera to view the programs? Really, considering the technology that is employed by the NFL, MLB, NHL, the majors in tennis...Australian Open, French Open, Wimbledon, US Open (IBM tracker system)....heck, even the NCAA in basketball and football....the ISU can't chase a sponsor like Nikon to develop a new system with real time play back from multiple camera angles....what a difference that would make in the sport!

 

[Aussie Willy]

Having worked with the system extensively on Tech panels, including Data and Video cutting, just some of the issues I come across:

• I am not sure about international events but the person operating the video camera that feeds through to the video replay can be a volunteer which happens at events I am involved in. They usually put up their hand to do it or are seconded. Thus you get variable standards of video, whether they are close to the skater or far away.
• The cutting software while not complex can only do one thing at a time. The person doing the cutting has to click start to start filming the element and stop after the element has concluded. There would either need to be more video cutting operators if you are going to have more cameras. Requires more resources. Again more volunteer roles to fill.
• At the moment the data entry operator is the one who replays the video back. More options of video means more time taken for review.

I am all for seeing improvements. There would just need to be options for international events but then your national and grass roots events.

I know what people want to see, but those who want to slap judges around the head because they think they know better might want to start volunteering so they actually understand what happens on panels and how things get done.

 

[tony]

(I would think) there is most certainly a technology involving digitally saving video that can sync different angles to the same time stamp so that cutting said jump or element would result in all of the different angles having the same cut. That shouldn’t be any problem and shouldn’t result in having to hire more editors/spend more time. Then the panel can have the X amount of angles all presented on the screen and they can choose which one best determines the call.

Having talked to a few people in technical panel-positions (regardless of the level), sometimes the angle truly is terrible and I think ALL of the people in these positions would agree it needs to be more than one angle, and they’ve thought that way since the inception of replay. I’ve been fortunate enough to see an example of what the panels actually get to review, and sometimes it is absolutely ludicrous that they have to make a judgment from such. It makes me understand the questionable calls much more, but I still don’t know how a << can be missed and it not be called even < unless the camera truly was not in focus, not at the skaters feet, or on the very opposite end of the rink. More cameras hopefully solves that issue.

Eventually, I’m sure someone will come up with technology like in tennis (with challenges on in/out) where they can show exactly where/what angle the blade landed automatically without needing any additional equipment.

 

[MacMadame]

I appreciate the difficulties of using whatever is used at ISU events at local club events but in other sports, they have different standards for different levels and different leagues. The NFL has all sorts of fancy equipment and multiple umpires but the colleges and High Schools and certainly the Pop Warner levels do not.

There's no reason that we couldn't improve judging at ISU Championships first and then let it trickle down to other ISU events and then only trickle down further if the various NGBs want it too and it makes sense.

 

[Aussie Willy]

(I would think) there is most certainly a technology involving digitally saving video that can sync different angles to the same time stamp so that cutting said jump or element would result in all of the different angles having the same cut. That shouldn’t be any problem and shouldn’t result in having to hire more editors/spend more time. Then the panel can have the X amount of angles all presented on the screen and they can choose which one best determines the call.

Having talked to a few people in technical panel-positions (regardless of the level), sometimes the angle truly is terrible and I think ALL of the people in these positions would agree it needs to be more than one angle, and they’ve thought that way since the inception of replay. I’ve been fortunate enough to see an example of what the panels actually get to review, and sometimes it is absolutely ludicrous that they have to make a judgment from such. It makes me understand the questionable calls much more, but I still don’t know how a << can be missed and it not be called even < unless the camera truly was not in focus, not at the skaters feet, or on the very opposite end of the rink. More cameras hopefully solves that issue.

Eventually, I’m sure someone will come up with technology like in tennis (with challenges on in/out) where they can show exactly where/what angle the blade landed automatically without needing any additional equipment.

I think you have summed up the problems very well. One of the biggest problems is resourcing and willingness to pay. The world over the sport is volunteer run, which probably also includes those in the top levels. If they are paid it is not at market rates. It means you never get the investment that you should to help the technology develop and grow.

 

[gkelly]

If there are any IT and video production people here who would like to volunteer to study the systems currently in use and develop improvements on your own time, go for it!

 

[Aussie Willy]

I agree. My problem with the system's answer to jump technique issues is that it was always penalty-based when it should be solution-based. There should be more of an effort to help coaches and skaters adopt correct jump techniques. Yet, the work is mostly left to individual coaching teams as it has always been. And it is obvious not all coaches are great at teaching jump technique. Therefore, even more than a decade after IJS start enforcing these rules, URs are still a prevalent issue. Skaters are getting called more than ever but I don't really see a significant improvement in the actual jumping technique among the skaters. Not to mention that the excessive emphasis on jumps have been a detriment to everything else that was great about skating. I rather see more of an focus on accurately scoring skating skills.

I agree. It does amaze me when there are so many coaches out there teaching correct technique that there are still lots of technical issues with jump technique caused by not as competent coaches.

 

[Vagabond]

What exactly would be gained by having jumps tracked by cameras at several different angles?

As it is, both the Technical Panel and the judges are supposed to be giving skaters the benefit of the doubt. If that is what actually happens, then adding camera angles is presumably going to lead to more jumps being downgraded or called as underrotated. Is that what people want? And if so, why? Or do people think that Technical Panels and the judges are not doing what they are supposed to do and that additional cameras will somehow solve the problem?

 

[VGThuy]

Just for me, I want a system that is much more accurate and consistent. When medals, results, and careers of skaters who have worked all their lives and pretty much sacrificed their age and bodies to compete are on the line, then I support anything that results in more fair play. That's the least we can do for the competitors.

I also think having a more accurate and consistent system will help skaters as they won't be as confused as many of them are right now with why their calls can be so inconsistent and thus their results not being consistent. Of course, I'm sort of coming from an ice dance perspective with that statement and I think the cameras need to review turns and steps as well.

 

[manhn]

If they are underrotated, why is it bad for them to be marked as underrotated?

 

[Vagabond]

If they are underrotated, why is it bad for them to be marked as underrotated?

If the underrotation cannot be detected by the naked eye from where the Technical Panel sits, even with video replay, why should the jump to be marked as underrotated?

Direct, intelligible answers, not sophistry, please.

 

[VALuvsMKwan]

Breaking your questions apart to give my perspective on what is now the sport of ice jumping aka gymnastics on ice (don't bother calling it figure skating any more):

What exactly would be gained by having jumps tracked by cameras at several different angles?

As it is, both the Technical Panel and the judges are supposed to be giving skaters the benefit of the doubt. If that is what actually happens, then adding camera angles is presumably going to lead to more jumps being downgraded kr called as underrotated.

If they are indeed that, call them accurately so.

Is that what people want? And if so, why?

If this points-based system is to be accurately used according to the rules, than what is actually done should be accurately judged and scored within the judging and scoring specifications. If technology can aid flawed judging in calculating the overcomplicated metrics, so be it.

Or do people think that Technical Panels and the judges are not doing what they are supposed to do

Often, yes, because humans, even trying their best, are not capable of seeing the most granular aspects of jumps/skating moves due to distance, human response time and possibility of bad angles.

and that additional cameras will somehow solve the problem?

With the benefits of technology available in 2019 and forward, and similar to what has been and is being introduced into other sports, why not?

If we were still talking about figure skating as a holistic art/sport competition, under less technically specific rules and points systems, and using different judging metrics and criteria, I might not be so inclined to argue for such enhanced uses of technology. But we are in the IJS ice jumping era (for singles at least) and the drama/twizzle competition alternatively known as ice dance, so let this increasingly non-artistic sport be technology-driven as far as it can go for judging and for the statistics freaks.

 

[VGThuy]

I think the issue is that we now have a system that demands machine-like accuracy where jumps, steps, and other skills are measured to the exact degree, but we don't have anything in place to give us that in the judging. The judges do the best any human can, but with so much on the line now, we don't have to have tools that can match what the precision that the system is demanding. We need to find ways to bridge that gap because, otherwise, we're doing a great disservice to the skaters. Not to mention some very excellent judges become undeserved targets of ire.

 

[MacMadame]

I see several options available to make things better.

(1) We can say that if imperfections in jumps can't be seen in real-time with the naked eye, then they aren't important enough to call. In that scenario, we could not have any reviews at all or we can have them for cases where the judge was writing on their paper when the issue happened or blinked or had their view blocked. But no slow-mo and it's not clear if we'd need multiple angles from multiple cameras or not.

This would leave some inconsistency but it would also stop the examination of jumps and other elements with a magnifying glass and penalizing for tiny imperfections when they can be found.

(2) We can go the other way and have multiple cameras with many angles, review everything with slo-mo (right now some aspects of an element aren't allowed to be reviewed that way but some are), and penalize everything you can find.

This would increase consistency in judging, but some people might not like how fussy such a system would be

I actually like saying it has to be seen by the naked eye. Many under-rotations are blatant and problems with a jump that obvious should be penalized IMO. But if you have to get out a magnifying glass, examine the jump from multiple angles and maybe even have new technology like sensors on skaters' blades to see it, how important is it?

OTOH, I'm a sucker for consistency and I agree with @VGThuy that skaters deserve that.

 

[VGThuy]

I see several options available to make things better.

(1) We can say that if imperfections in jumps can't be seen in real-time with the naked eye, then they aren't important enough to call. In that scenario, we could not have any reviews at all or we can have them for cases where the judge was writing on their paper when the issue happened or blinked or had their view blocked. But no slow-mo and it's not clear if we'd need multiple angles from multiple cameras or not.

This would leave some inconsistency but it would also stop the examination of jumps and other elements with a magnifying glass and penalizing for tiny imperfections when they can be found.

(2) We can go the other way and have multiple cameras with many angles, review everything with slo-mo (right now some aspects of an element aren't allowed to be reviewed that way but some are), and penalize everything you can find.

This would increase consistency in judging, but some people might not like how fussy such a system would be

I actually like saying it has to be seen by the naked eye. Many under-rotations are blatant and problems with a jump that obvious should be penalized IMO. But if you have to get out a magnifying glass, examine the jump from multiple angles and maybe even have new technology like sensors on skaters' blades to see it, how important is it?

OTOH, I'm a sucker for consistency and I agree with @VGThuy that skaters deserve that.

The skating fan in me is like you in that I think if we can't see it with the naked eye, then don't bother marking it down. If you need THAT much scrutiny and technology to see it, then we're just feeding some obsessive compulsiveness rather than doing anything that actually helps the sport - penalizing just to penalize but does nothing to sell the sport to the masses or are we penalizing anything that people actually care about (outside the ubers who want to use it to attack the rivals of their faves and complain about judging)? The only question I have now is regarding step sequences, especially in ice dance. Should ice dance judging and code of points should be completely overhauled to reflect this new approach? And if so, then maybe we're putting too power back into the judges' hands again which is a whole other topic/debate.

However, if I imagine myself as a skater, or even a skating parent who is putting up the money and investing in this sport, I think I would prefer the second approach because if I'm going to spend all that time, energy, labor, and resources into this sport and devote my life to it, I better not have a system where human error/subjectivity/favortism is so rampant that it could seriously cost rightful berths to various competitions, medals, prize money, and titles.

 

[misskarne]

The problem with the naked eye test is that some people can see it and some can't.

I've trained my eyes to be able to spot it real-time/live. But a lot of fans don't. So we'll still end up with the situation where people whine and complain when the TP calls it.

Additionally, naked-eye keeps that very human element, because different tech panels might be made up of different people whose naked-eye test is different.

 

[Japanfan]

The problem with the naked eye test is that some people can see it and some can't.

And some things can't be seen with the naked eye (like what edge was taken off of).

As I said, if skating is to change, I'd like to see more variation and innovative music choices.

 

[misskarne]

And some things can't be seen with the naked eye (like what edge was taken off of).

That can be seen naked eye. But that's also why we have ! as well as e

 

[Icandance]

Interesting article! Very technical. The author is a photographer at ISU events, so no doubt his technical knowledge regarding cameras, etc., is sound.

I doubt more technology will be the answer here. It would be expensive to implement and, as the author notes, there’s no indication the ISU is making this a project priority. They prefer to spend their time, apparently, developing elaborate proposals for a new “artistic” event—even as the existing sport becomes ever more technical, with results largely determined by quads, yet with the whole technical side of the judging system resting on a shaky foundation, as this article argues. The cognitive dissonance is .

The article just reinforces my feeling that UR calls should have less impact in the scoring. they still haven’t found a way to make the calls fairly and consistently and in a way that’s not controversial. And as much as some will argue the importance of pristine jump technique, in the absence of a fair and correct way to judge URs, I think they carry too much weight in the scoring.

I do find it interesting that the author argues that the human eye can’t effectively determine URs in real time. I know that’s true for me at least—I can only tell in slow motion replay, and even then, it can be quite difficult sometimes, depending on the angle.

Well I have known George for 30 years. He is also a skating judge at local California level and a retired astrophysicist.

 

[Rock2]

Waiting for the follow up piece entitled 'ISU Not up to the Task of Caring about Accurate Calls'

Need to separate the symptoms of issues (technology) from root cause.

 

[manhn]

With all of this "human eye" talk, wouldn't the easier thing to do is change how much "under" the full rotation is?

I can name plenty of performances where the existence of underrotations did not detract from my enjoyment of that performance. Doesn't mean they should ignore those underrotations.

 

[jeffisjeff]

I've trained my eyes to be able to spot it real-time/live. But a lot of fans don't.

Aren't you special!

 

[MacMadame]

She's a judge. I would certainly hope she could judge these things in real-time.

 

[thvu]

Aren't you special!

Are you really trying to shame the learning of a skill?

 

[jeffisjeff]

Are you really trying to shame the learning of a skill?

Not at all. I'm just skeptical of those fans who claim to be experts while simultaneously declaring that other fans aren't.

Also, context matters. In this case, the context is a history of obsessively bashing certain skaters for URs (skaters who just happen to be US rivals of her favorites), even when the actual judges sometimes call the jumps differently.

She wants us to believe she has a highly-trained unbiased eye, unlike so many other fans, but her posting history indicates otherwise.