ISU training videos - whaddya think?

[Mafke]

I'm starting this because there was a lot of discussion about this on another thread and the suggestion was made that some of the side issues discussed would benefit by their own thread.

I'll start : The problem I have with these videos is that there is no.... rigor.

That is, there's a lot of mumbo jumbo using vague terms that have different meanings for different people and then clips of skaters that are supposed to be examples.

My problems:

The features the viewer is supposed to be looking for are very subjective. It all has a very Emperor's New Clothes feel about it. With the example of Harding vs Asada they don't mention (for instance) that Harding seems to spend a lot more time on one foot and her power seems directly generated from the blades (and the timing of her stroking seems better to my untrained eye). Asada spends more time on two feet and is using her whole body to generate speed.

The technology is so primitive! Why don't they use computer enhancements (and super-imposed graphics) to show or highlight things like:
- length of stroking versus height of skater
- one foot vs two foot skating
- balance checks
- movement or lack of same in upper body
- degree of lean (against a vertical axis)
- balance on the blade
- etc

The wording: "Skaters with good flow and glide". Surely 'Skating with good flow and glide" would be better, it seems a trivial point but it's training judges to judge the skater and not the performance. And by using active skaters it's all but saying "See this is who you're supposed to give good marks to".

That'll do for a start.

[Aussie Willy]

I have the full set of the component DVDs, along with other videos and DVDs that were produced in the past, including the ones for elements, synchronised and Ice Dance.

Hate to say this but the audience is not the general public but rather judges, skaters and coaches to provide them with guidance on the components. And when you are watching them you really should be looking at each concept and example on it's own merit. Take the personality out of it. However you may go away with a whole new understanding about a particular skater which will either confirm or change your opinion about what you thought of them in the first place. You sometimes come away saying "Ah I see what they mean by that" and get that little light bulb moment.

From a judging point of view, I have found them very good. They do help with providing a set of tools and key words that you can use to evaluate skaters with and in understanding the concepts of the components. You need to use real life examples and there is no point just showing the good. You need to show examples of what is not considered good technique.

For example on the Presentation DVD they use Totmianin and Marinin to demonstrate the concept of Clarity. I really appreciated that example because it showed that nothing they did was blurred, everything was finished off and the pictures they created were clear. Next time I was judging it was something I was able to apply my set of judging tools, particularly when you really have one skater that highlights that quality in their own skating.

[gkelly]

The first thing I'd say about these videos and about trying to discuss their value is that they need to be watched in context, all segments for a given component one right after the other in order. Unfortunately the presentation on youtube doesn't make it easy to watch them that way.

They also presuppose a prior base knowledge about skating skill. They're not designed to train the untrained eye, but rather to educate trained observers to refine their eyes to notice more detail in a more systematic manner than 6.0 judging required and to apply what they already know to this specific scoring system. In many cases they're more reminders than introductions.

So if you're watching them out of order, with little or no prior experience watching skaters at all levels from beginner to senior live and analyzing differences in power, flow, edge quality, etc., using those terms, these videos alone probably won't help you master those concepts, because they're qualities that are much more apparent live than on video.

Whatever method was chosen to select skaters to illustrate each point, there could be potential problems in terms of effect on the skaters' reputations. Since these videos were not intended for the general public, who are likely to be more focused on the personalities of recognizable skaters than on the concepts, then the effect on the public's opinion of these skaters was not a concern in their selection.

The effect on judges' opinions of skaters who are still competing, whom the judges studying the videos might be in a position to judge someday soon, would be more significant.

The intended audience would include

1) experienced international judges who have formed their own opinions about most of the general concepts and who may have already judged some of these specific skaters (and in some cases seen them develop from lower skill levels to their current competitive level) and formed their own opinions about their strengths and weaknesses

2) less experienced international judges, e.g., new international appointees from smaller skating countries, who may not have judged elite senior international competitions in the past but who are looking forward to doing so soon enough to encounter some of these current still-competing skaters featured

3) aspiring national and international judges who probably won't be judging elite senior internationals while the current stars featured are still competing, but who might encounter one or more star skaters who compete domestically in their own country

4) lower-level judges, especially from larger countries, who may never achieve international appointments, or not until long after the current crop of skaters has retired

Category 2 would be most susceptible to being influenced by the training materials in their opinions of skaters that they may soon be called on to judge against each other.

The technology is so primitive! Why don't they use computer enhancements (and super-imposed graphics) to show or highlight things like:
- length of stroking versus height of skater
- one foot vs two foot skating
- balance checks
- movement or lack of same in upper body
- degree of lean (against a vertical axis)
- balance on the blade
- etc

These graphics would be more useful for a "skating appreciation" course for fans who don't have access to years of watching tests and competitions at all levels. Potential international and even lower-level judges would already know a lot of these basics, so the IJS component training is more a way of shaping that knowledge than imparting it in the first place. And when they're called on to judge these qualities in competition, they'll need to rely on their own naked eyes in real time.

The wording: "Skaters with good flow and glide". Surely 'Skating with good flow and glide" would be better, it seems a trivial point but it's training judges to judge the skater and not the performance.

A valid point. It would be more significant if these videos were the primary source of training for judges who will actually have to judge these specific skaters. Since that's not the case, I do consider it trivial.


Personally, I found the Transitions and Choreography segments the most useful to me in breaking down ways to evaluate the various criteria for those components.

[RFOS]

I don't have time to look at them now but I do recall them using superimposed graphics in the versions I watched to show use of personal & public space, ice coverage, and use of music during a footwork sequence.

[aftershocks]

...
For example on the Presentation DVD they use Totmianin and Marinin to demonstrate the concept of Clarity. I really appreciated that example because it showed that nothing they did was blurred, everything was finished off and the pictures they created were clear...

Ah well, IMO, yes T/M had beautiful line, extension and clarity but precious little connection with each other or the audience. Aside from their earlier West Side Story and Cotton Club performances, they lacked character and excitement. But okay, I guess ISU judges may feel those qualities are a bonus if skaters have them. Hells Bells, no matter if they don't, cause "clarity" and "flow" mean so much more.

[Coco]

As far as the issue of singling out skaters goes, perhaps it would be 'nicest' if they used a skater's prime vs their pre-prime/developmental phase to show the right way and the wrong way, respectively.

And they should avoid using current skaters, as they are basically telling a judge - "mark them off for this" or "reward them for that."

[RFOS]

As far as the issue of singling out skaters goes, perhaps it would be 'nicest' if they used a skater's prime vs their pre-prime/developmental phase to show the right way and the wrong way, respectively.

And they should avoid using current skaters, as they are basically telling a judge - "mark them off for this" or "reward them for that."

I was also thinking another way they could do it if they could find a good example would be to show one program from a skater where there is a major difference in a certain component aspect between different sections of the program. For example, a skater who started out performing and executing very well, but began to make mistakes and give up on the performance halfway through, or a skater who started out with good flow but lost stamina and got more choppy and labored at the end.

[Proustable]

As far as the issue of singling out skaters goes, perhaps it would be 'nicest' if they used a skater's prime vs their pre-prime/developmental phase to show the right way and the wrong way, respectively.

And they should avoid using current skaters, as they are basically telling a judge - "mark them off for this" or "reward them for that."

I think this would be best, though I wonder how much video footage is available for skaters not currently competing at two different stages as clearly as the ISU would like it.

[Aussie Willy]

Ah well, IMO, yes T/M had beautiful line, extension and clarity but precious little connection with each other or the audience. Aside from their earlier West Side Story and Cotton Club performances, they lacked character and excitement. But okay, I guess ISU judges may feel those qualities are a bonus if skaters have them. Hells Bells, no matter if they don't, cause "clarity" and "flow" mean so much more.

But they were only using them as an example of clarity, not the other aspects of Performance. And for that aspect of Performance they are an excellent example. They were not being used as what to look for Performance generally.

People sometimes need to get their headspace out of looking at a whole and rather focussing on a single aspect of a component. It is like listening to a piece of music and concentrating on listening to just one instrument.

It is also looking at even though you might not like a skater generally, you might find something that you can really appreciate about them.

[gkelly]

As far as the issue of singling out skaters goes, perhaps it would be 'nicest' if they used a skater's prime vs their pre-prime/developmental phase to show the right way and the wrong way, respectively.

Yeah, that would be cool, if they could track down contrasting videos of skaters who made noticeable improvement in a specific criterion.

Remember most of the clips are demonstrating specific points rather than overall skill.

And they should avoid using current skaters, as they are basically telling a judge - "mark them off for this" or "reward them for that."

Of course, most of the judges who are at a point in their careers when they would be most influenced by the skater selection in the videos would only have the opportunity to judge those particular skaters at a senior B or national event. Most judges who are already at the point of judging ISU championships and GPs probably already have a lot of experience and their own strong opinions about those skaters.

Suppose they don't redo these videos soon and continue using them for training in 2015 and beyond, when even most of the younger skaters used would have retired from competition. Looked at from that perspective, we can ignore the potential problem about influencing opinions of current competitors and focus on the points being made.

So what do people think about the actual points? Should we choose one component to watch through in order and discuss together?

I was also thinking another way they could do it if they could find a good example would be to show one program from a skater where there is a major difference in a certain component aspect between different sections of the program. For example, a skater who started out performing and executing very well, but began to make mistakes and give up on the performance halfway through, or a skater who started out with good flow but lost stamina and got more choppy and labored at the end.

They did something similar with the side-by-side videos of the same skater having a good day performing "on" the music and being off the music in a less successful performance.

More of that?

BTW, I was wondering recently, for skaters who struggle with stamina, what kind of music would be the wisest choice for the end of a long program when the skater doesn't have enough energy to be either powerful or quick but still wants to keep up with performing the program?

[Jun Y]

BTW, I was wondering recently, for skaters who struggle with stamina, what kind of music would be the wisest choice for the end of a long program when the skater doesn't have enough energy to be either powerful or quick but still wants to keep up with performing the program?

Perhaps the said skaters should work harder on their stamina rather than using choreography to disguise their weakness?

[Jun Y]

First of all, I want to acknowledge that the current component system is better than 6.0 system, at least for a non-judge person like me.

Second, I agree with the poster's particular opinion below:

That is, there's a lot of mumbo jumbo using vague terms that have different meanings for different people and then clips of skaters that are supposed to be examples.

The terms used in the IJS program component rules are problematic. I say this from the point of view of a person who is NOT a judge, but a science writer. The writing quality on these rules is cringe worthy and great variation in interpretation. Reality confirms this great variation. You only need to look at any competition's protocol, across the judges' scores for each component for the same program by the same skater. I think the imprecision of the component rules as written is partially to blame for the jarring inter-judge variation.

Clear, precise, informative, transparent training for figure skating judges should not be intended for judges only. At least, skaters, coaches, and choreographers can benefit as well.

[gkelly]

How could the components be defined in such a way as to create more consistency across judging panels while still preserving the qualitative and in some cases subjective aspects of what they're designed to measure? Any suggestions?


Maybe someday technology will be used not just to train viewers to see some of the qualities Mafke listed in the first post, but to measure them officially -- and also absolute speed and depth of edge.

Eventually, those aspects of skating skills could be scored by machine rather than by human eyesight.

And someone could be assigned to count the number of turns and strokes in each direction throughout the program, not just in step sequences.

But still, I don't see how qualities like "soft knees" can be measured, although they can certainly be appreciated by human eyes. So as long as that's a quality the sport wants to reward -- and I'm sure it always will be -- it will always have to rely on human perception to some degree.

And then you get the problem of turning those analog perceptions, which may vary somewhat from one observer to another, into digital scores. How can that process be made more consistent? Can judges be taught to calibrate their mental 0-10 scales to be more consistent with each other? Would that happen through wording of the rules, or more from experience judging and comparing notes with each other?

It gets even harder when you get to Performance/Execution and Interpretation, because some of those criteria are very much rooted in the observer's emotional response to the performance.

But would we want to eliminate these subjective aspects from the reward system entirely, in the interests of more consistent quantification?

[Vagabond]

Yeah, that would be cool, if they could track down contrasting videos of skaters who made noticeable improvement in a specific criterion.

One of the segments compares Shen & Zhou's Free Skates at Worlds in 2001 and 2004 to demonstrate how that pair increased the variety of their transitions.

[Aussie Willy]

I think whether you consider the way the components are worded is a problem is in the eye of the beholder. I don't have a problem with them and understand the concepts of what they are trying to convey. But that is me. But then the DVDs are designed so that even if judges are not sure they can get a better understanding. But at the end of the day the system is for judges to apply, not necessarily to give the average Joe Blow what they want. It is a judging system.

[gkelly]

Has anyone ever watched the Spectator's Guide to Figure Skating that's for sale on the ISU website? It looks like it would be a useful introduction to fans who want to make the transition from casual to knowledgeable fan.

The rest of the videos on the site seem mostly aimed either at coaches/federations or at casual fans.

I wish the ISU would develop more videos for fans who can now recognize the elements but want to go on to understand the fine points.

E.g., make a video for fans that's just about appreciating the technical side of skating skills between the elements. And one or more videos about the (other) program components, about the technical panel procedures and what they're looking at, maybe one about GOEs unless it's included in one of the above. A separate video about pair skills, and at least one about ice dance (and synchro).

Also make the judge-training videos available for sale, to open up and demystify the knowledge base.

I don't know what the technical restrictions would be on selling online pay-per-view at a reasonable price, which may be cheaper than mailing out DVDs.

The number of potential buyers around the world for each of these videos may number only in the low thousands, and charging too high a price would discourage viewers who would otherwise be interested. So I don't know how much money the ISU would be willing to invest in producing and marketing videos to fans who want to learn more about the sport as sport, to build an educated audience. But I wish they would have the motivation and pricing points to do so.

Include mentions of the materials available, maybe actual ads, in their broadcast contracts with TV networks.

Maybe make small educational video segments for networks to actually include in their broadcasts, and to be played on Jumbotrons during resurfaces at live events.

[Jun Y]

How could the components be defined in such a way as to create more consistency across judging panels while still preserving the qualitative and in some cases subjective aspects of what they're designed to measure? Any suggestions?

My first suggestion would be to hire someone who thinks and writes clearly, instead of relying so heavily on verbal, face-to-face group trainings to convey the meaning of all the rulebook terms. However, we have seen instances where ISU favors ambiguous language in its rules to allow for flexibility in their application of the rules (cough, GP selection rules, cough). So who knows. Perhaps ambiguity can be useful sometimes to some people.

And then you get the problem of turning those analog perceptions, which may vary somewhat from one observer to another, into digital scores. How can that process be made more consistent? Can judges be taught to calibrate their mental 0-10 scales to be more consistent with each other? Would that happen through wording of the rules, or more from experience judging and comparing notes with each other?

Why not all of the above? I was only pointing out that the precision and clarity of the rules can be improved. I doubt there will be skating judges who would come out on a public forum and admit that they have been confused by the rulebook one time or another, but that doesn't mean they don't exist. How many? Hehe, it's certainly not for me to judge.

It gets even harder when you get to Performance/Execution and Interpretation, because some of those criteria are very much rooted in the observer's emotional response to the performance.

But would we want to eliminate these subjective aspects from the reward system entirely, in the interests of more consistent quantification?

My opinion is no, I personally do not want to eliminate the subject aspects of figure skating, but that is also not for me to judge. There are plenty of people within the sport who thinks that figure skating should be more like a sport and less like performance art, more objective and athletic and less subjective or "moving one's heart." Certainly, by quantifying the technical elements in IJS, the intention is largely related to making the sport more objective, even though even the technical aspects are not as objective as the numbers suggest. For example, how can one know for sure that, objectively, a quad toe loop is "equivalent" to, say, two triple lutzes? So I do not absolutely trust in the infallibility of numbers.

However, this issue is not only for judges, with all due respect to them. Skaters may enjoy expressing themselves or performing for the audience regardless of whether they are rewarded by the rules. Audience or "average fans" may love the sport and/or certain skaters for subjective and intangible reasons regardless of the rules. How much do such concerns matter to the sport, even if they may or may not matter to judges? Should skaters be rewarded or penalized for performance aspects? Should figure skating be judged as objectively as possible and let the audience sort out the performance and "artistic" aspects? Or should the performance aspects be factored into the judging? If yes, how much does that count in the context of the athletic/objective aspects? That is the eternal philosophical question of figure skating and there is (so far) no definite agreement.

Beyond the philosophical question, let's assume for a moment that the performance aspects (including expression of music and dance quality, etc.) should and do count in the judging. How should skaters with different styles and intentions be judged fairly and consistently by judges (who are, in some sense, also spectators) who also differ in personal taste and cultural background? One man's passion is another man's frivolity and superficiality. One man's subtlety is another man's obscurity. One man's cultural authenticity is another man's indecipherable symbols. Should the rules attempt to reduce (not eliminate as it is impossible) inter-judge variability by sacrificing some subjectivity? Should the rules minimize subjectivity? Will skaters stop trying to express their personal style and feelings if subjective/performance aspects are no longer rewarded? I don't know the answer to these questions, except that I do not think vague terminology helps.

Also, when it comes to evaluating the performance and music/dance appreciation, just how sophisticated and reliable are skating judges' opinions? Is the experience of having sat through thousands of programs sufficient to hone every judge's ability to appreciate and evaluate the performance aspects? I remember hearing Joe Inman talking about the interpretation and choreography components and encouraging young skaters in the seminar to listen to more music, watch more theater/dance performances, learn to understand performance arts outside of skating, etc. I respect his good intention and agree with him in principle, but still, it was a bit amusing to see the faces of kids 10 and above hearing this and many not getting it. Are all or most skating judges better at appreciating music, dance, and performance quality than, say, fans who are not so knowledgeable about skating techniques but have great appreciation for music, dance, theater? Neither all judges nor all spectators are monolithic blocks who think and understand the same way on the same level.

[Ziggy]

There is a fundamental problem with the PCS guidelines being very loose and not precise at all, meaning anything goes.

I don't mean the kind of precision that the what 1-10 means big table provided. That was totally pointless as it was impossible to use.

ISU should come up with some kind of methodology of judging. A checklist, a way to quickly calculate features, etc. Something like this definitely could be done. Just requires some thinking and effort put into it.

[gkelly]

Are all or most skating judges better at appreciating music, dance, and performance quality than, say, fans who are not so knowledgeable about skating techniques but have great appreciation for music, dance, theater?

Don't know about most, but certainly not all.

The population of skating fans is probably more knowledgeable about music, dance, theatre than the general population at large, and possibly than the population of skating judges.

The trick is to educate the judges (and coaches/choreographers and skaters) more about performing arts, and also to educate the arts-oriented fans more about the technical and athletic demands of skating that make some aesthetic effects easier to achieve on the ice and others much harder than in stage performances.

ISU should come up with some kind of methodology of judging. A checklist, a way to quickly calculate features, etc. Something like this definitely could be done. Just requires some thinking and effort put into it.

Well, ~10 years ago they did that for technical content, and have continued tweaking it every year.

Could program components be systematized into a checklist in a similar way? Any suggestions what such a checklist might look like?

[lulu]

The technology is so primitive! Why don't they use computer enhancements (and super-imposed graphics) to show or highlight things like:
- length of stroking versus height of skater
- one foot vs two foot skating
- balance checks
- movement or lack of same in upper body
- degree of lean (against a vertical axis)
- balance on the blade
- etc

The wording: "Skaters with good flow and glide". Surely 'Skating with good flow and glide" would be better, it seems a trivial point but it's training judges to judge the skater and not the performance. And by using active skaters it's all but saying "See this is who you're supposed to give good marks to".

That'll do for a start.

All of these are excellent suggestions, particularly using computer enhancements to better illustrate the points. I'm watching the videos via youtube, and overall, I am finding them helpful, especially the interpretation and presentation clips.