judging system

[dinakt]

Well, if any of them also want to learn to judge the technical side of figure skating, don't wait for the ISU to change the system -- start trial judging!

If there is a separate PCS-only panel, should it be for top-flight senior internationals only? All senior and junior internationals, and national championships too? All IJS competitions anywhere at any level, unless a local club can't afford it and is allowed to compromise by having a handful of skating judges judge everything? I.e., how many "artistic judges" would be willing to spend a weekend in a rink maybe once a month, judging the artistic qualities of performances by kids who are still struggling with their skating skills and double jumps?

I don't know how many I know that if there were such a possibility, I personally would eat it up- though probably a bit later in life, when life is (hopefully) a little calmer. Don't know about others.
As for lower- level... You ask great questions, something to ponder, even theoretically. I don't think it can be only top- level seniors who have a divided panel, that's just unfair. Perhaps starting with Regionals and Juniors? And everything before- have "bullet points" that all judges are qualified to give? Also, I don't want to be insulting to judges. There's Inman, for example- it is my understanding he has training in arts. I am sure there are LOTS of current judges who can easily do both. The trick is- with 6.0, "artistic second mark" system it was more of a general impression/ perfection of execution. I might like one skater, the judges another. But with the current system, things are spelled out in professionally artistic terms, so not following them becomes blatantly obvious to members of said professions.
For example:
IN- Interpretation
Definition: The personal and creative translation of the music to movement on ice.
To reward the skater who through movement creates a personal and creative translation of the music.
As the tempo binds all notes in time, the ability to use the tempos and rhythms of the music in a variety
of ways, along with the subtle use of finesse to reflect the nuances of all the fundamentals of music:
melody, rhythm, harmony, color, texture, and form creates a mastery of interpretation.

Can one - especially without musical training- really judge "the subtle use of finesse to reflect the nuances of all ( ALL!!! LOL) fundamentals of music", especially while judging everything else? Absolutely not. And as a result we have Kozuka and Suzuki undermarked on IN, Lipnitskaya overmarked on IN. Etc. If it had not been spelled out so very specifically in musical terms, I would not be boring everybody with constant wailing. But once it is spelled out, ISU has to live up to it one way or another, or judging PCS feels fake.

[Aussie Willy]

I don't know how many There's Inman, for example- it is my understanding he has training in arts. I am sure there are LOTS of current judges who can easily do both.

Joe Inman I think has super dooper qualifications in music. When I first started judging I attended an IJS seminar with Joe which definately has made an impact on my judging of components, particularly the ones most related to music. But maybe it is because I already understand music, it definately helped me apply that understanding with what to look for in the components.

[kwanfan1818]

Can one - especially without musical training- really judge "the subtle use of finesse to reflect the nuances of all ( ALL!!! LOL) fundamentals of music", especially while judging everything else? Absolutely not. And as a result we have Kozuka and Suzuki undermarked on IN, Lipnitskaya overmarked on IN.

The lack of musical expertise and understanding leads to judges asking the skaters to educate them. The "in your face" skaters do that and often to the two CD's worth of music with which the judges are most familiar.

It was the ones who were doing GOE only who were bored.

They were doing all of the elements at the time, too. How boring it must be to use one's technical expertise to judge multiple phases and transitions of each element properly, and especially the steps, which take up chunks of a program, especially since they must mark on demand, when the screen tells them to.

How would this work below the world-class level?

If there was a panel of five, there unfortunate compromise would be two jump judges, one spin/FW judge, and two PCS judges. It's possible to have a single PCS judge. At lower levels, the

Also, were there PCS judges at the local level, it should be a lot easier to find volunteers locally for the PCS judging, since there is dance and music expertise everywhere at a greater ratio than people who are expert enough to judge edges and the more technical aspects, like under-rotations.

I'm not sure how the technical panels work at the local level, but there would have to be an extra PCS levels panel person, and this would have to be someone who is a figure skating expert, since only a figure skating expert would know the relative difficulty of the edges.

Even a panel of 7 or 9 would strip away most of the anonymity, were it split 2 jumps, 2 spins/FW, 3 PCS, or 3/3/3.

The addition of the technical panels has already added considerably to the costs of running a competition, at any level.

Considering that the skaters find the system most valuable, the addition of the technical panels is probably the best money they could spend. Even if they increasing the tech panel by one or to have a leveling panel, they should be able to get PCS judges more easily and cheaply, expense-wise.

I also don't think at the lowest levels that the system needs to be identical to junior and senior championships. It would be a waste of time and not even all that valuable to the skaters at the lowest levels to have their difficulty calculated. If I'm wrong, chances are the difference between the levels among eight-year-olds is probably small enough to allow the technical panel to determine one. It could work as a jury, with a discussion at the end.

With 6 judges, if you split the judging panel into three sets of task you could have only 2 judges in each section. But if you require four sections, or if there just aren't enough officials available to the local club, then you'd end up with only one judge making all the decisions on his/her own for the whole subset of skills assigned, OR you'd have some or all of the judges covering more than one subset anyway.

I have three sections: for singles the jumps, the spins & FW (including the CH sequence), and the PCS. For dance, the breakdown for the FD at least would be FW/twizzles, lifts/spins, PCS. For Pairs, probably throws/jumps/FW, lifts/spins, and PCS.

I don't have a problem with one judge making the decision on a subset, since that judge's name would be on the results.

Judges at club competitions or even national competitions are likely to be less experienced than international judges, but they would have more authority than the more experienced judges at the big competitions that can afford larger panels. So by the time the judges got to the national and international level, even if we exclude all of the currently trained judges who are used to doing all GOEs and all PCS, they would likely have years of experience judging all or most of those skills at once. Then when they get to the higher judging levels, you give them less to do.

Yes, but that's true now, for example when panels remove parts of the technical panel. That's true in just about all areas of sports officiating, where fewer people do more at a lower level, and then are, for example, the foul line official. A first grade teacher teaches many subjects at a low level hours and hours each day; a university professor rarely does. At the top level, they'd have a sub-specialty of expertise, and they'd be held to a very high standard.

If we want to be honest about it, we should first analyze exactly what about the current status quo is the system being designed wrong (good reasons to redesign), what's a case of most judges consistently implementing it wrong (need better training across the board), and what are isolated cases of individuals making occasional mistakes (need better training for those who make such mistakes frequently, and maybe failsafe devices to catch the mistakes before certifying them into the official scores).

I don't believe in changing systems because of outliers; however, I think there's something fundamentally wrong with the system in place. There's little incentive to judge to the written code -- at least the technical team has the possibility of two of three pulling someone back in line, but there's no discussion among the judges -- but there's disincentive: the wrath of the Federation for not toeing the line and the threat of having assignments pulled, much greater disincentive than being called up by the ISU and tossed, and the intrinsic problem of being judged by the corridor, not the written criteria. I don't buy that it's a matter of training. It's contrary to human nature to assume that judges that are raised through politics as well as expertise are going to sacrifice a lifetime of volunteering at the lower levels to be martyrs when they get to the top.

There's little there at the top levels, when the prize is in hand, to make judges accountable. When Joe Inman has to explain that Evgeny Plushenko shouldn't be getting 8+'s in TR, and then he is vilified -- the 6.X in the SP was the difference between the Olympic silver and gold -- there's something fundamentally wrong with judging.

They max out at +3. And the guidelines for positive GOE say

So if they found 4 positive bullet points, they would generally be starting from +2, not +3. If they found 3, they'd start from either +1 or +2 -- most likely +1.

The rules don't say so, except in allowing discretion as mentioned in the bit I quoted above, but I would imagine that some judges might consider that if the description of a bullet point says "good" then if they think that aspect of the element was good they'll award one bullet point, but if they think that aspect of the element was OMG! phenomenal, they might count it twice.

Thank you for the explanation. I suspect enough of them are thinking in terms of double-counting to distinguish between high and over-the-boards high, for example.

http://www.fsuniverse.net/forum/show...ystem/page3How many arts experts would be willing to give up several weeks of their income-generating work, even if it's mostly a long weekend here and there, to volunteer to judge figure skating?[/QUOTE]
I think that would be a lot easier than finding accountants and engineers with technical figure skating expertise to arrange with their employers to take the time off, especially when many are free-lancers or private teachers and who, if given ample notice, could arrange their schedule around competitions.

Also, since there's an age limit to becoming an international judge, there are people with expertise in music and dance who would be judging exclusively at the lower levels and at local levels, because they are too old to be considered for the top-end competitions.

[gkelly]

They were doing all of the elements at the time, too. How boring it must be to use one's technical expertise to judge multiple phases and transitions of each element properly, and especially the steps, which take up chunks of a program, especially since they must mark on demand, when the screen tells them to.

I'm not sure what your point is here, how much of this sarcastic and how much straightforward.

In a senior program -- and even more so in 1990s/early 2000s programs than in IJS programs -- once an element is over there is often 10 or 20 seconds before the next element starts when what is going on is either "choreography" or getting from one end of the ice to the other -- maybe in an interesting way that enhances the artistic value of the program, and maybe nothing but basic crossovers, or somewhere in between. If you were judging program components, there would be several things to think about exactly what the skater was doing during that time. If your only assignment is to give grades of execution on all the elements, you get to turn your brain off during that time, until it's clear that the skater is getting ready to execute the next element.

For judges who were used to judging all the in-betweens holistically along with the elements under 6.0 and judging all the program components under IJS, not having anything to judge during those parts of the program was boring.

If there was a panel of five, there unfortunate compromise would be two jump judges, one spin/FW judge, and two PCS judges. It's possible to have a single PCS judge.

Since PCS are more subjective than jumps, I'd rather allocate more judges there .

Also, were there PCS judges at the local level, it should be a lot easier to find volunteers locally for the PCS judging, since there is dance and music expertise everywhere at a greater ratio than people who are expert enough to judge edges and the more technical aspects, like under-rotations.

Yes, most towns that have skating clubs also have musicians and dancers and actors and painters, even if they're only amateurs at those arts as well. The trick is getting them well trained enough in figure skating to understand what they're watching if it happens to be a location that holds one skating competition a year.

It would be great if a cadre of arts experts who are also figure skating fans want to travel around at their own expense to develop expertise in judging the artistic side of skating (and help technically oriented skating judges better appreciate the artistic side of skating while they're at it). But practically it just won't work for all locations. If the Skating Club of Outer Exurbia hosts a yearly club competition that attracts maybe two senior and three junior ladies, who should they invite to serve as artistic judges? Are they better off with the local piano teacher and the high school drama coach, who both claim to be big figure skating fans because they watch it on TV several times a year? Or paying travel and hotel expenses for a professional musician or art professor from 200 miles away who art-judges skating several times a year because there are more or bigger clubs close to where they live? Or a former skater that city or further who was always interested in the artistic side of the sport as a competitor and who is still involved in the arts and/or with skating in a capacity that would not involve conflicts of interest?

I'm not sure how the technical panels work at the local level, but there would have to be an extra PCS levels panel person, and this would have to be someone who is a figure skating expert, since only a figure skating expert would know the relative difficulty of the edges.

What are PCS levels? Are you going to invent the concept and explain it to us?

Even a panel of 7 or 9 would strip away most of the anonymity, were it split 2 jumps, 2 spins/FW, 3 PCS, or 3/3/3.

Anonymity is a red herring. Most of us here would agree that we should just scrap it entirely. Those at the ISU who thinks it serves a valuable purpose would make sure that any new system would continue to protect those they think need protecting.

Considering that the skaters find the system most valuable, the addition of the technical panels is probably the best money they could spend.

Agreed.

I also don't think at the lowest levels that the system needs to be identical to junior and senior championships. It would be a waste of time and not even all that valuable to the skaters at the lowest levels to have their difficulty calculated. If I'm wrong, chances are the difference between the levels among eight-year-olds is probably small enough to allow the technical panel to determine one. It could work as a jury, with a discussion at the end.

In the US, IJS is used starting at juvenile level (13 and under -- most are ~10-12, most are attempting most double jumps). The program requirements and restrictions are different at each level, and the PCS factors, but the rules for level features and the program component descriptions are the same regardless of level.

Other countries use IJS at all levels down to beginners often with modifications of the rules such as limits feature/levels and fewer program components. I'm not familiar with details. Maybe Aussie Willy or others can shed some light.

At small club competitions there may be 0 or 1 skater with realistic national and international aspirations, and the standout talented skater might currently be at one of the middle levels, not yet senior or junior.

At larger club competitions and the first level of official qualifying competitions, you might get most of the potential national and international competitors from that part of the country (in large skating countries) the whole country (in smaller ones). Club competitions that can offer scoring that most resembles what the skaters will experience at the qualifying events have more value for the serious competitors. Some tiny local competitions in the US will still operate under 6.0 only even at the higher levels, but if they want to attract more upper level entries they need to offer IJS with at least some experienced officials on the panels.

If there were a separate judging appointment for PCS only, then they would want experienced PCS judges on the panels, whether those individuals started from a skating background or another arts background.

I don't have a problem with one judge making the decision on a subset, since that judge's name would be on the results.

Let's put all their names on the results and then figure out the best breakdown according to where honest disagreement is most likely to occur.

Yes, but that's true now, for example when panels remove parts of the technical panel.

To some extent, yes, when a competition has to make do without an ATS, which is not ideal. Doing without the video replay operator and the data operator doesn't happen so much because the people aren't available as that the video replay equipment isn't available so all the decisions are made based on what the panel saw in real time. Again, not as valuable to the skaters as having the replay, but more focused than the 6.0 approach.

I don't believe in changing systems because of outliers; however, I think there's something fundamentally wrong with the system in place. There's little incentive to judge to the written code -- at least the technical team has the possibility of two of three pulling someone back in line, but there's no discussion among the judges -- but there's disincentive: the wrath of the Federation for not toeing the line and the threat of having assignments pulled, much greater disincentive than being called up by the ISU and tossed, and the intrinsic problem of being judged by the corridor, not the written criteria.

I guess my problem with people trying to address the problem of bad or dishonest judging first -- both the ISU and its critics -- is that it ignores a more fundamental question.

How can we even have an idea of what bad judging is until we've established what good judging is?

So I'd rather ask, in an ideal world where everyone is as honest as possible but still subject to the limits of human perception, knowledge, and opinions, and given the complexity of the object of evaluation that is a figure skating program, what would be the best system for arriving at a consensus about who skated best and communicating the reasons for the decisions to the skaters (and to other interested observers)?

I think the first problem to solve is how to reconcile different valid evaluations from different expert observers. How can we measure the objective aspects as accurately as possible, and how can we balance the conflicting valid opinions about the more subjective aspects? How should we break up the tasks to make sure everyone is as expert as possible -- the higher the competitive level the higher the level of expertise?

Then, even if we come up with the best possible system in theory, there will be problems with the execution just because everyone does not start out with perfect knowledge or perfect understanding of the theory.

So how do we develop that expertise; how do we solve the problem of lack of knowledge and lack of training for less experienced officials, especially those who have less skating experience of their own (or less arts experience for those criteria of those program components where it is relevant)?

And then let's look at the problem of politics. Would our theoretical best system only work with perfectly honest judges and fall apart as soon as any of them have any incentive to score skaters on any basis other than honest evaluation according to the theoretically optimal rules? Or can we design the system to allow for optimal honest judging while also safeguarding against dishonesty?

Also, since there's an age limit to becoming an international judge, there are people with expertise in music and dance who would be judging exclusively at the lower levels and at local levels, because they are too old to be considered for the top-end competitions.

I've heard some fans say they know they'd be better at judging than the real judges, but they don't enjoy low-level skating and are not willing to put in the years of judging tests and low-level competitions.

I would hope some such fans, especially arts experts, love skating enough to be satisfied by the process even when the level of skating is much lower than what they see on TV.

It is probably usually easier to judge at the higher levels because it's a lot clearer what the skaters are actually doing technically. Less time needs to be spent sorting through errors and unclear intentions. But because the stakes are higher at the higher levels, that's where we want the most experienced, most knowledgeable, judges.

And because the stakes are higher, that's also where politics are more likely to come into play.

So safeguards against political judging, whatever might be effective, would be more important at the higher levels. But the basic process can't be fundamentally different at the highest levels than at middle levels or the junior and senior qualifying competitions that lead to those international elite events.

For one thing, if we want the most experienced officials at the elite events, then lower tier events along the way are exactly where they will develop that experience.

And skaters who qualify for the next competition should be those who have (or were able to demonstrate when it counted) the best skills in the areas that will be judged at the next level.

That, BTW, was my problem with the ISU invitational interpretive competitions, including the recent Medal Winner's event, and to a lesser extent with non-ISU pro competitions. Skaters "qualified" to be invited to these events based on their athletic accomplishments, with jumps often the determining factor, but then they're expected to compete in an artistically oriented event. Some of the invitees rose to the challenge of developing top-notch artistry after turning pro or just generally passing their athletic peak; some had already gotten most of the way there while still competing at the highest level. But skaters who had won medals and name recognition on the basis of jumps they could no longer execute (or school figures) could be invited, while skaters who had always been athletically weaker but artistically superior were forgotten and couldn't earn their way into these artistically oriented events. (Sorry for the digression.)

[Iceman]

And someone wanted to put this thread in the Trash Can. It is the most fascinating thread on any skating forum that I have read in a long, long time. That being said, my head is spinning. lol I need an Excedrin.

[PDilemma]

And someone wanted to put this thread in the Trash Can. It is the most fascinating thread on any skating forum that I have read in a long, long time. That being said, my head is spinning. lol

Agreed. This is a great thread. I think the first rational discussion about COP and its weaknesses. No one screaming that if you don't think it is perfect you just can't count. No one yelling that 6.0 is not coming back so people should get over it. Just a rational discussion of what is actually imperfect and what could be done to improve the system.

I wish I could rep you all ten times.

[kwanfan1818]

In a senior program -- and even more so in 1990s/early 2000s programs than in IJS programs -- once an element is over there is often 10 or 20 seconds before the next element starts when what is going on is either "choreography" or getting from one end of the ice to the other -- maybe in an interesting way that enhances the artistic value of the program, and maybe nothing but basic crossovers, or somewhere in between. If you were judging program components, there would be several things to think about exactly what the skater was doing during that time. If your only assignment is to give grades of execution on all the elements, you get to turn your brain off during that time, until it's clear that the skater is getting ready to execute the next element.

It's the judges' responsibility to watch everything between the elements, because the entrances and exits are part of the GOE, not just transitions. If the judges were focused on the elements, they wouldn't be distracted by having to assess component scores when keeping an eye out for entrances and exits. For skaters like Chan, the Farris SP, Takahashi, Kozuka, Abbott, and others as well, it can be a full-time job figuring out where one element begins and another ends, after the big tricks at the beginning of the program are over.

For judges who were used to judging all the in-betweens holistically along with the elements under 6.0 and judging all the program components under IJS, not having anything to judge during those parts of the program was boring.

Yes, most towns that have skating clubs also have musicians and dancers and actors and painters, even if they're only amateurs at those arts as well. The trick is getting them well trained enough in figure skating to understand what they're watching if it happens to be a location that holds one skating competition a year.

I don't know about painters or people who are not versed in music or movement, but the gap between being trained in figure skating enough to judge interpretation, form, body movement, and musicality isn't very large, IMO. It's a much bigger gap to train people with figure skating experience to understand these things at the same level they understand technique and to get beyond what they like.

If the Skating Club of Outer Exurbia hosts a yearly club competition that attracts maybe two senior and three junior ladies, who should they invite to serve as artistic judges? Are they better off with the local piano teacher and the high school drama coach, who both claim to be big figure skating fans because they watch it on TV several times a year? Or paying travel and hotel expenses for a professional musician or art professor from 200 miles away who art-judges skating several times a year because there are more or bigger clubs close to where they live? Or a former skater that city or further who was always interested in the artistic side of the sport as a competitor and who is still involved in the arts and/or with skating in a capacity that would not involve conflicts of interest?

Conflicts of interest more than your uncle judging you at the Olympics? Or your Federation head's wife?

I think I'd start with the people who are interested and go from there. I wouldn't focus on actors, though, unless they had solid music and dance training, because I don't think acting is particularly relevant. That local piano teacher might have studied at Julliaird. There are semi-professional-level musicians and dancers in far-away places, many more than people who are expert in figure skating technique.

What are PCS levels? Are you going to invent the concept and explain it to us?

.
Yes: a trained specialist or set of specialists -- ideally would be three, but that would be a pipe dream -- with training based on ISU-level documentation, seminars, and videos, who would set a level for the difficulty, intricacy, variety of SS, CH, and TR. For Pairs and Dance, there could be a separate score for the balance of work between them. The PCS judges would give GOEs for intepretation, musicality, form, and, for pairs and dance, unison/balance.
There wouldn't be a 1:1 correlation between GOE and levels/rotations/both as there are for technical elements. PCS would be additive, like gymnastics for a difficulty score (although this would include variety, intricacy) + GOE.

I don't expect a separate scoring system for lower-level competitors that now use IJS. I think local organizers could make adjustments to the tech and judging panels, as they do now.

(I just realized I'm late for class. I'll respond to the rest of your post later.)

[gkelly]

It's the judges' responsibility to watch everything between the elements, because the entrances and exits are part of the GOE, not just transitions. If the judges were focused on the elements, they wouldn't be distracted by having to assess component scores when keeping an eye out for entrances and exits. For skaters like Chan, the Farris SP, Takahashi, Kozuka, Abbott, and others as well, it can be a full-time job figuring out where one element begins and another ends, after the big tricks at the beginning of the program are over.

I don't think it's that hard, for experienced judges or other skating experts -- especially if they have literally nothing else to do. If they're also judging skating skills and transitions, they'll stay more engaged without being overburdened.

I don't know about painters or people who are not versed in music or movement, but the gap between being trained in figure skating enough to judge interpretation, form, body movement, and musicality isn't very large, IMO. It's a much bigger gap to train people with figure skating experience to understand these things at the same level they understand technique and to get beyond what they like.

One question I have is whether the process of recruiting and training these officials should start from the top down (ISU, looking to staff panels at the championships and Grand Prix and not caring much about what individual federations do domestically) or from the bottom up (clubs offer alternative events or official recruitment at the local level, federations codify the grassroots practices into national programs, and eventually the ISU adjusts its practices to take advantage of the successful innovations).

For example, the US now has national programs for artistic skating through National Showcase and Theatre on Ice.

There is now a special judging appointment for TOI. I'm not sure what the requirements are, or what the structures are in other countries that participate. I do know there is more specific emphasis on choreographic principles.

So maybe this would be a good place to start, and to recruit interested artists who may have no skating background into the artistic judging ranks.

What if the ISU said "We're looking at the option of restructuring the officials' panels beginning with the 2019 season, which will include a new international and ISU "artistic judging" appointment. We plan to try out shadow panels at a few senior B events and to sponsor some test events starting in 2014-15. We will offer six or eight seminars around the world about artistic judging starting next year, and we want federations to disseminate that knowledge at home and also develop their own best practices and share information with other federations and with the new ISU artistic components commission. We encourage recruitment and training of nonskaters into this judging track. Starting in 2016 or 2017 we plan to offer appointment seminars -- federations will nominate individuals who have shown aptitude for artistic judging to attend one of these seminars, at the end of which they will be given the opportunity to pass practical and written tests and receive an International artistic judging appointment."

Then it would be up to the federations to train some interested arts experts to know enough about skating that they could be nominated for the international appointment seminars -- and have other trained artistic judges to serve on artistic judging panels at domestic events if and when the judging system changes to need them.

Conflicts of interest more than your uncle judging you at the Olympics? Or your Federation head's wife?

I was thinking of the same kinds of conflicts of interest that we get with technical specialists whose day job is coaching -- often they're not useful to competition organizers in their local area because they have coached at least one skater in too many of the events. Could be even more of a problem with someone who's working full-time as a skating choreographer, although unless they're so in demand that skaters travel to them from all over, they probably could officiate at competitions in other geographic areas.

Non-skating arts experts who want to become artistic judges would each start with different areas of expertise and different levels of awareness about skating. So we would want to get them all on the same page as much as possible when it comes to what figure skating wants to reward artistically. At least they'll all have a head start on understanding the artistic criteria over judges with no prior artistic interest or training.

The best potential artistic judges would probably be former skaters (whether elite or mid level) who also had professional performing arts experience.

At the beginning of training toward this appointment, there might need to be separate lessons about skating basics for nonskaters and about aesthetic basics for nonartists. But by the time they're ready to apply for an international appointment, they should have similar knowledge bases, in part from sharing their individual areas of expertise with each other at training seminars and in the officials' room when judging competitions together.

[Aussie Willy]

So maybe this would be a good place to start, and to recruit interested artists who may have no skating background into the artistic judging ranks.

How would you get these types of people involved? What incentives would you give them? And what would be their motivation for doing this?

Sports like figure skating operate because of generational factors. Not so much the family kind but rather those who are involved as skaters or coaches then move into judging and technical roles. It is a group who have an investment in being involved. That could be because they are totally passionate about the sport. Or they are a coach who enjoys the role and challenges of being involved in the technical side (and there are coaches who do it because they get a better understand of the system to help their own students). Or also recognise the need to have people in those roles.

I question this because unless you have been involved in the sport, I don't see a person from outside a skating background having that same drive to be involved.

[gkelly]

I don't see people at the top of their fields getting involved.

For world-class international competition, it might be good marketing for skating to bring in people who are celebrities in their own fields, but it would probably lead to bad judging because they wouldn't have the time to invest in developing common standards for applying performing arts knowledge to figure skating. They're busy enough with their own careers.

What I could see happening is skating fans who also have professional or serious amateur careers or college degrees in the arts deciding that they would like to become actively involved with the sport they love to watch. If they're well into adulthood and have never skated, except maybe if they were dancers and have a lot of time to practice now, they're not going to get to a high level as skaters themselves. They could become judges if they're willing to put in the time to learn the standards and learn to recognize the techniques.

And if there were a division of labor on the judging panels such that there were separate judging appointments for technical judges and artistic judges, then I could see people who started judging with little skating experience of their own but lots of arts experience getting further up the ladder of artistic judging appointments than technical ones.

But I don't think there's a place in a meaningful judging system for celebrity or dilettante judges to drop into skating competitions to judge once a year bringing to the table only their outside arts knowledge and whatever they've learned about skating from TV.

[skateboy]

I love this system in the sp. In the LP/FD however, it fails me as a viewer. I wish they would judge in broader categories. Why not just have a box for jumps, spins, footwork, and moves in the field coupled with PCS as it is. It would be more subjective than it is now, but a lot less subjective than what we had in 6.0 (a single mark for all technical merit). Spin limits and Zayak guidelines could still apply. Would we have results we disagree with? Yes, but we have them now, and we've had them before. At least this way, the skaters would have more room to create programs that highlight their strengths, and a lot more freedom to choreograph creatively to the music.

This, this THIS! Why has no one commented on this as an option?

[kwanfan1818]

I don't think it's that hard, for experienced judges or other skating experts -- especially if they have literally nothing else to do. If they're also judging skating skills and transitions, they'll stay more engaged without being overburdened.

To take another sport, in the Olympics, diving judges watch five or six rounds of 32 divers in the first preliminaries, five or six rounds of 18 divers in the second preliminaries, and five or six rounds of 12 divers in the finals. They do at least two of those phases on the same day. Each dive lasts a few seconds, with wait times between dives. After the first prelims, CBC interviewed Tom Daley, who was a bit rusty and qualified 15th after having a bad 5th dive. He said it was hard because the competition lasted hours, and keeping fresh and focused waiting was so difficult. The judges had to sit through it all. They still show up, and they have plenty of down time.

In synchronized diving, there is a split panel, and with three judges judging each individual diver -- only the middle score counts -- and five synchro judges. From what I understand, synchro judges are also "regular" judges in other competitions. I don't know that they have a problem with "only" judging synchronicity.

If the current judges showed that they could judge SS and TR at the highest levels, then perhaps those two elements could be distributed to technical judges. However, I think judging for SS and TR has been a travesty. Take the example of the Olympics Men's competition. I'd seen Plushenko regularly at championships through 2005 in Moscow, when he withdrew after the SP. That SP locked in my respect for him, because he was clearly in a lot of pain, yet he finished the final footwork (and probably a spin) with strength. The skater I saw in Vancouver was a shadow of his former self skating wise. He was in the bottom 1/3 in terms of speed and power, and was quite sluggish. If speed and power were all that are important for SS, he still would not have deserved a tie for 3rd highest SS in the SP and 2nd highest in the FS for half of the bullet points (or parts of bullet points), like balance and rhythmic knee action, low and effortless glide. However, I don't see where he demonstrated precision of foot placement, cleanness and sureness of deep edges, steps and turns outside his FW passes, and certainly not mastery of multi directional for a program that was single-directional for most of it, or mastery of one foot skating much more than most of the men in the competition. Certainly not 2010 Plushenko. The reason for justifying Lambiel's highest (SP) and 4th highest (FS) SS, when he was glacial in parts of his program, was that he did show mastery of one-footed, multi-directional skating with sureness of edges and turns, if he was weak on the power and glide for much of his program. He showed variety, intricacy, and complexity in his TR as well as quality. Weir, who's skating was brittle in the SP, received a higher SS score there (7.9) than in the FS (7.7), where he got his mojo back and was the Weir pre-Zmievskaya, and his power, flow, and knee-action were significantly better.

Then there are the infamous transitions. In the SP, the judges, presumably pressured by Joe Inman's memo, gave Plushenko a 6.8, which was already high for content. By the FS, that was 7.25, .7, or 8% lower than Chan's. Really? When empty Morozovian programs like V/T's are given TR scores of mid-high 8's, is this credible, when the criteria demand variety, intricacy, complexity, and quality? I'm not a big fan of Duhamel/Radford, but the TR in their programs, particularly their SP, are very impressive, and this isn't reflected in their scores, absolute or relative.

These are judges with years of training and seminars. They either can't judge by all of the criteria, or they won't, IMO.

One question I have is whether the process of recruiting and training these officials should start from the top down (ISU, looking to staff panels at the championships and Grand Prix and not caring much about what individual federations do domestically) or from the bottom up (clubs offer alternative events or official recruitment at the local level, federations codify the grassroots practices into national programs, and eventually the ISU adjusts its practices to take advantage of the successful innovations).

I don't know what the logistics would be of doing it from either end, but I suspect it would stick better if it were bottom up, much like some of the more progressive political initiatives have been driven from younger people and their changing social attitudes. If people started judging in a new system, they wouldn't be "losing" anything later on. The problem with this is that skaters would be used to being judged correctly at the lower levels, and then slapped in the head when they got to competitions with more traditional judging panels who cluster scores for the most part.

What if the ISU said "We're looking at the option of restructuring the officials' panels beginning with the 2019 season, which will include a new international and ISU "artistic judging" appointment. We plan to try out shadow panels at a few senior B events and to sponsor some test events starting in 2014-15. We will offer six or eight seminars around the world about artistic judging starting next year, and we want federations to disseminate that knowledge at home and also develop their own best practices and share information with other federations and with the new ISU artistic components commission. We encourage recruitment and training of nonskaters into this judging track. Starting in 2016 or 2017 we plan to offer appointment seminars -- federations will nominate individuals who have shown aptitude for artistic judging to attend one of these seminars, at the end of which they will be given the opportunity to pass practical and written tests and receive an International artistic judging appointment."

That would give Federations ample time to 1. ignore it like they did IJS, expecting to overturn it before the 2006 Olympics or 2. deliberately undermine it. I don't see what incentives the Federations would have to pick people based on ability rather their ability to follow directions.

I was thinking of the same kinds of conflicts of interest that we get with technical specialists whose day job is coaching -- often they're not useful to competition organizers in their local area because they have coached at least one skater in too many of the events. Could be even more of a problem with someone who's working full-time as a skating choreographer, although unless they're so in demand that skaters travel to them from all over, they probably could officiate at competitions in other geographic areas.

I don't think it's necessary to pick choreographers or anyone involved in skating.

Non-skating arts experts who want to become artistic judges would each start with different areas of expertise and different levels of awareness about skating. So we would want to get them all on the same page as much as possible when it comes to what figure skating wants to reward artistically. At least they'll all have a head start on understanding the artistic criteria over judges with no prior artistic interest or training.

Many non-skating professionals or semi-professionals in music and dance have experience working in evaluations, where there are systems like Royal Conservatory, or RAD (for dance), and on juries, where they are told what the goal of the given competition is, and where the weights are, or in the audition process, often where some very specific characteristics are sought. They wouldn't be starting from scratch.

The best potential artistic judges would probably be former skaters (whether elite or mid level) who also had professional performing arts experience.

Maybe, if they weren't tied to everything they knew and experienced before and imposed that on the competition.

At the beginning of training toward this appointment, there might need to be separate lessons about skating basics for nonskaters and about aesthetic basics for nonartists. But by the time they're ready to apply for an international appointment, they should have similar knowledge bases, in part from sharing their individual areas of expertise with each other at training seminars and in the officials' room when judging competitions together.

Do all of the judges have the same base expertise now? Members of the Technical Panels have more common expertise, based on the requirements for the job.

How would you get these types of people involved? What incentives would you give them? And what would be their motivation for doing this?

For one, advertise and outreach. What incentives would they need? Presumably they'd do it because they were interested in figure skating as an expression of music and/or movement.

I question this because unless you have been involved in the sport, I don't see a person from outside a skating background having that same drive to be involved.

I think insiders believe that, because that's their experience, and there hasn't been an opportunity for it to be otherwise. How many just on this board pay money and stay up in the middle of the night to watch relatively obscure skaters? My one trip to Moscow Worlds in 2005 could have paid for a year of volunteer-related costs.

I don't see people at the top of their fields getting involved.

Why would they? Hillary Hahn, who was trained in ballet as well as being one of the greatest violinists doesn't have the time. She might not even be the right person had she the time, since it's not her skill set. I know music teachers, dancers, choreographers, singers, and instrumentalists who have much better skill sets for the job: experience with students who have a wide range of ability and talent, evaluations, and jury work. A Ballet Mistress has a better eye than most dancers, because that's her day job.

But are current figure skating judges really the best qualified to do their jobs, or are they people who have self-identified, worked the politics in their favor and/or are connected, and able to work out judging on a volunteer basis. I don't believe this, and think that if there were professional, paid judges that the quality and accuracy would go up a lot.

What I could see happening is skating fans who also have professional or serious amateur careers or college degrees in the arts deciding that they would like to become actively involved with the sport they love to watch. If they're well into adulthood and have never skated, except maybe if they were dancers and have a lot of time to practice now, they're not going to get to a high level as skaters themselves. They could become judges if they're willing to put in the time to learn the standards and learn to recognize the techniques.

If this were an actual opportunity, I don't think this would be a difficulty, because there are many more people in arts-related fields who would be qualified, given basic training, than there are people who are at the same levels technically and involved in skating.

And if there were a division of labor on the judging panels such that there were separate judging appointments for technical judges and artistic judges, then I could see people who started judging with little skating experience of their own but lots of arts experience getting further up the ladder of artistic judging appointments than technical ones.

Sure, because that's where their expertise and interest would lie anyway. Unless Katherine Healy is interested.

But I don't think there's a place in a meaningful judging system for celebrity or dilettante judges to drop into skating competitions to judge once a year bringing to the table only their outside arts knowledge and whatever they've learned about skating from TV.

That would be horrid, not to mention unprofessional. Lady Gaga was embarrassing enough on "So You Think You Can Dance."

[Aussie Willy]

For one, advertise and outreach. What incentives would they need? Presumably they'd do it because they were interested in figure skating as an expression of music and/or movement.

Yes but then you would be asking for commitment and dedication. And if you are a person from outside the sport why would you do it? You don't get any recognition, you generally sit in a freezing cold rink and you don't get paid. People do need a motivation and incentive to give their time and effort. Just being interested is not enough.

[gkelly]

So we agree that if there were a separate track for artistic judging


What we might not agree on is whether these people would actually do a better job if they actually have no skating experience of their own. You seem to be suggesting that anyone who took years of skating lessons themselves, even if they focused on artistic skating themselves and put little emphasis on standard competition, even if they also had years of music and dance lessons and off-ice performing experience and academic degrees in arts fields, would be LESS qualified to judge the artistic side of competitive sport skating than someone who had only the arts background and had never set foot in a training rink. That the more you know about skating technique, the less qualified you are to judge and should not be allowed to pursue an artistic judging appointment. Let alone anyone who has previously trained and served as an overall judge, at any level from local to international, under 6.0 or original-version IJS.

I will continue to disagree with that.

For one thing, I think you're seeing that existing judges and federations have had some problems with politicizing judging at the international level and from there generalizing that they have nothing but problems with politics to offer. You'd rather throw the baby out with the bathwater because when you hear the words "international judge" or "federation" all you see is dirty bathwater. I'd recommend looking closer.

But let's take it as a given. No one with any skating background before they start is allowed to be an artistic judge, whether they have arts training or not. And no one who is just a fan of artistic skating but has not trained in music or dance themselves. This position is ONLY available to professional performing artists who are willing to volunteer many hours of their time over several years to work their way up from local artistic judge to international artistic judge.

So how should these aspiring artistic judges be trained, some who have experience with evaluation and some who do not? Some who have focused solely on classical music and dance and others who have focused primarily on popular styles? Some who were primarily performers, some teachers, some theorists. People from many different cultural backgrounds on different different areas of different continents. How can we get them all on the same page?

Who's going to (re)write the guidelines and organize the training?

What do we consider a good minimum basis of skating knowledge that all these artistic evaluators should share? Is understanding the basics (without having grown up within the sport) a good thing or a bad thing?

For example, should they accept fundamental principles of appreciating skating such as valuing choreography performed while moving more than that performed in poses while standing still, and movement on one foot more than on both feet side by side, on curves more than straight lines? Should they understand why they will see a vast majority of counterclockwise curves and turns from most skaters, clockwise from a minority, and be taught to value a more equal balance?

Or do you think that understanding the physics and technical basis of the medium actually gets in the way of evaluating its artistic effect?

Of course, the more these skating outsiders spend time volunteering hundreds of hours a year at competitions in their own geographical areas, the more they will see what skaters at all levels are doing in their areas. They'll see and mentally synthesize for themselves what artistic qualities tend to vary based more on individual skaters' interests and talents than on skill level and what qualities are highly dependent on technique.

They'll come to expect qualities that are popular in their area, and then when they rise to the level of art-judging skaters from other parts of the country or other countries they will encountr regional differences.

And even if they can nominate themselves to ISU appointments rather than being sent by federations, they will be influenced by their own national backgrounds and the practices and preferences in the skaters they've judged the most often as well as the practices and preferences of their own off-ice artforms.

Meanwhile, for fans who think they would make good judges and don't want to wait around for the invention of a separate no-skating-knowledge-needed (or wanted?) judging track, you might be able to get involved with judging at the local level and eventually beyond by learning the whole sport.

If you're in the US, here's where to find information about how to get started trial judging: http://www.usfsa.org/About.asp?id=108

Skate Canada doesn't give as much information on their website: http://skatecanada.ca/en-us/getinvol...ltraining.aspx

They do say

Judges/evaluators are usually former skaters. Candidates who wish to become qualified to judge/ evaluate, must attend Skate Canada approved clinics, trial judge and pass both a written and video identification exam at each level. Each discipline of skating, singles, pair, dance and synchronized skating requires individualized training.

"Usually" suggests it's not required to be a former skater.

And you could always become a current -- adult -- skater.

I don't know much about the need or process for becoming a judge in other countries. Some have larger grass-roots programs than others. It helps to live close to an active skating club.

I think most non-skaters who become judges are parents of skaters, who spend the time in rinks anyway and become fascinated and want to get more involved on their own, even to continue after their own kids move on to other things. The important thing is willingness to put in the time and learn the sport.

[kwanfan1818]

I don't think that everyone with an arts background is more qualified to judge artistic skating than everyone who is judging, but I do think there are a subset of arts professionals who would be far more qualified to judge form, musicality, and interpretation, than almost every top level judge out there. I just don't think those people are the famous ones.

One issue with existing technically based judges is the acceptance of poor quality in each of those areas because "that's what skating is" or "that's the way skating looks," or "well, it takes energy away from this or that" and when a Curry or a Takahashi or a Jeannette or a Gosselin or a Chan or a Cranston proves otherwise, he or she is seen seen as an outlier, rather than representing benchmarks to be achieved and possibilities to be realized, because the judges given equal or very close marks to skaters who zip around the rink or do the most difficult jumps. If elite skaters were given proper marks in this regard, the 4's, 5's, and 6's, with a varied range that reflected their individual relative strengths and weaknesses, not only would this be representative of what is put on the ice, but it would be incentive to improve those areas, because they're leaving a lot of points on the table. I thinks it's telling that most elite skaters or coaches say they're aiming to improve their "components," and apart from singling out speed, since speed = greater SS scores, and the rest are anchored to SS, they rarely mention an individual one. Get that SS down, and the rest will follow. Look at Zhivanshina/Gazsi's FD, which was quite remarkable in CH and IN, even if their SS weren't.

There isn't a major dance form out there where technique isn't prized, regardless of the technique, and when a dancer doesn't have the best but has other attributes, like musicality or charisma, that is acknowledged as a trade-off. Were it scored in some way, there would be differing scores in each aspect. A dance professional does not look at a manege (circle of jumps) of turns, for example, and say, well, the barrel turns were really great, but the ending sauts de basque were low, the landing wobbly, and the leg position weak: the best were +2 or +3, so I'll take a little off and give them +GOE. A dance professional would look at the manege as a whole -- a difference creature than an isolated jump -- and say, -GOE or 0, because as a unit, it was neutral or poorly done, however much the person thinks "Too bad, the barrel turns were among the best I've see this year." In figure skating, combinations, especially 3-jump combinations, I see lots of poor combinations getting a lot of good GOE, presumably anchored by the quality of the first jump, especially if that jump is a quad (for men). A dance professional looking at a spin that has awkward and sloppy transitions, and where the spin doesn't change speed deliberately, to follow the music, would be considered, at best, a neutral. If the dancer misses a bunch of diffeicult steps leading into a big trick or takes a slow preparation, not only would dance professionals but many fans would say, "Yeah, but he didn't do the [insert tricky step]," not "Great quad, +GOE," when steps into a solo jump is one thing, different than solo jumps. (I'm talking to you Kozuka, but not only you.)

A dance professional might not prefer Kozuka's more introverted style, but a dance professional would not deny it. The audience might prefer this or that, and frankly, from the scores, I think most judges judge form -- when as technical judges they should know better, at least on the technical elements, and be able to differentiate between muscled jumps where the skater hunches and technically sound entrances, or jumps that are high, but land straight down without flow or extension out of them or jumps with mule kicks getting high GOE, or speed gained by back-pumping -- interpretation, choreography, and the performance aspects of PE like audience.

A professional can distinguish between what they like and what has merit. A professional does not say, "I'll only give top credit for what's familiar, be it music or movement vocabulary."

One of the most fascinating experiences I've had was sitting next to an FSUer's sibling during a skating event. She had danced with a major dance company for many years, and is a professor and (brilliant) choreographer now. To hear her comments on movement and musicality and to see the unlauded skaters she praised was illuminating.

As far as becoming a judge, it's not that I haven't considered it seriously. I've looked into it several times, but apart from the initial freeze-out for not being connected to figure skating or having had serious lessons as a child, which would not have been insurmountable, I didn't think I would ever get enough of a technical grasp of the specifics to be able to be an all-around judge. Parents spend huge moeny on their kids' training, and it's only fair to them and the kids that the judges are equally competent in all aspects of skating, given the current system.

Were there a track for artistic judging, I would be right on it. In some of my spare time, I am an arts reviewer, spending most of my discretionary income on traveling to see the performances I review, as well as a great deal of my spare time, including attending rehearsals and many art forms I don't write about. I've done this despite some jobs that demanded 50-60- hour weeks for years on end and significant business travel. I know what it's like to delve into something a volunteer basis and to be committed to something long term. I don't think I am by any means alone in this. It would not be a huge leap to commit to seminars and go to rinks on my own to observe coaches and skaters in training; it would replace something else I do with the same level of commitment.

[gkelly]

I would just say that if it were deemed practical and beneficial to separate out the PE/CH/IN judging from the technical judging and make it a separate judging appointment, in my opinion it would be best to encourage existing judges who are interested or experienced in the arts AND former skaters/adult skaters who are interested or experienced in the arts AND skating fans who are interested or experienced to participate in the same training and to share their knowledge and thought processes with each other so that everyone improves their knowledge base and best practices, not to shut out any of these groups.

Before training rank and file of artistic judges to serve at domestic and international competitions around the world, there should be a committee of skating officials and skating choreographers/coaches/retired competitors and experts in arts evaluation to come up with the standards and procedures to apply to competitive skating performances.

If that's your area of expertise, maybe you would be a good person to get involved in such a committee (assuming the ISU ever formed one and sought out experts from outside the skating world). I think to some extent they may have done this at some training seminars, and I know they have brought in some ballroom experts to consult on original/short dances for the following season, but not in a systematic way as far as I know.

But I don't think it's true that skating judges are all devoid of artistic knowledge and should all be forbidden from judging the artistic aspects of skating, or that if they do have substantial knowledge in that area they lose or chose to ignore it the minute they get an international appointment and are pressured to judge politically. Because they are not all performing up to ideal standards, you seem to be dismissing that there is any value in having experienced judges on this part of the panel, and I disagree.

We can imagine tearing down the existing structure and rebuilding with entirely new officials from scratch as a thought experiment, but in practice that's not going to happen. So it would be better to look at how to make better use of the artistic knowledge that already exists within the judging corps, to improve the knowledge of those judges who don't have an artistic background but want to learn, to let existing judges who prefer to focus just on the technical aspects opt out of getting certified to judge on the PE/CH/IN panel (or fail to certify them if they can't pass the certification exams), as well as to bring in outside experts who could enhance the process of judging these components.

I don't think bringing in only outside experts to replace all skating-knowledgeable judges in this task is the answer, but that often seems to be what your suggesting. Nor do I think that bringing in outsiders will prevent them from being subject to some of the same political forces that judges with skating backgrounds have, once they're inside the system.

[kwanfan1818]

I don't see existing figure skating judges showing that they can competently mark the components. I don't think they even get the SS, the anchor component, successfully according to the critera. Perhaps according to the way it used to be, but not to the criteria. I wouldn't block them: I just wouldn't assume their competence until they proved themselves, which I don't think scoring at the senior level shows.

I do expect, human nature being what it is, that artistic judges would be subject to the same political pressures and ambitions as current judges. I'm enough of a cynic to realize that if the current juggernaut exploded, releasing the grip of the durrent order, a new order would replace it, even if all of my pipe dream would come to pass -- split panel and split judges and professional, paid judges. Until then, it would be nice if the judges showed this alleged competence by scoring the components properly, according to the criteria, some of which could use a renaming and revamping, like PE.

[gkelly]

Until then, it would be nice if the judges showed this alleged competence by scoring the components properly, according to the criteria, some of which could use a renaming and revamping, like PE.

I think we should focus more on defining "properly" and developing a consensus among the judging corps (as it now exists or as it could with revamping). If you're defining it as "the way kwanfan1818 would have scored these performances," of course no one but you will always get it "right."

[kwanfan1818]

I read the bullet points and evaluate through the bullet points. Sometimes I wish the bullet points were different, but they are what they are, even if evaluating "intellectual involvement" is not possible. "Properly" means by the criteria of the time. Reviewing the scores by the judges, especially for the performances I've seen live, I can't say they do the same. I don't assume my interpretations are entirely correct, but that's what rigorous debate is for.

Why should they score to the written criteria, when there's no penalty for not doing so, and not doing so in a corridor is rewarded? Why should they have expertise, when scoring the way they do is rewarded by their Federations and consolidates their power? Is is always correct when there is a strong majority doing something in a given way?

IJS was not developed through a consensus of the judging corps. Where the ISU lost it was creating the system, and then not enforcing it, as if going to some seminars was going to change the political landscape.

[Iceman]

Maybe this thread should be sent to the ISU.