Hypothetical: Would a Qualifying Score system for US Nationals be more fair?

Discussion in 'The Trash Can' started by Sylvia, Sep 5, 2010.

  1. Sylvia

    Sylvia Whee, summer club comps!

    Joined:
    May 27, 2002
    Messages:
    30,020
    Food for thought from the U.S. Senior Ladies thread:
    Any thoughts and/or practical proposals on how to implement such a system and to make it as "fair" as possible?
  2. Coco

    Coco Well-Known Member

    Joined:
    Dec 14, 2002
    Messages:
    10,582
    I'm blushing, :)

    Advantages: doesn't automatically eliminate skaters with injuries during either of the two qualifying systems; lets skaters from stronger regions make it through the process at the expense of weaker skaters from, well, weaker regions; gives judges an incentive - over time - to get it right for the good of the country.

    Disadvantages: variances in applications of the scoring system from panel to panel will skew results and give judges an immediate incentive to inflate their scores. Skaters have as many chances as they have competitions, this might force them to compete too much, or give them a unproductive attitude that they always have a 2nd chance.

    That's why I like the "top 20" score vs. a qualifying score standard (everyone above X qualifies). You don't know what others will do, so there will be an incentive to do as much as possible. I think regionals and sectionals will still happen, they just won't be the only chances for skaters.

    As for making it "fair", well the only way to do that is to judge accurately.

    Assuming, sadly, that's not going to happen, I think a formula that takes base value and -GOE at 100%, and multiplies +GOE and PCS with a factor that minimizes variances in judging panels is best. I'm thinking of something that would look at all the scores from that particular judging panel in that competition, and determine trends in GOE and PCS. Sounds expensive and complicated, but preferable to just taking the raw scores.
  3. DreamSkates

    DreamSkates New Member

    Joined:
    Feb 22, 2007
    Messages:
    815
    It's all "greek" to me! I'd have to take a refresher math class to figure it out.
  4. haribobo

    haribobo Well-Known Member

    Joined:
    Feb 20, 2002
    Messages:
    6,554
    I'd be all for it. Maybe dump Regionals and just have Sectionals, with Qualifying Rounds. I think a combination process would be good- allow the top 10 summer/fall comp scores a bye to Nationals, and then take the top 3 from each Sectional, ensuring multiple chances. If they wanted smaller junior and novice fields, they could take the top 6 summer scores and then top 2 from each Sectional. In that case, the following skaters would currently be in line for a bye to Nationals in Junior Ladies- Gracie Gold, McKinzie Daniels, Angela Wang, Courtney Hicks, Lauren Dinh, and Mary Beth Marley. That would significantly reduce the current imbalance between Sections. More spots for Pacs and Mids ladies, and less for Easterns. Not to be hating, but come on now, do we want the best at Nationals, or not? CoP makes it so easy to do this, so why not? I don't even think score tweaking would be necessary- so far I think the judging has been pretty normal and balanced. Maybe they could assign certain summer/fall comps as qualifying events (Liberty, Detroit, Golden West, Indy, Lake Placid, Mid Atlantics, Glacier Falls, DuPage, etc) so you don't get some wacky backyard comp judged by the skater's parents as a counting score. :p
    Last edited: Sep 5, 2010
  5. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Which levels are you talking about? It might work for senior only or junior and senior, but sending all the novice ladies straight to sectionals would make sectionals unmanageably large, which kind of defeats the purpose.

    What I could see doing for the upper levels (junior and senior only) would be to change qualifying from the current funnel-shaped process (regional qual rounds narrowed down to final rounds, narrowed down to sectionals and then to nationals) to a sieve-like process instead, or something like the grand prix system.

    Offer junior and senior events at all the regionals that are being held anyway for the lower levels, and at all the sectionals that are being held anyway for novices (who qualified through regionals) and for pairs and dance (who start there), and also at one or more designated summer competitions in each section. JGP, GP, and senior B internationals would also be designated. Skaters who want to qualify to Nationals have to enter (at least?) two such competitions, but the first is not a qualifier for the second. They can choose two events on the basis of convenience, which could be two different regionals, two different sectionals, or one of the above plus one summer competition or one international. But they'd have to indicate by the Sept. 1 deadline which two events they intend to count.

    Definitely the summer competitions would need to be from after July 1, when new ISU rules go into effect. There were some big changes this year, for example.

    Then take the combined, average, or best score from the two designated competitions to determine who qualifies for Nationals.

    A "bye" might mean the option of using two international scores. Except, perhaps, for last year's top 5 seniors who have the option not to compete at all in the fall and go straight to Nationals.

    Using scores would be better than using placements because of differences in the strengths of the fields at these different kinds of events. But there are still potential inequities in using scores as well, as already discussed in this thread. No system will work perfectly.

    That won't happen. Judges aren't allowed to judge their own kids.

    They can sometimes judge their children's direct competitors, so in theory that would be an opportunity for funny business.
  6. jlai

    jlai Title-less

    Joined:
    Nov 21, 2002
    Messages:
    8,777
    Continuing the discussion in the senior ladies thread...

    I agree top 5 should get byes. Hypothetically (making up a new system here) fOr the rest they either do well enough in interationals to earn their byes (like placing top 6 in their gp or making the jgpf) or they qualify through regionals and sectionals. But if a skater goes to gp and places 11th, I think that skater should probably go through qualifying.

    Regionals and sectionals will decide that. I just think those with not-so-spectacular or stagnant/declining international results should be doing some US qualifying competition too. For instance I think it won't hurt someone like Czisny to do a qualifying this year. It's good for the development of skaters to get that "jolt" once in a while. Didn't Todd Eldredge do regionals and sectionals when he came back in 00-01?

    I don't think it has to come down to "forbidding them (skaters) to accept the invitation". Like I said, if the sectionals are more spread out, those with poor gp showings could be asked to do sectionals, as long as USFSA allows them to compete in a sectional that they don't normally belong in. That may make skaters focus on qualifying for their sectionals and skip their gp.

    I don't think one less US skater on the GP will mean much in the long run unless that skater is extremely promising. In fact, US has had many many ladies on the gp the last couple years, but having that many ladies hasn't improved US's chance on the gpf (or eventually worlds). For that you only need 3-4 very good ladies (ala the Japanese team).

    It's probably just me, but last year the byes ended up with a large field of competitors but the competition at US nationals wasn't necessarily better. I want a good competition, not necessarily a large one. Of course, if the goal is to qualify as many as possible then forget what I said.
  7. Jenna

    Jenna Well-Known Member

    Joined:
    Jun 20, 2009
    Messages:
    7,139
    Haribobo brought up the idea of having qualifying competitions, and I like that idea very much. Choose, let's say, Liberty, DuPage, Detroit, Golden West and Middle Atlantics. The top 20 combined scores from those events would move on to the National Championships. The key is that the same exact panel would judge/call each event to eliminate bias. Therefore, this would probably only limit this system to the novice, junior, and senior levels. Otherwise it could become exhausting.
  8. sk9tingfan

    sk9tingfan Well-Known Member

    Joined:
    Jan 24, 2006
    Messages:
    1,944
    IMO, the bottom line as to whether or not the judging system can be considered as valid and reliable depends on COP's reliability and validity in statistical terms. Validity is defined as the system measures what it purports to measure. Reliability is defined as the system measures what it purports to measure consistently.

    I think that the jury is still out as to the absolute reliability and validity of COP, let alone intra-judge(within a given judge's assessments) and inter-judge (between judge) reliability. Judging, IMO appears to becoming more valid, although not always.

    It would make a great dissertation topic though. ;)
  9. Coco

    Coco Well-Known Member

    Joined:
    Dec 14, 2002
    Messages:
    10,582
    But this, having the same panel at these events, doesn't happen now, right?

    I'm wondering if it would be concentrating too much power and responsibility in too small of a group. Also, even amongst the same panel, scores can be influenced by the size or quality of the competitors as a whole.

    At some point, the scoring system needs to WORK and allow for inter-competition comparisons, within the same season. I would hate to discount PCS entirely, that would be a mistake, but that is the area that is most likely to lead to unfairness.

    It will be interesting to compare, after sectionals, the scores of nationals qualifiers with the scores of non-qualifiers.
  10. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Yes, you're right that this is where the discussion belongs.

    So it sounds like you're not so much interested in making sure that skaters who deserve it get to Nationals as in keeping out skaters who haven't proved they deserve it based on this season's skating.

    Historically, in the past couple decades there have usually been about 18-20 senior ladies, and often senior men, at US Nationals, more or less. Less, of course, when there weren't that many senior men in the US at all some years, or when there were fewer byes given and taken.

    I think fields of more than 24 would be too large.

    So aim for 18 or 20 but expect it might end up anywhere between 15 and 24 in any given year.

    The fluctuation in size might have to do with byes, injuries, withdrawals and retirements, how many JGP skaters are competing domestically in senior vs. junior, timing of international events, etc. There will rarely be a direct correlation between number of competitors and quality of the competition.

    And it's not really possible to predict in advance how many not-really-national-level skaters will qualify or how many generally good skaters will skate badly at Nationals.

    Olympic years are likely to have larger fields and more pressure on the skaters who hope to make the team. Post-Olympic years will tend to have smaller fields with fewer stars.

    In the ladies' event, the cut between 4th and 5th at sectionals is likely to be between skaters of similar ability, depending who had a better two days at the qualifying event. And that fourth-place qualifier from some regions may end up in last place at Nationals or bottom of the top 10 or anywhere in between. Especially when we're dealing with talented but inconsistent skaters.

    That's sometimes true for the men as well.

    In other disciplines, though, it has often been the case that there only are four or five skaters at sectionals at all, or there are fewer than four entries who have what it takes (but may not show it today), so the third and fourth place slots may go to skaters who just don't have elite-potential talent to begin with.

    But you can't predict from year to year how many good skaters will enter from each region or section. So for the sake of fairness, up to four from each section have always been allowed to advance.

    I attended a few Nationals in the 1990s, and a few sectional competitions, I've seen tapes from some others from the early 90s and 1980s. More recently, I've watched on IceNetwork. There has been some pretty bad skating at Nationals in the bottom of some fields in some years. And bad performances from good skaters that landed them in the middle of the field, lower than they had achieved or would later achieve in other years.

    Some years, even the final freeskate group is a splatfest.

    So what? You can't predict all the potential disasters and weed them out in advance. As long as the size of the field is still in the manageable range, including more skaters doesn't just increase the potential for disaster. It also increases the potential for some really good performances from skaters other than the established stars. And the bad performances put the good ones in perspective.

    I don't think there's any point in trying to keep Nationals as small and elite as possible.

    If the goal is to try to get as many of the good skaters there as possible, so what if an occasional bad skater sneaks in?

    No, it wouldn't hurt her, but it might hurt the skater who finishes a close 5th at her sectionals. And it wouldn't especially hurt her or you if she doesn't do a qualifying competition.

    But does USFS necessarily know when making assignments in the spring and summer which skaters would most benefit from that jolt? They don't have to go out of their way to make sure that former medalists all get byes, but they also don't have to go out of their way providing jolts to selected skaters who disappointed recently. If after shuffling international assignments a bye works out, fine. If not, the skater can go qualify.

    But what if the poor GP showings take place during or after the last sectionals? Or during the next-to-last one, which would mean a last minute change in plans (expensive travel arrangements, coach not necessarily available) once it's known how well the skater did at the second international.

    For the most part, if skaters get to skip both regionals and sectionals it's because of the timing of their international events. For the JGPF qualifiers, some skaters had conflicts and some didn't, so for fairness they all have been given byes.

    Occasionally a skater with international scheduling conflicts will have bad performances at the internationals and get to Nationals anyway. This affects at most a handful of skaters a year, so I don't see it as a problem that needs to be solved.

    You could insist that all skaters (except last year's senior top 5) must attend a domestic qualifier and spread out the timing of those qualifiers so all international skaters can find at least one without a conflict. They'd have to register in advance and expect to compete. Maybe skaters with especially good international performances before the qualifier they'd signed up for would be told they don't have go qualify after all. But it's hard to make it equitable since some skaters would (choose to, or have to) fulfill that obligation at their regionals before the international event.

    Keep in mind, though, that sectionals also need to be scheduled more than a week or two after the last of their respective regionals, not during Thanksgiving or Christmas week, preferably not during the same week as Junior Nationals (for availability of officials). So there's only so far they could be spread out.
  11. crzesk8dad

    crzesk8dad Active Member

    Joined:
    Jan 9, 2005
    Messages:
    463
    If I recall...I brought this up a year or so back and was told that I was full of Zamboni droppings....:huh:
  12. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Those events are all in the summer. You'd still need to have some in the fall. Currently the various regionals, which are the qualifiers for large fields of juveniles and intermediates. Might as well continue to hold higher level events there as well, although skaters at the upper levels could have the option of skating their qualifying events in the summer rather than fall if they prefer.

    The test deadline for qualifying is currently September 1. Most of the summer club competitions do not require a skater to have passed the test already (i.e., skaters are allowed to "skate up" one level). Many of them hold short programs and long programs as separate events.

    And remember that there are a few hundred novice ladies in the US, and historically only 12 go to Nationals. Most of the regions have novice qual rounds. Most of the big club competitions have novice qual rounds.

    Over a hundred nationwide for junior and senior ladies.

    I don't think we want to replace the nine regionals with five or six club competitions. Maybe augment the regionals with additional club comps.

    I don't think this would work very well for novices, especially the ladies.

    For skaters who are not contenders for the top 12 or top 20, it doesn't really make much difference whether they only enter one phase of the event or competed at a level they hadn't passed the test for. They're not going to qualify anyway.

    But for those top few dozen who are in contention, how do you decide which scores to count?

    *What if the skater scored higher in qual rounds than in final round? Do you use the qualifying score?

    *What if the skater qualified for the final round in short program but not in long, or vice versa? Can you make a combined score using one qualifying round and one final round?

    *What if the skater tested in August and earned her highest score in June or July? Can the earlier score count, or does she need to use another competition from when she was officially at that test level?

    *What if a skater entered different competitions at different levels over during the summer-fall qualifying season?

    Suppose, e.g., a talented young skater wins some novice competitions in spring or early summer, enters a junior competition later in the summer just to test the waters, and scores well there as well. She has one of each in a designated summer qualifying competition. Both or either of her novice and junior high scores might hold up to qualify her, or they might not. She doesn't know in August what other skaters might do in October. So should she test up or not? Should she compete again in the fall? Does her test status as of Sept. 1 determine which level she's eligible to qualify in? What if that doesn't match her test status at the time the scores were earned?

    And then at junior and senior level, you also have the whole question of how international events figure into the qualifying process for the top skaters.
  13. bek

    bek Guest

    I think top 20 scores would be to hard and unfair. Perhaps a fairer system would be to say the winners of sectionals/regionals make it through to ensure every section gets some representation. and then judge everyone else through a qualifying score. And the score would have to be reached through an accredited competition though either national/club.

    Then except for those who pre qualified through last years results etc, everyone else would HAVE to put up that score somewhere, if you have 2 GPS you would have to put up that score at your GP event.

    They could always aim for a somewhat high score, and then lower the score if not enough people put that score up. I think a qualifying score would be significantly more fair than the current system. Its a bit detrimental when a talented skater like Agnes gets left off the JPG because she missed nationals due to a tough sectional.

    If you have an injury etc, than a person with documentation who made that score last season could maybe get a bye.
  14. Coco

    Coco Well-Known Member

    Joined:
    Dec 14, 2002
    Messages:
    10,582
    Just curious, bek: why is it important to have every section of the country represented?

    I'm also wondering if being from a strong section makes a young skater become better than they otherwise would have been. They develop knowing that they have to be really sharp just to get to Nationals, and they have tougher competition from a younger age. This has to have a positive impact on them.

    Conversely, how many skaters from weaker sections aren't pushed to become the best they could be because they can "get away" with efforts that wouldn't get them to sectionals in other regions?

    By no means do I think someone is loafing it on purpose. But w/o being pushed, who can say what they might have achieved?

    In any event, switching to a qualifying score system won't eliminate any of the benefits of the top 4 from sectionals, but it will get around that perceived drawback.

    I like limiting it to the top X number of skaters above a certain qualifying score, because that gives you the advantage of fixing the maximum number of skaters (and practice groups) in a discipline and setting it in stone. Is that best for the sport? Well, it doesn't hurt from the business end of things. And I don't think the US is really hurting itself by excluding the 25th best skater in a discipline.

    I know in my sport (women's lacrosse) there are MANY differences in officiating from region to region. It's something we really struggle with. Not sure if this would be a problem in skating. You don't want to give a region's judges and callers incentives to inflate scores and be lenient on calling levels and rotations.

    As an aside, I do think that the new downgrade "partial credit" rule will go a long way towards improving consistency amongst callers regarding jumps. I think some thought the previous penalty too harsh, and resisted hitting skaters with downgrades. Now they don't have to worry so much.

    And with a qualifying score system, no one would be getting a bye unless they legitimately couldn't compete between July 1 and the cuttoff point (Christmas?). USFS can decide if they want to give someone a bye for "promoting the sport", like Lysacek is arguably doing this fall. But really, it would be more democratic, imo.

    I think regionals and sectionals will still be very popular, well attended competitions. These are later in the season than the summer comps and will represent skaters best chances of earning a qualifying score.

    Maybe USFS could require sectionals to be judged by officials from another section? Would that help?
  15. HisWeirness

    HisWeirness Yay, new board!

    Joined:
    Nov 10, 2007
    Messages:
    11,600
    You could have (like in other sports) an "A" score and a "B" score. The "A" score would be enough to guarantee you a spot at Nationals and a spot the following year if you have an injury. Something around 160 maybe for ladies, or the A score could be the 5th place score at US Nats the previous season. You could require meeting the A score once or twice to get the bye.

    The "B" score could be the nth (n = 10 or 15, for example) place score at US Nats the previous season. You could increase the value of n to meet a target # of skaters getting this bye to Nats if necessary. If you qualify to Nats via the B score, you do not get the injury bye that the A score gives you.

    They could also implement a minimum required score to be eligible to go to Nationals. For example, if a lady can't score more than 100 or 110 for SP+FS, then she cannot go to Nats (unless they win their sectional, as you suggest). They could follow the ISU model and make the minimum score TES only to eliminate PCS issues.
  16. bek

    bek Guest

    Well I think traditionally its important. BUT, I also think that winning sectionals or regionals should probably mean something. I'm not sure if they want to do top 4 or whatever, but I do think its a good idea if your going to have sections (and I think some kind of competition like that is necessary) than I think winning should mean something.

    I must agree that perhaps coming from a strong region makes you better. Perhaps missing nationals one year made Agnes and Mirai better skaters. However, when you consider that nationals is the be end/end all and it can mean junior skaters losing out on international competitions (junior skaters like Agnes who skated fairly well at their sectional) there is a problem.

    Its hard to know. I think in general coming from tougher competitions CAN make you better, but I also think that if the US is looking for the best possible people to represent them, than having uneven sections is a problem.

    I agree about the problem of some competitions becoming "inflation reputation competitions". Surely though there could be strict warnings about said competition develops it, the competition for the next year could be placed on review, which means tapes of all "qualifying" routines would have to be submitted or something like that. If a REAL abuse was going on. But the fact that the winners of the section would move on I think would stop those kind of things from happening. I think the benefits out way the risks.
    Last edited by a moderator: Sep 7, 2010
  17. jlai

    jlai Title-less

    Joined:
    Nov 21, 2002
    Messages:
    8,777
    I base this discussion on the rather large ladies field from last year, which was 22, and that wasn't counting the early withdrawals like Musademba and Maxwell. Had Meissner also competed, the field would end up being 24, which is large even by your definition. If the intended field should be 18-20 like you said, that means last year the intended field of 24 was too large, with a couple too many byes.

    The men's field was 22 + 2 withdrawals (Carriere and MIner), which means the intended field was 24.

    I think my tweaking only would cull the skaters down to about 18-20, which is near what you're shooting for too.

    Whether a skater should do qualifying competition should depend on the needs of a skater and not the needs of other skaters from that region. If a skater hasn't been skating well and should probably go through qualifying there's no reason to keep him/her out of it just to benefit someone else. Sending a skater to the same internationals year after year for mediocre results isn't always the best thing to do. Besides, giving the SA TBA spot based on recent qualifying competitions may not be a bad idea. That way the assignments could get more spread out also if some up-comers end up doing very well recently

    I think your argument is "can't tell in advance", but in many cases, there're telltale signs months in advance. We all had concerns about how E. Hughes could manage school and skating. And some had heated arguments over her being sent to GP the last couple years. Fanshere also had concerns about other skaters who've had issues long before they competed at GPs.

    a handful of skaters in a field of about 20. That's 10% of the competition.

    Or we could have minimum scores like what has been suggested in the thread--if a skater is ill-prepared and ends up with 130 in a gp (say, ladies) then that skater doesn't go to nationals automatically.
    Last edited: Sep 7, 2010
  18. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    I meant a handful across all disciplines in junior and senior, not just in senior ladies.

    That would be a big change in the way qualifying works. But as I said in my first post in this thread, if we use scores from various designated competitions (big club comps, regionals/sectionals, and internationals) as qualifying criteria, then skaters with two or more internationals that conflict with domestic qualifiers could have the option of using the international scores as qualifiers. Not a direct bye to Nationals, but a pass out of regionals/sectionals.

    How close are we getting to scores being comparable between different events and different kinds of events?

    When we get there, such a system will work. It'll never be exact, but I think they are getting somewhat more consistent.
  19. bek

    bek Guest

    Well if they make it qualification by TES like the ISU is doing and another person mentioned, I think it will be difficult for the scores to be that different per competition. (And another reason I thought letting sectional winners go to nationals is that it will provide a compromise. Perhaps people from weaker sections won't be willing to vote for a system that may mean nobody from their section can go to Nationals. But if they are thrown the bone of a guarantee of one person, it may make a difference.

    I think the main thing is making the system better. I don't think the system is soo much a problem for the Seniors (I dont' feel sorry for Alissa that she has to qualify given her performance at Nationals) but I do think it presents significant issues with Junior skaters and younger ones. When you have someone like Agnes who could have used the JGP experience last year, and the fact that it really will affect her world ranking going into the Senior GP...
  20. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Another point that occurred to me is that by using scores from many competitions, a lot of skaters won't know for sure whether they qualified for Nationals until long after their qualifying event is over.

    As things stand now, if you go to regionals and sectionals and place top 4 at your sectionals, you know as soon as the results are posted that you've qualified for Nationals, currently mid-November. If you have internationals that conflict with sectionals are granted a bye to Nationals, you know as soon as the assignment and the bye are confirmed. If you have early JGP assignments and don't know after your last event whether made the final you have to wait a few weeks to find out whether you're going to the JGP final and get a bye to Nationals or whether you need to compete at sectionals. But either way you'd have the answer to that question in mid-October, and then if you do compete at sectionals you'd know right away whether you qualified or not.

    Suppose the sectionals are spread out so there are 2 in November and 1 in December. JGPF is also in December. Top X scores from various events will qualify for Nationals (and sectional winners qualify regardless of score). Now a skater who places 2-4 at the first or second sectional event, or maybe lower than that but with one high score from their one JGP or a summer event, has to wait and see how their scores hold up as the other skaters' scores come in from the remaining GP and senior B events and from the JGP final. Some skaters could be finished competing the second week of November and be "on the bubble" not knowing whether they qualified until 5 weeks later. Others may only have to wait 1-3 weeks after their own last event. But that's still a lot of uncertainty for a lot of people.

    Other points about whether to use of TES scores only as qualifiers.

    Yes, they're much more objective than PCS and therefore more comparable across different events with different panels. But are they really the best way of deciding which skaters are worthy of advancing to Nationals?

    The ISU is not using scores at previous events to determine who qualifies for the championships. The minimum score requirements are just thresholds to determine whether a skater may even be considered. But a federation that has multiple skaters who meet that threshold can use whatever method they choose (often results of national championships) to determine which of them to send to the ISU events.

    So the scores function more like test credentials than qualifying competitions. They're not being used to compare skaters and choose between them.

    If we are using scores to choose which skaters advance, then using only TES could lead to a paradox:

    Suppose skaters A and B both compete at the same sectionals.

    Skater A gets a couple of downgrades or intentionally doubles some triples and gets a couple of level 1 spin calls, average GOEs around +1 even with minuses for the downgrades. So the TES in relatively low. But the basic skating level and the performance quality are high, so s/he gets PCS in the 5-6 range and finishes 2nd or 3rd in the field.

    Skater B has harder jumps in the SP, 7 rotated triples in the LP (or rotated but not cleanly landed quad), and all level 4 spins, GOEs averaging -1. The TES is high on the strength of the base mark, but the skating and performance are less strong, with PCS averaging in the low 4s. S/he finishes behind skater A in the overall sectional standings.

    And neither of those skaters had a higher score from another competition to be considered instead. It's the sectionals scores that will decide their fate.

    Now it's time to determine Nationals qualifiers. If you use only TES, skater B has a higher total. But skater A had higher TSS in both short and long program, clearly defeated B in both programs and in the overall sectionals standings.

    What if skater B's technical scores are high enough to fall 20th on the list of potential qualifiers and skater A's TES are only 27th on the list? Do we really want B to qualify to Nationals and A to miss out on the basis of performances at a competition where A clearly defeated B?
  21. Polymer Bob

    Polymer Bob New Member

    Joined:
    Feb 17, 2007
    Messages:
    2,667
    A good question actually. Should every country be represented at the Olympics, or should the Japanese send 7 ladies because they have that many good ones? I guess there is really no totally fair way to do it.
  22. Sylvia

    Sylvia Whee, summer club comps!

    Joined:
    May 27, 2002
    Messages:
    30,020
    I believe any plan to change the current qualification system for Nationals must be approved by a vote at Governing Council (by delegates from all the figure skating clubs in every region/section/state of the country). Some of us, I am sure, remember what happened when the "12/4" plan was proposed years ago and the reasons why it didn't pass at Governing Council...

    But not every talented skater can afford to relocate to a strong section -- or, more specifically, a major training center. And is it fair to penalize the "weaker" regions that currently don't have major training centers and/or a larger group of elite or name coaches?
  23. bek

    bek Guest

    If the qualifying score is set before the season starts, and the competitions where you can earn the qualifying score is also set, than the skaters should know as soon as they get their scores, if they made it to nationals or not. They may reserve the right (if not enough people have reached the standard to lower the qualifying score before Nationals.

    But its also not fair to penalize stronger skaters because they come from stronger regions. And I think its specifically problematic when we are talking about young skaters and giving them JGP's etc. If they are going to keep the system, they need to make some changes on how JGP selections are made out so a skater like Agnes doesn't miss out no JGP spots. Or another skater doesn't miss out do to injury etc. That's why I think that a system that allows sectional winners and maybe even more than that but also introduces a qualifying scores for others would be a good compromise. Weaker sections could ensure their own representation, while the USFSA could ensure the best skaters are at nationals. Win/Win.
    The thing is though that the ISU championships are at the end of the line. The ISU itself isn't trying to figure out what skaters it wants to represent them at a bigger meet. It seems to me part of the point of Nationals is to figure out who will represent us internationally. So a system that excludes some of our best skaters is counterproductive towards that aim.
    Last edited by a moderator: Sep 7, 2010
  24. Coco

    Coco Well-Known Member

    Joined:
    Dec 14, 2002
    Messages:
    10,582
    I'm not sure those things are absolutely necessary for a young skater to be competitive, although they surely help.

    It may take a few years initially, but I think a qualifying score system will push skaters from weaker sections to improve more than the currest system does.

    But if allowing each sectional's winner, or highest ranking non-qualifier, provided they are 4th or higher, to qualify is necessary to get the change made, so be it ;)

    Addressing the issue gkelly raised, about not knowing what score is needed until very late in the process, perhaps this could be a two tiered thing, where a skater needs to hit a qualifying score by a specified date (a week after the last regional). Or instead of a qualifying score, they need to be in the top 50 scores, or be within 25% of an average of the top 5 scores, etc., there are a million ways to do this.

    Then from there, the top X scores as of 12/1 will be taken. It doesn't eliminate that issue, but it does lower the odds someone fluyky will come from out of nowhere at the 11th hour. It also it spares parents of skaters who really don't have a chance from spending gobs of money to travel to "just-before-the-deadline" competitions. It also rewards consistency, to some degree.

    Unfortunately, there will still be people that come oh so agonizingly close, only to miss a trip to nationals. However, I think it is more fair for those people to lose out on a score comparison nationwide, then to lose out because they are from a strong section, and clearly performing better than qualifiers from a weak section.

    Maybe a formula that is base value + (-GOE) + 1/2(+GOE +PCS)? Wonder what that would yield...
  25. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Oh, you're going to start out setting the score pretty high and everyone who meets that score will qualify?

    What if they think they're setting a high bar but either there was a mistake in setting it that didn't take into account last-minute scoring changes, there were mistakes or manipulation or just a general trend toward generosity from some of the panels (whether domestic or international), the skaters really rise to the challenge, or whatever, and 30 or 40 skaters meet that minimum?

    Right. And some of the best skaters might not deliver the best technical content at their qualifying events. If they can beat weaker skaters with harder jumps even if they water down their tech content at sectionals, shouldn't they have the opportunity to go on to Nationals and show what they can do with fuller content, if they're able?

    Remember, when they get to Nationals and when they get sent on to international events, they're going to be judged on PCS as well as TES. So the decisions of who to send on should be based on total scores, not just TES.

    If that's too inconsistent, then don't use total scores at all -- use placements, as is currently the case.
    Last edited: Sep 7, 2010
  26. bek

    bek Guest

    Well those stronger skaters will have more than just one competition to meet that score. If they haven't met the score anywhere, than one could ask if they really are that strong.

    Maybe using PCS is fine/makes sense. I'm not sold on either using TES or PCS.
    If the score is set high enough than 30/40 skaters won't meet it. If they are going to allow placements at sectionals than there's no reason to have a low score it can be something even as high as lets say 150 total for a Senior woman.
    Last edited by a moderator: Sep 7, 2010
  27. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    How does that work?

    Regional, or sectional?

    So they know in October or November that they're within the top 50 scores or they've hit the preset minimum, but they don't know until December whether they made the cut for top 24 or whatever the maximum number at Nationals has decided to be?

    The problem isn't that someone fluky would come from nowhere in December. It's that a skater who had been assigned to the last GP event or senior B event and hadn't met the minimum score yet, or who qualified for the JGP with relatively low scores in relatively weak fields, might earn a high score in December and qualify for Nationals, knocking out someone who had been among the top 24 or whatever the cutoff is since sectionals ended.

    A skater who has a GP assignment or who qualified for the JGP would not be considered a fluke to qualify for Nationals. But until they actually earn the qualifying score, they haven't claimed one of the Nationals slots yet, and a skater who is currently 23rd in the standings may or may not end up getting bumped.

    Or is the idea that everyone who hits a certain minimum goes to Nationals and there will be qualifying rounds there for the larger (i.e., ladies') fields?

    Is the plan to eliminate sectionals? What about pairs and dance, which now only start at sectionals? What about senior men, who often don't have enough skaters in one region to compete against and start at sectionals?

    Different posters in this threads have different plans, so I'm not sure we're always talking about the same thing.

    But the skaters who don't really have a chance do like to go to regionals -- that's their big event of the season. And if they happen to qualify for sectionals, that's their big accomplishment. If they beat some former national competitors in the process, more sense of accomplishment.

    If they live in Montana or someplace like that, they're going to have to travel anyway to get to a designated competition, whether it's club comp, regionals, or sectionals. Maybe they can really only afford one out-of-state trip per year, two (i.e., qualifying for sectionals in the current system, or for nationals if they're in an event that doesn't have regional competition) would be a stretch. For a skater whose current just-within-reach big goal is getting to sectionals to compete on the same ice with nationals contenders and who will be "pushed" by doing so, in the qualifying score proposal would they be best off planning their one big trip to be out of region to a big summer competition or to regionals, or sectionals if they have the option of choosing that as their one event?


    I agree with that, to a certain degree, but not if the score comparison is TES only.

    Complicated calculations, but that would be more fair.
  28. bek

    bek Guest

    That's what I'm thinking.

    It would depend on their goals, and how they feel they'd do at their sectionals. If your sectional is pretty weak and you think you have a shot at qualifying directly there, you'd obviously go there. Perhaps though the rule could be that scores in sectionals count unless the skater is skating internationally (at assigned events) and or the skater has had a documented injury and then a qualifying score at a club competition could be used. That might keep things more organized. Maybe IF the skater showed up to sectionals and had a score that was high enough for club competitions, that could be used to. But they'd have to show up for sectionals unless injured (etc)
  29. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    OK, here's an idea.

    Pick a qualifying score now for junior ladies, senior ladies, junior men, and senior men based on what we expect to be a good cutoff under this year's rules. (Ideally it would have been chosen in June, as soon as the new rules were known and before the first summer competition that counts.)

    Skaters who have already competed at summer competitions or the first JGP already have scores.

    Over the fall, let's keep track of scores from the last few club competitions and international events, regionals, and sectionals.

    Can JGP skaters who have passed their US senior tests use scores from junior events to qualify for seniors? Since the difference in content is only one ChSp or ChSt, plus an extra 30 seconds for in-betweens, it won't make too much difference.

    In December let's see how many skaters meet those minimums and how closely the nationals fields that would be chosen by this method match those that are actually chosen by the current system of sectionals placements + byes.

    Then, or after we see how the skaters actually do at Nationals, we can evaluate whether it looks like a qualifying score system would work just as well or better at choosing the nationals qualifiers.
  30. Coco

    Coco Well-Known Member

    Joined:
    Dec 14, 2002
    Messages:
    10,582
    To clarify my earlier post....

    I'm thinking that the qualifying score should be hit by the week after regionals, yes regionals, and it will serve a similar function to regionals.

    The top X scores, I'm thinking top 18 scores, of skaters from amongst those who've hit the qualifying score by the specified time, will then be selected for nationals. They could go by rankings as of 12/1 or 12/15, something like that, definitely after sectionals.

    As a "bone" for the weaker sectionals, consider this:

    Skaters A, B, C & D finish 1-4 at a sectional. Skaters A & B qualify for nationals by virtue of their top 18 score. Skaters C&D do not. Skater C will still qualify to nationals by virtue of being the highest ranked skater from sectionals, (from amongst those ranked 4th or higher), who doesn't qualify through her score.

    I think it's better than allowing the winner from the sectional, as most sectional winners will always be in the top 18.

    I'm in favor of capping the senior national fields at 24. At the other levels, a case could be made for a bigger field. But in any event, to get at 24:

    1) you take the top 18 scores from that season, scores earned in either international competitions or designated domestic events between July 1 and December 15,

    2) a max of 3 "wild cards" from amongst the non-qualifying top four finishers at each of the 3 sectionals,

    3) the remaining number of spaces (minimum: 3) from either injury byes or returning champions, or skaters with scores ranked 19-24, chosen in order of their rank.

    As for the skaters with late season GPs, competing only once is a risk they take. If they are truly on the buble, then that's just the breaks. If they legitimately couldn't compete prior to that, and something happens at the GP that prevents them from skating to their abilities, they can petition for a bye.

    Perhaps there should be one last ditch competition after the last GP and JGP events, and after sectionals. It would be open to skaters who'd met the qualifying score, or were recognized through a bye as having a legitimate reason to not meet the qualifying score by the end of regionals cutoff date. It would also be open to skaters who were safely going to be w/in the top 18, but wanted feedback on levels, more experience, etc. I would totally pay to watch this online :)

    For purposes of ranking scores, it would be 100% of Base Value, 100% of -GOE, and 50% of +GOE and 50% of PCS.

    ETA: and yes, I'm all for doing this and comparing it to actual qualifiers.
  31. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    This seems to be fairly well thought out. Some significant differences from what jlai and bek are suggesting. Do we want to agree on a (hypothetical) plan and refine it together for comparison purposes?

    Correct me if some of the assumptions below are not what you had in mind.

    Several club competitions in July through September will be designated as official qualifier competitions. Skaters can enter as many of these as they choose in whatever location(s) they like and use their high scores there as qualifying scores.

    Regionals will still be held in October. Juvenile and intermediate singles skaters will still qualify through their regions directly to Junior Nationals, with qual rounds. Whether they use scores or placements or a combination to determine who goes can be decided later, because this thread is focused on juniors and seniors.

    Novice singles skaters (a couple hundred ladies nationwide) still qualify from regionals to sectionals and sectionals to Nationals?

    Junior and senior events will be offered at regionals and sectionals. However, skaters at these levels can choose to enter any regional or any sectional, not necessarily in their home region or section and without having to qualify from region to section? Or they can choose to enter none of them if they're happy with their summer scores and/or have international conflicts?

    Or are they required to enter one if they don't have a conflict? Are sectional junior/senior events open to anyone at those levels who prefers competing in November than October, or open only to those who place well at regionals?

    Possibly senior men's event is offered at sectionals but not regionals. Or, since it's all about scores and not placements, an event can be held if there's only one skater, for him to earn official scores.

    Pairs and dance teams at all levels use sectionals or the few designated pair and dance summer events, or internationals if applicable.

    At the time of regionals are over in October, some of the skaters will be finished competing and some (a few top skaters with later GP/B internationals) won't have competed at all yet. A significant number will already have a score but will also still be planning to compete at least once more at sectionals or a later international.

    What does this mean? The "qualifying" score is the minimum score needed to be eligible for nationals that year, but doesn't in itself qualify you for nationals?

    Does anyone who has met that minimum (whether at summer competition, regionals, or early international) get to go to sectionals unless they have a conflict? Does where they earned the score affect which sectional they go to?

    Do they have to go to sectionals (or later international) or can they choose to use the score they've already achieved to count toward their national qualification?

    This is not the top 18 as of the end of regionals, though. So after regionals, skaters only know that they are qualified to compete at sectionals

    At the end of sectionals, they know how well they did at sectionals and what their high score is. If they're high up in the top 18 they may be guaranteed a spot to Nationals. If there are still other skaters scheduled to compete internationally who haven't qualified yet, the skaters ranked about 16-18 as of the end of sectionals are still "on the bubble" and waiting for everyone else to finish competing to learn whether they qualified.

    Actually, historically the junior and novice levels are always smaller. Novice fields have pretty much always been 12 because there are almost no byes.

    Juveniles and intermediates have qual rounds at Junior Nationals, but they don't go to sectionals. And their programs are shorter :), and juveniles (and intermediate pairs) don't do SPs.

    Nationals is already a week-long competition. If you added qual rounds for all disciplines that need them at all levels, you'd need at a week and a half.

    Which means you can't finalize the field until after Dec. 15. It's not just a matter of adding a few more, but of skaters at the bottom of the top 18 after sectionals waiting to find out if they'll be knocked down to 19 or lower, and potentially whether the person who knocked them down was from their section or elsewhere.

    3) the remaining number of spaces (minimum: 3) from either injury byes or returning champions, or skaters with scores ranked 19-24, chosen in order of their rank. [/quote]

    Do you mean maximum 3?

    How are injury byes awarded? They were eliminated a number of years ago -- why bring them back?

    By "returning champions" above, do you mean people who actually won Nationals or Worlds/Olympics recently enough to qualify directly to Nationals without competing according to the current rules (e.g., Sasha Cohen 2010)?
    Are there also still byes for skaters who finished in the top 3? 4? 5? at Nationals last year, who have probably gotten Grand Prix assignments but can afford to skip them, and also skip domestic events, if they're injured, because they already have a top-X bye, which is not the same as an injury bye?

    E.g., if Mirai Nagasu is still injured and unable to compete at her GP assignments (or regionals/sectionals) this fall, she gets a bye to Nationals because she's a former national champion and/or because she was 2nd last year? But, e.g., Caroline Zhang or Alexe Gilles or Kristine Musademba would not have that option -- if they can't do their GPs or sectionals, they're out of luck.

    This would have to be scheduled in late December (or early January in years when Nationals is a few weeks later). I don't think it would happen.
  32. Jayar

    Jayar Well-Known Member

    Joined:
    Dec 22, 2001
    Messages:
    8,602
    If there is a minimum score needed to make Nationals, or there is a required score to get to Nationals, you can bet that every skater will migrate towards the Midwestern Sectional. The scores are ALWAYS inflated there, and they would have the best shot of making it from there.

    A more interesting way would be to have the top 6 from each Section go to Nationals qualifying. The top 12 get in...
  33. mgobluegirl

    mgobluegirl Well-Known Member

    Joined:
    Aug 13, 2005
    Messages:
    2,028
    This is true, but its not like using scores would take spots away from anyone long term, it would change every year, depending on where the top scores are coming from. I think everyone is alluding to the fact that Easterns is very weak in junior ladies this year, while both mids and pacs are absolutely stacked. But it certainly isn't always like this, and it's not like that this year because the East lacks training centers or good coaches or anything. It's just how it's shaping up this year. When "region hopping" was banned and replaced with the percentage qualifying system, it was only discussed for the developmental levels (juvenile and intermediate). It was perceived that region hopping hurt the development of skating in smaller or weaker sections, but if that talent hasn't developed by the novice level, the skaters from the weaker region don't make it out of their sectional. I sort of see using qualifying scores as a nationwide scale of this.

    I don't necessarily think this is true. Maybe recently, scores have been higher at Midwesterns, but I think that's because the quality of skating has been pretty impressive.
  34. jlai

    jlai Title-less

    Joined:
    Nov 21, 2002
    Messages:
    8,777
    Re: Question of qualifying by region or qualifying by top/best scores

    It's a tough one. The reality is that in Olympics and worlds, the countries without a strong skating tradition get to enter 1 person --even a skater without a full set of triples--while the strong countries get to enter 3 at most. Geog representation is a big part of any Olympic sport, and I think regionals/sectionals only mirrors that geographic representation system within the US. Unless we do away with geog representation all together (kind of like tennis grand slams) that will remain part of the skating system, whether within US or beyond.

    Is geog representation a fair system? Kind of hard to say. Depending on your values I guess.

    ETA: IMHO qualifying systems will always mean uneven fields to an extent, e.g. the qualifying rounds at worlds weren't even back when they existed, and that was based on ranking and such. ANd the fields at GP are also rather uneven too, which makes qualifying for gpf sometimes a combo of luck and skill.
    Last edited: Sep 7, 2010
  35. Jayar

    Jayar Well-Known Member

    Joined:
    Dec 22, 2001
    Messages:
    8,602
    It is very true. Skaters from the Midwestern Section almost always score higher in their sectional than they do at Nationals. And their scores are seemingly always higher than the other sections. It's been happening since 6.0... remember when AP McDonough got a 6.0 at Sectionals?
  36. gkelly

    gkelly Well-Known Member

    Joined:
    Sep 26, 2003
    Messages:
    10,494
    Just tossing around an idea, which would certainly need more refinement.

    As of next year, juveniles and intermediates will be going to sectionals, so those events will continue to be held in November in any case.

    For the lower levels, there does need to be some kind of funnel system because there are just so many girls at those levels in many of the regions, and letting them all do two events would just compound that problem.

    Suppose that each regionals operates as it always has for novice and below.

    For junior and senior singles only:

    Every regionals in October offers junior and senior men's and ladies' events open to any eligible competitors with the appropriate test credentials. Each sectional event in November also offers open junior and senior events -- at these levels, the only difference between the regional and sectional events are the months when they're held. It would be expected that skaters would enter the event in their home regions, but it might not be required.

    Each skater could register for two of these events at the beginning of the qualifying season. I'm not sure it should be possible to add another event after the deadline if one does unexpectedly well at the first.

    Skaters who are aiming to qualify for Nationals should enter two events. Then use a point system to determine the top 12 qualifiers, who move on to Nationals.

    I'm not sure it would make sense to include summer competitions because the dates don't work out with the qualifying deadline. But probably that could be worked out if needed.

    For skaters whose international assignments allow them to enter only one of the US qualifying events, there should be a way to use the international placement as the second event for qualifying points.

    Would it need to be weighted somehow so that, e.g., 5th at a GP event counts for more than 5th at a senior B, and both count for more than 5th in seniors at a regional event?

    Skaters whose international assignments preclude them from entering any of the qualifying events get a bye directly to Nationals, assuming they actually attend both events.

    Specific policies for timing of international fall events relative to US qualifiers and for withdrawals from one event with good results at another would need to be worked out.
  37. gotice

    gotice New Member

    Joined:
    Nov 17, 2008
    Messages:
    99
    Thanks Sylvia for bringing this back - wasn't aware, great stuff!! Some issues still ensue with these ladies listed from last year - yes, Daniels went onto 4th in Jr. at nats four months later, yet only a year later from this post she's not anywhere near getting a bye to nationals as a senior. And then those who are moving up levels.

    Then you have the juniors who did actually change sections this season to Easterns for example in juniors this year who are trying to get to nationals Shpilband and McBeath. So did we penalize (Gold/Keiser/Wang/etc) the girls last year for not changing regions/sections? They did have that opportunity. Yet, going back farther (and I'm only using skaters names as examples not trying to call anyone down - these kids are all so talented) Ashley Wagner never competed out of SA which is far more competitive region than NWP, yet she trained on the East Coast. Yes, she has two GP assignments this season, but since she didn't get top 5, wouldn't she have had to qualified at regionals if she wasn't getting a bye thru south atlantics to skate canada.

    So, the current system is utilized by some skaters to benefit their situation to get where they desire. Some don't. Lots get lost in the shuffle who should have had a better shot. I'm all for leveling the mark and getting the right people where they belong.
  38. haribobo

    haribobo Well-Known Member

    Joined:
    Feb 20, 2002
    Messages:
    6,554
    If my system as proposed was used last season, the junior ladies at Nationals would be: Hicks, Marley, Wang, Dinh, Daniels, Gold (for top 6 summer scores), then Kulgeyko and Edmunds (top 2 Pacs), Jiang and Cain (top 2 Mids), and Timlen and Dunne (top 2 East). So in the end Wang and Gold would have replaced Eng and Rajic. Keiser still wouldn't have made it but at least she had more chances. So yes, I still approve of my own suggestion. :D

    This season we would see G. Gold, H. Miller, B. Long, L. Verhulst, K. Shpilband, and P. Edmunds as auto qualifiers for the top 6 summer scores (thanks to the Unseen scoretracker!)
  39. Jayar

    Jayar Well-Known Member

    Joined:
    Dec 22, 2001
    Messages:
    8,602
    Most of you are advocating for a hippy qualification system that includes everyone. Not everyone should qualify for Nationals-- only the best. How many other sports have qualification to Nationals based on where you live? Not many. The fairest way is to have Regionals all over the country like they do now and have a minimum qualifying standard to Nationals Qualification. At Nationals Qualification, send the top X number of skaters to Nationals. Summer competitions aren't qualifying competitions-- if you want to see skaters burn out, then make the summer just as important as qualies.
  40. query5

    query5 New Member

    Joined:
    Jul 3, 2009
    Messages:
    672
    i know it is hypotheical,
    b ut i wonder what brought in on.
    a skater you like that has to qualify and don't want to.
    or a skater/team you don't like that don't have to qualify and should.
    or a skater you don't like that want to qualify and not make it to nationals.

    just wondering that is all.