I can, without too much effort, think of 3 other "worst things."
Worst thing #1 is a shortage of qualified judges (an issue the US Nationals might be facing, according to Philip Hersh), with the event ultimately decided by last minute replacement judges, and Olympic spots determined and questioned as a result.
Worst thing #2 is skaters stranded in Sweden, unable to return to their home country or the country where they are coached.
Worst thing #3 (and my own personal worst thing) is that a March Worlds turns into a superspreader event, with coaches, judges and skaters getting sick and bringing the sickness back with them.
I'm sure there are high level skaters who lose motivation over the course of their careers. There are generally a number of post Olympic retirements. And it could well be that there are skaters now who are looking at an uncertain situation and weighing it against going to school or entering the work world.
But my guess is the ones who really want to compete in the Olympics will have the same, or greater, motivation, and a postponed (or even an cancelled) Worlds 2021 won't change that.