Neutrality is a well defined concept in international law. It is best described as "you two fight and I'll make money".
But generally, the language of "you're either with us or against us", as practiced by W. Bush, is oppressive. It's not that countries have "opposing viewpoints" on the Ukraine war. It's that for many countries, they simply don't care about the conflict enough to take sides, and they don't feel like they should. It's far, it's not their business, and it's not like people who are forcing them to care cared when other wars were affecting other countries. As someone who works in international development, I can tell you that no one in Africa or LAC cares about that war.
Sorry, but I think it's ludicrous to suggest that Bush is the best or primary example of the idea that "if you're not for us, you're against us." The concept goes back millennia, and has been used in many contexts historically, both ethically laudable, such as resistance to Hitler's expansionism in WW2, and ethically dubious, such as Bush's actions in the Gulf. It is not, in an of itself, an oppressive concept. Rather, it is the context within which it is mobilized, that makes it oppressive or liberatory in effect.
And of course there will always be countries that remain "neutral" in contexts of international aggression. Just as Switzerland did in WW2. Often these countries benefit financially, as Switzerland did with Nazi bank accounts. Currently for South Africa and Brazil and likely India too, BRICS plays a large role in their not opposing Russian aggression in Ukraine. Despite this, many South Africans are opposed to Russian expansionism and disappointed in the government's privileging of BRICS. The South African government is not "neutral," though, it is actively pro Russia in its international policy. It has allowed Russia to conduct military exercises off its coast, for example. I think, when you scratch the surface of "neutrality," underlying alliances are quite easily exposed.
And, of course, it goes without saying, that just because countries choose putative neutrality, doesn't make it ethically acceptable. History will bear this out.
Neither are those decisions for "neutrality" made easily or as unambiguously as you imply, or embraced as wholeheartedly, because they will have negative repercussions also, even economically, down the road.
So, in short, I think you have oversimplified the issue, while also focusing on a point that is really only tangential to this discussion, which is primarily about sports boycotts, how to implement then, and their efficacy. In the case of Russia, whose international prestige and internal cohesiveness rests, to some extent, on its international sporting success (one has only to trace the money behind its efforts to be readmitted to the international sporting community to see this), I think a blanket boycott is an effective strategy.