Yes, because racism is all about people being hypersensitive to comments that weren't meant to be racist
You know, sometimes people make racist comments because they genuinely believe that people not like them are stupid, uppity, inferior, dangerous, etc. etc.
If Harry grew up in an environment where no one seemed to think it was maybe not OK to throw a "colonial"-themed party, to wear a Nazi uniform for fun, to say things like "slitty eyes" at public events....maybe he did need some re-education.