There is something to gain, because when you call him a neo-Nazi and he obviously isn't, then nobody will engage with your arguments because you're very obvious

There is something to gain, because when you call him a neo-Nazi and he obviously isn't, then nobody will engage with your arguments because you're very obviously wrong. If you say he seems to be racist and supports lots of far-right groups that are overtly racist... then people can't just ignore you.