No.
The goal of neo-Nazis is to reach as wide an audience as possible.
They hate bans or restrictions, because they know it hampers their ability to recruit. Why do you think they want to federate?
Allowing them to federate with us is EXACTLY what would tear us apart, as they start convincing a sizeable chunk of the fediverse that people of a different skin colour/religion/sexuality deserve to die.