One instance blocking another is is not about freedom of speech. If you own a server, you are allowed to choose who has access to it.
If you say every server must allow all people access to it, are you going to allow spammers access? Or identity thieves? Or scammers?