In the same vein, if you as a conference organizer don't want to deal with the drama that an attendee might bring, you should have every right to exercise your

In the same vein, if you as a conference organizer don't want to deal with the drama that an attendee might bring, you should have every right to exercise your freedom to deny speakers. The same people who would criticize you for being racist will criticize you for screening racists from speaking. There is no winning in these situations. I personally value individual freedom over anything else. If you want to be a bigot and make people uncomfortable, or run around naked and make people uncomfortable, you should have every right to. Just as people have the right to not like you because you are doing things that make them uncomfortable.