I think the assumption is that given that other successful conferences have had speakers that were good (technically), that were non white/male, that if such sp

I think the assumption is that given that other successful conferences have had speakers that were good (technically), that were non white/male, that if such speakers had been considered for this conference, the organisers wouldn't have ended up with a lineup of 100% white & male speakers. You're right though that we have no real evidence for this, this is just a natural assumption to make given the evidence we do have (the lineup). As a poster said above me, this could have possibly been mitigated by publishing who they considered and possibly even selected that were non-white/male and couldn't make it. (this would give us some of the evidence you mention).