Supposing for a second that you're right, you still need the average upvoter/downvoter to be able to tell the difference between the opinions of world-class exp

Supposing for a second that you're right, you still need the average upvoter/downvoter to be able to tell the difference between the opinions of world-class experts and convincing but baseless speculation for the whole system not to become a misinformation machine. And even if you have that, there's always a risk that someone out there reads the most dangerously stupid comments and interpret them as gospel no matter how downvoted.