I agree there is a lot of "averaging" going on, but here are two things that I think make it somewhat better than "100IQ". First, it seems like the people who t

I agree there is a lot of "averaging" going on, but here are two things that I think make it somewhat better than "100IQ". First, it seems like the people who talk the most about many narrow subjects actually have more interest and perhaps capability in them than the median person, and it will train mostly on the people who talk the most about particular token sets. Second, there is the unreasonable "wisdom of crowds" which works for some set of things (though it may no apply to LLMs). So it's weird, but I expect LLMs to give me better answers in narrow well discussed fields than broad but highly argued fields. I guess it's still really important to know how to ask the right question.