this is indeed a costly problem, but I suspect there are economical solutions using technology. Expert identification can, to a degree, be bootstrapped out of

this is indeed a costly problem, but I suspect there are economical solutions using technology. Expert identification can, to a degree, be bootstrapped out of a crowd using something like Prelec's Bayesian Truth Serum (https://www.science.org/doi/abs/10.1126/science.1102081 https://www.science.org/doi/abs/10.1126/science.1102081). The tldr is that it fishes out holders of unusually surprising knowledge relative to the population. I was involved in a study that used BTS effectively to identify expert dermatologists. I imagine it's possible to find a sweet spot between voluntary querying (a-la slashdot) by a rotating, small party combined with a variant of BTS which surfaces the experts as needed. The cost of identification will still scale with the group, but it might be doable at ratios that are similar to moderator:users in forums.