3 ms·

> they also outperform human expertise directly When measured statistically. This is not a takedown of that statement, but the reason we've trouble with this

by gopalv 1y ago

> they also outperform human expertise directly When measured statistically. This is not a takedown of that statement, but the reason we've trouble with this idea is that it works in the lab and not always in real life. To set up a clean experiment, you have define what success looks like before you conduct the experiment - that the output variable is defined. Once you know what to measure ahead of time to determine success, then statistical models tend to not be as random as a group of humans in achieving that target. The variance is bad in an experiment, but variance jitter is needed in an ever changing world even if most variants are worse off. For example, if you can predict someone's earning potential from their birth zipcode, it is not wrong and often more right than otherwise. And then if you base student loans and business loan interest rates on the basis of birth zipcodes, the original prediction does become more right. The experimental version that's a win, but in real life that's a terrible loss to society.

bobsomers 1y ago
> > they also outperform human expertise directly > When measured statistically. THANK YOU. It's mildly infuriating how often people forget that one of the things most human experts are good at is knowing when they are looking at something that is likely in distribution vs. out of distribution (and thus, updating their priors).
- jonahx 1y ago
  The original article discusses this explicitly.
AstralStorm 1y ago
Ah yes, the self fulfilling prophecies or hallucinations based on models trained on models. Overfitting. Ending up in an evolutionary dead end... Type 4 error of not asking a question one should also exists. So thing is, suppose you're handling the common cases right - you have software that's say 95% correct. The important bit is how critical the remaining 5% failures are. If one of them happens to be "I give up my computer and data to the exploit" or "everything is destroyed" or "a lot of people die", then the extra 1% better average is no good to any inside observer. It so happens that a lot of people believe themselves to be outside observers, especially rich. (What's the success bonus for someone getting treated nicely?)

by gopalv 1y ago

bobsomers 1y ago

> > they also outperform human expertise directly > When measured statistically. THANK YOU. It's mildly infuriating how often people forget that one of the things most human experts are good at is knowing when they are looking at something that is likely in distribution vs. out of distribution (and thus, updating their priors).

jonahx 1y ago
The original article discusses this explicitly.

AstralStorm 1y ago

Ah yes, the self fulfilling prophecies or hallucinations based on models trained on models. Overfitting. Ending up in an evolutionary dead end... Type 4 error of not asking a question one should also exists. So thing is, suppose you're handling the common cases right - you have software that's say 95% correct. The important bit is how critical the remaining 5% failures are. If one of them happens to be "I give up my computer and data to the exploit" or "everything is destroyed" or "a lot of people die", then the extra 1% better average is no good to any inside observer. It so happens that a lot of people believe themselves to be outside observers, especially rich. (What's the success bonus for someone getting treated nicely?)