3 ms·

Probably true, but the big question is "where is the efficient frontier, and is it better than what we can achieve now with human institutions?" I like to thin

Probably true, but the big question is "where is the efficient frontier, and is it better than what we can achieve now with human institutions?" I like to think of AI not as "artificial" intelligence but as "aggregate" intelligence. Computers are really good at aggregating the opinions of millions of individual humans. That's what search algorithms like PageRank do, it's what recommendation systems do, it's what voting systems do, it's what markets do, it's what statistical quant trading systems that try to predict the movements of markets do. For these purposes, a computer seems to be reliably better than humans. So why not use computers for what they're good at, aggregating the behavior of millions of individual humans, while still letting humans give their own input.

ben_w 2y ago
While computers are indeed very efficient at this: to the extent that the aggregation is useful, it gets gamed. PageRank was great, until SEO was invented by someone who realised there was an opportunity if they gamed it for free advertising, then it became a Red Queen race between SEO teams and Google. Markets are mostly pretty good, but the USA has a special law about onion futures because someone realised they could game the free market: https://en.wikipedia.org/wiki/Onion_Futures_Act https://en.wikipedia.org/wiki/Onion_Futures_Act Etc.
- nostrademons 2y ago
  But powerful humans are also gamed, and usually more easily than computers. Haven't you ever told an exec what they wanted to hear, or refrained from telling a manager something that might make you or your project look bad? The whole lobbying apparatus is about altering the perceptions of lawmakers so that you get more favorable laws. People usually operate on fewer pieces of data than computers, and so one falsehood can alter their decision-making more than it would a computers. It doesn't have to be perfect, it just has to be better than what we have now.
  - ben_w 2y ago
    > But powerful humans are also gamed, and usually more easily than computers. It's the "more easily" that I'd dispute. A computer can be examined in detail, its weaknesses probed ruthlessly and systematically until a vulnerability is found. Any one human will have traits in common with others, and differences from those others. For example, your own question: > Haven't you ever told an exec what they wanted to hear, or refrained from telling a manager something that might make you or your project look bad? this would be a dangerous similarity, one that can be manipulated — but not universally, not all of us, and not all the time: there's people who react very negatively to such sycophancy or boasting. > The whole lobbying apparatus is about altering the perceptions of lawmakers so that you get more favorable laws. People usually operate on fewer pieces of data than computers, and so one falsehood can alter their decision-making more than it would a computers. Yup, absolutely. But if the software developer misses something that an outsider can learn about — this can be something that just didn't even exist at the time the developer made their choices, or just a bug they didn't spot — it can be abused in the same way. When it comes to accidents, the USA almost started WW3 because someone forgot to account for the fact that the moon rises in the same part of the sky that a Soviet strike group would come from, and also doesn't respond to IFF pings. The Soviet union made their own mistake with sun glare. When it comes to things outsiders learn and the original developer didn't think of, other than all bugs and exploits, consider also things like this: https://arxiv.org/pdf/1707.07397 https://arxiv.org/pdf/1707.07397

ben_w 2y ago

While computers are indeed very efficient at this: to the extent that the aggregation is useful, it gets gamed. PageRank was great, until SEO was invented by someone who realised there was an opportunity if they gamed it for free advertising, then it became a Red Queen race between SEO teams and Google. Markets are mostly pretty good, but the USA has a special law about onion futures because someone realised they could game the free market: https://en.wikipedia.org/wiki/Onion_Futures_Act https://en.wikipedia.org/wiki/Onion_Futures_Act Etc.

nostrademons 2y ago
But powerful humans are also gamed, and usually more easily than computers. Haven't you ever told an exec what they wanted to hear, or refrained from telling a manager something that might make you or your project look bad? The whole lobbying apparatus is about altering the perceptions of lawmakers so that you get more favorable laws. People usually operate on fewer pieces of data than computers, and so one falsehood can alter their decision-making more than it would a computers. It doesn't have to be perfect, it just has to be better than what we have now.
- ben_w 2y ago
  > But powerful humans are also gamed, and usually more easily than computers. It's the "more easily" that I'd dispute. A computer can be examined in detail, its weaknesses probed ruthlessly and systematically until a vulnerability is found. Any one human will have traits in common with others, and differences from those others. For example, your own question: > Haven't you ever told an exec what they wanted to hear, or refrained from telling a manager something that might make you or your project look bad? this would be a dangerous similarity, one that can be manipulated — but not universally, not all of us, and not all the time: there's people who react very negatively to such sycophancy or boasting. > The whole lobbying apparatus is about altering the perceptions of lawmakers so that you get more favorable laws. People usually operate on fewer pieces of data than computers, and so one falsehood can alter their decision-making more than it would a computers. Yup, absolutely. But if the software developer misses something that an outsider can learn about — this can be something that just didn't even exist at the time the developer made their choices, or just a bug they didn't spot — it can be abused in the same way. When it comes to accidents, the USA almost started WW3 because someone forgot to account for the fact that the moon rises in the same part of the sky that a Soviet strike group would come from, and also doesn't respond to IFF pings. The Soviet union made their own mistake with sun glare. When it comes to things outsiders learn and the original developer didn't think of, other than all bugs and exploits, consider also things like this: https://arxiv.org/pdf/1707.07397 https://arxiv.org/pdf/1707.07397