2 ms·

If you'd like to join me in speculating a bit, let's take an imaginary look at ten years from now. Good news! AI has advanced enough that it is now possible fo

If you'd like to join me in speculating a bit, let's take an imaginary look at ten years from now. Good news! AI has advanced enough that it is now possible for you to have a customized bot that handles all of the crappy and mundane parts of life: scheduling travel, getting groceries, making sure the house is clean, and so on. The bot learns about you and is able to anticipate your needs. And because we have this tech, we also have bots that will play your emotions like a finely-tuned fiddle. Think Elvis never died? Afraid that lizard-people in the government are controlling your thoughts? Well, now there's a bot that will find others like you and deliver to you a continuous stream of information confirming your beliefs -- and warning you of the dangers lurking out there. Now guess which of these bots makes more money for its creators, the one scheduling airline flights or the one keeping you emotionally-connected with 400 people just like you and making sure that all of you are living a dramatic life on the edge where you're fighting the bad guys?

manjushri 9y ago
So it seems the theory here is that strong AI whose primary objective is to make money is ultimately self destructive?
wpietri 9y ago
Definitely. It's the kind of manipulative dystopia that might have been designed by timeshare salesmen and propagandists. But the weird part to me is that conscious design seems basically unnecessary. It's dystopia metastasized. I've interviewed the occasional person looking to leave Facebook, and one of the surprising things to me is how small everybody's job is. They're all optimizing micro-metrics for micro-gains, with very little thought as to the systemic impact. Those micro-metrics in turn roll up to plausible-sounding things like revenue, DAU, time on site, and engagement. Yay engagement! How could that be bad? But I've never heard anybody from Facebook, employee or exec, say, "Gosh, could we be acting like tobacco company execs or slot machine designers?" They don't even seem to understand addiction as a topic, let alone a concern. It's very much as Upton Sinclair said: "“It is difficult to get a man to understand something when his salary depends on his not understanding it.”
- michaelfeathers 9y ago
  I think it's because we don't understand systems as well as we should. This article (http://www.miamiherald.com/news/business/article207841309.html http://www.miamiherald.com/news/business/article207841309.ht...) is a decent intro to what modeling shows about engagement and polarization. The paper it refers to (https://arxiv.org/pdf/1712.06009.pdf https://arxiv.org/pdf/1712.06009.pdf) goes way deeper.
cwkoss 9y ago
This is basically the state of Twitter today.

manjushri 9y ago

So it seems the theory here is that strong AI whose primary objective is to make money is ultimately self destructive?

wpietri 9y ago

Definitely. It's the kind of manipulative dystopia that might have been designed by timeshare salesmen and propagandists. But the weird part to me is that conscious design seems basically unnecessary. It's dystopia metastasized. I've interviewed the occasional person looking to leave Facebook, and one of the surprising things to me is how small everybody's job is. They're all optimizing micro-metrics for micro-gains, with very little thought as to the systemic impact. Those micro-metrics in turn roll up to plausible-sounding things like revenue, DAU, time on site, and engagement. Yay engagement! How could that be bad? But I've never heard anybody from Facebook, employee or exec, say, "Gosh, could we be acting like tobacco company execs or slot machine designers?" They don't even seem to understand addiction as a topic, let alone a concern. It's very much as Upton Sinclair said: "“It is difficult to get a man to understand something when his salary depends on his not understanding it.”

michaelfeathers 9y ago
I think it's because we don't understand systems as well as we should. This article (http://www.miamiherald.com/news/business/article207841309.html http://www.miamiherald.com/news/business/article207841309.ht...) is a decent intro to what modeling shows about engagement and polarization. The paper it refers to (https://arxiv.org/pdf/1712.06009.pdf https://arxiv.org/pdf/1712.06009.pdf) goes way deeper.

cwkoss 9y ago

This is basically the state of Twitter today.