3 ms·
> the far more serious alignment problem we have is the one we’re currently running 24/7: maximizing profit with no regard for externalities That's more well-k
by ben_w 4y ago
> the far more serious alignment problem we have is the one we’re currently running 24/7: maximizing profit with no regard for externalities
That's more well-known rather than more serious. It's basically why the Communist Manifesto was written.
"Money" is basically a really good approximation for what we really care about, and the difference between it and what matters — the reason Bhopal/Union Carbide[0] is seen as a disaster rather than a legitimate cost of doing business — is that law and other interventions are able to update and respond at a "close enough" rate.
AI might still be slow enough, if humans are in the loop.
An agentic AI (definitely including GOFAI[1], not just LLMs or future tech AGI/ASI) not so much.
Even where it's aligned, you get weird problems just because there's less chance to do investigate journalism before disasters happen:
Imagine an alternate reality where 9/11 didn't happen in 2001, and in 2030 a pure AI business starts and operates a substantial lights-out cyanide factory in central Manhattan that nobody even notices because it's all done carefully and safely right up until 2031/09/11 because that is this hypothetical alternative reality version of the terror attacks, and in this case those attacks rip apart the factory and spread the cyanide over much of the city.
[0] https://en.wikipedia.org/wiki/Bhopal_disaster https://en.wikipedia.org/wiki/Bhopal_disaster
[1] https://analyticsindiamag.com/8-real-life-examples-algorithms-turned-rogue-causing-disastrous-results/ https://analyticsindiamag.com/8-real-life-examples-algorithm...
- serverholic 4y ago[dead]