3 ms·

> What we've learnt about technology is that what it does depends greatly on who it serves. You made many good points, but if this one resonates with anyone in

by e1g 4y ago

> What we've learnt about technology is that what it does depends greatly on who it serves. You made many good points, but if this one resonates with anyone in the context of AI advancements, I recommend this long-form essay, "Meditations On Moloch" [1]. It's a thought-provoking exploration of the unrelenting drive for competition and self-interest, often leading to suboptimal outcomes for individuals and society in politics, education, economics, and the environment. [1] https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ https://slatestarcodex.com/2014/07/30/meditations-on-moloch/ or the audio version at https://sscpodcast.libsyn.com/meditations-on-moloch https://sscpodcast.libsyn.com/meditations-on-moloch

dredmorbius 4y ago
Care to summarise what you think the core applicable message here of SSC's Moloch is? If it's that behaviours are emergent ... yes, I get that. (I've skimmed bits of the piece numerous times. Each time I attempt to dig in, the length puts me off. Formatted for print, the main text runs > 50 pages, and with comments to nearly 300.)
- e1g 4y ago
  I drew parallels here because Ezra Klein reached the same conclusions as SSC did at the end of those ~50 pages a decade earlier: when building a god machine, we must slow down forcefully if necessary. It's practically impossible, but the stakes are too high. Both Klein and SSC discuss four inescapable problems stemming from such complex systems: 1. Uncontrolled arms race: The competitive development of AI technologies is already an arms race where BigTech and VCs rush to create increasingly powerful AI systems without taking adequate safety precautions. 2. Misaligned Goals: systems are designed to optimize specific objectives, but if those objectives are not perfectly aligned with human values, the system will continue to take actions that are harmful to humans (the "alignment problem" in AI safety). 3. Economic Disruption: The widespread deployment will lead to significant economic and social disruption, including job displacement and inequality. Those people will not be getting better jobs. 4. Loss of Control: Highly autonomous systems could become problematic for humans to understand or control, especially if they are capable of self-improvement. The central thesis of Moloch is that in a competitive environment, or any environment that requires tradeoffs or optimization, the inevitable terminal state of the system is that everything that is not _the objective_ will be sacrificed to pursue _the objective_. When the desired objective is tightly coupled to human values, this is good because society will sacrifice unimportant things to pursue those important things. But those objectives can get decoupled from our values ("when a measure becomes a target..."), and that unavoidably creates a "race to the bottom," where individuals or groups are incentivized to engage in behavior that is ultimately harmful to themselves and others to gain a competitive advantage towards a Pyrrhic victory. SSC uses the metaphor of "Moloch" to describe this phenomenon, suggesting that Moloch represents the destructive force that arises from the uncontrolled competition and the pursuit of self-interest. edit: It took me two years to consume Meditations On Moloch. I blame the opening poem as it was incomprehensible and barely relevant to the rest of the piece. Ultimately, I found the audio version more approachable - just like listening to a lecture on YouTube. After that, I returned to the text version to reflect on specific chapters and follow up on references.
  - dredmorbius 4y ago
    Thanks. I'm still chewing my way through it, I'll keep this interpretation in mind.

by e1g 4y ago

dredmorbius 4y ago

Care to summarise what you think the core applicable message here of SSC's Moloch is? If it's that behaviours are emergent ... yes, I get that. (I've skimmed bits of the piece numerous times. Each time I attempt to dig in, the length puts me off. Formatted for print, the main text runs > 50 pages, and with comments to nearly 300.)

e1g 4y ago
I drew parallels here because Ezra Klein reached the same conclusions as SSC did at the end of those ~50 pages a decade earlier: when building a god machine, we must slow down forcefully if necessary. It's practically impossible, but the stakes are too high. Both Klein and SSC discuss four inescapable problems stemming from such complex systems: 1. Uncontrolled arms race: The competitive development of AI technologies is already an arms race where BigTech and VCs rush to create increasingly powerful AI systems without taking adequate safety precautions. 2. Misaligned Goals: systems are designed to optimize specific objectives, but if those objectives are not perfectly aligned with human values, the system will continue to take actions that are harmful to humans (the "alignment problem" in AI safety). 3. Economic Disruption: The widespread deployment will lead to significant economic and social disruption, including job displacement and inequality. Those people will not be getting better jobs. 4. Loss of Control: Highly autonomous systems could become problematic for humans to understand or control, especially if they are capable of self-improvement. The central thesis of Moloch is that in a competitive environment, or any environment that requires tradeoffs or optimization, the inevitable terminal state of the system is that everything that is not _the objective_ will be sacrificed to pursue _the objective_. When the desired objective is tightly coupled to human values, this is good because society will sacrifice unimportant things to pursue those important things. But those objectives can get decoupled from our values ("when a measure becomes a target..."), and that unavoidably creates a "race to the bottom," where individuals or groups are incentivized to engage in behavior that is ultimately harmful to themselves and others to gain a competitive advantage towards a Pyrrhic victory. SSC uses the metaphor of "Moloch" to describe this phenomenon, suggesting that Moloch represents the destructive force that arises from the uncontrolled competition and the pursuit of self-interest. edit: It took me two years to consume Meditations On Moloch. I blame the opening poem as it was incomprehensible and barely relevant to the rest of the piece. Ultimately, I found the audio version more approachable - just like listening to a lecture on YouTube. After that, I returned to the text version to reflect on specific chapters and follow up on references.
- dredmorbius 4y ago
  Thanks. I'm still chewing my way through it, I'll keep this interpretation in mind.