4 ms·

He quotes Scott Alexander on "Moloch" (an essay I haven't read yet), "In some competition optimizing for X, the opportunity arises to throw some other value und

by fernly 8y ago

He quotes Scott Alexander on "Moloch" (an essay I haven't read yet), "In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out.... The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse." Is this not a fancy way of describing what's usually called the Tragedy of the Commons? https://en.wikipedia.org/wiki/Tragedy_of_the_commons https://en.wikipedia.org/wiki/Tragedy_of_the_commons

wycs 8y ago
It is about the tragedy of the commons and what it implies when self-modification becomes possible, such as genetic engineering in humans and recursive self-improvement in potential future AIs. That essay is a hell of a piece or writing. It didn't so much change my object-level policy opinions for the here and now, but it ripped the heart out of my Libertarianism. I've always found markets and selection beautiful and fascinating and powerful, which is all true. But I also thought they had a teleology that was humanistic. This is false. To the extent this seems true now, it is only because humans are the current hardware markets are running on. Fundamentally, they are not our side in the long term. I have found few pieces of writing more powerful: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ Nick Bostrom explores similar themes in The Future of Human Evolution: https://nickbostrom.com/fut/evolution.html https://nickbostrom.com/fut/evolution.html
- sooheon 8y ago
  Loved this rabbit hole. Thanks!
elvinyung 8y ago
Moloch isn't just the tragedy of the commons. It's a generalization of it that applies to many more classes of decentralized behavior.

by fernly 8y ago

wycs 8y ago

It is about the tragedy of the commons and what it implies when self-modification becomes possible, such as genetic engineering in humans and recursive self-improvement in potential future AIs. That essay is a hell of a piece or writing. It didn't so much change my object-level policy opinions for the here and now, but it ripped the heart out of my Libertarianism. I've always found markets and selection beautiful and fascinating and powerful, which is all true. But I also thought they had a teleology that was humanistic. This is false. To the extent this seems true now, it is only because humans are the current hardware markets are running on. Fundamentally, they are not our side in the long term. I have found few pieces of writing more powerful: http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ http://slatestarcodex.com/2014/07/30/meditations-on-moloch/ Nick Bostrom explores similar themes in The Future of Human Evolution: https://nickbostrom.com/fut/evolution.html https://nickbostrom.com/fut/evolution.html

sooheon 8y ago
Loved this rabbit hole. Thanks!

elvinyung 8y ago

Moloch isn't just the tragedy of the commons. It's a generalization of it that applies to many more classes of decentralized behavior.