3 ms·

Game theory / Moloch should've never let open-source software happen. While not perfect it works surprisingly well despite potentially suffering from all the sa

by amasad 6y ago

Game theory / Moloch should've never let open-source software happen. While not perfect it works surprisingly well despite potentially suffering from all the same coordination problems outlined in this essay: free rider problem, tragedy of the commons, and all sorts of game theory race-to-the-bottom scenarios. I also notice random acts of kindness from people, spontaneous corporation amongst rivaling tribes or companies, and things a strict self-interested agent-based analysis will tell us it's not possible. It's an insightful model of the world but it's too simplistic and pessimistic.

wizzwizz4 6y ago
Part VIII. > Somewhere in this darkness is another god. He has also had many names. In the Kushiel books, his name was Elua. He is the god of flowers and free love and all soft and fragile things. Of art and science and philosophy and love. Of niceness, community, and civilization. He is a god of humans. > The other gods sit on their dark thrones and think “Ha ha, a god who doesn’t even control any hell-monsters or command his worshippers to become killing machines. What a weakling! This is going to be so easy!” > But somehow Elua is still here. No one knows exactly how. And the gods who oppose Him tend to find Themselves meeting with a surprising number of unfortunate accidents. > There are many gods, but this one is ours.
- toohotatopic 6y ago
  Flowers and free love, just the other side of the coin: >4. The Malthusian trap, at least at its extremely pure theoretical limits. Suppose you are one of the first rats introduced onto a pristine island. It is full of yummy plants and you live an idyllic life lounging about, eating, and composing great works of art (you’re one of those rats from The Rats of NIMH). >You live a long life, mate, and have a dozen children. All of them have a dozen children, and so on. In a couple generations, the island has ten thousand rats and has reached its carrying capacity. Now there’s not enough food and space to go around, and a certain percent of each new generation dies in order to keep the population steady at ten thousand.
slx26 6y ago
Moloch does require people willing to sacrifice some values or externalities in exchange for individual benefit. At scale, there's always people willing-to or forced-to or unknowingly doing that. But the examples you write about are mostly related to "individual contributions". You don't need all individual contributions to be like that for Moloch to be able to exist. You only need some, and scale. And then, the more pressure we have in a system, the lower the bar becomes for individuals to be willing to sacrifice certain things. I don't believe what you say points at any real contradiction. That being said, I do not necessarily agree with Scott about the conclusion that we really can't coordinate unless some super-AI "helps us destroy Moloch".
- chrchang523 6y ago
  Yeah, I think Scott loses his way at the end of that post. He is overestimating the chance that Warg Franklin’s approach is doomed; once you correct for that, it is not clear that trying to construct a values-aligned singleton is the best path forward, though I’m not opposed to research along that line.
RcouF1uZ4gsC 6y ago
> Moloch should've never let open-source software happen. Maybe Moloch is catching up to open source. We read regularly of open source maintainers who are poor, burned out, and quitting. A lot of the open source work that happens is actually controlled by for profit corporations. Despite Chrome and Android being open source, it is optimized for advertising revenue for Google. Successful open source projects are incorporated in a SaaS offering, often cutting out the original developers. Other open source projects are acquired by large corporations and are shuttered. There still is not a reliable strategy for open source developers to support themselves, other than trying to use that notoriety to get hired by a big corporation, or try to upsell to non-open source add ons.

by amasad 6y ago

wizzwizz4 6y ago

Part VIII. > Somewhere in this darkness is another god. He has also had many names. In the Kushiel books, his name was Elua. He is the god of flowers and free love and all soft and fragile things. Of art and science and philosophy and love. Of niceness, community, and civilization. He is a god of humans. > The other gods sit on their dark thrones and think “Ha ha, a god who doesn’t even control any hell-monsters or command his worshippers to become killing machines. What a weakling! This is going to be so easy!” > But somehow Elua is still here. No one knows exactly how. And the gods who oppose Him tend to find Themselves meeting with a surprising number of unfortunate accidents. > There are many gods, but this one is ours.

toohotatopic 6y ago
Flowers and free love, just the other side of the coin: >4. The Malthusian trap, at least at its extremely pure theoretical limits. Suppose you are one of the first rats introduced onto a pristine island. It is full of yummy plants and you live an idyllic life lounging about, eating, and composing great works of art (you’re one of those rats from The Rats of NIMH). >You live a long life, mate, and have a dozen children. All of them have a dozen children, and so on. In a couple generations, the island has ten thousand rats and has reached its carrying capacity. Now there’s not enough food and space to go around, and a certain percent of each new generation dies in order to keep the population steady at ten thousand.

slx26 6y ago

Moloch does require people willing to sacrifice some values or externalities in exchange for individual benefit. At scale, there's always people willing-to or forced-to or unknowingly doing that. But the examples you write about are mostly related to "individual contributions". You don't need all individual contributions to be like that for Moloch to be able to exist. You only need some, and scale. And then, the more pressure we have in a system, the lower the bar becomes for individuals to be willing to sacrifice certain things. I don't believe what you say points at any real contradiction. That being said, I do not necessarily agree with Scott about the conclusion that we really can't coordinate unless some super-AI "helps us destroy Moloch".

chrchang523 6y ago
Yeah, I think Scott loses his way at the end of that post. He is overestimating the chance that Warg Franklin’s approach is doomed; once you correct for that, it is not clear that trying to construct a values-aligned singleton is the best path forward, though I’m not opposed to research along that line.

RcouF1uZ4gsC 6y ago

> Moloch should've never let open-source software happen. Maybe Moloch is catching up to open source. We read regularly of open source maintainers who are poor, burned out, and quitting. A lot of the open source work that happens is actually controlled by for profit corporations. Despite Chrome and Android being open source, it is optimized for advertising revenue for Google. Successful open source projects are incorporated in a SaaS offering, often cutting out the original developers. Other open source projects are acquired by large corporations and are shuttered. There still is not a reliable strategy for open source developers to support themselves, other than trying to use that notoriety to get hired by a big corporation, or try to upsell to non-open source add ons.