3 ms·

When we step back and realize that corporations are already a type of collective superintelligence, it’s pretty clear to me that almost all of the things you’ve

When we step back and realize that corporations are already a type of collective superintelligence, it’s pretty clear to me that almost all of the things you’ve mentioned are in the process of taking place now. Corporations are this kind of alien entity that only cares about maximizing profits, and if that means poisoning people or undermining democracy, too bad. They are the paperclip maximizers the AI safety folks are concerned about. If they were human we would call them sociopaths, but unfortunately they are much more powerful. When we talk about “AI” in particular we can almost always reframe it as automation. Chess bots are automated chess players. LLMs are automated text generators. Autonomous cars are automated drivers. Autonomous weapons are automated soldiers. What does automation mean? It means that control of the system is concentrated in a very small group of people. We should ask what happens if the levers of control of many of our systems shrinks: it’s probably not good for democracy if many of the constituents aren’t necessary. > entire cities will be filled with humanoid robots Unfortunately I think the future may look much more boring. You don’t need robot cops if you have cameras. You don’t need mind control if you have algorithmically driven feeds.

atleastoptimal 2y ago
The issue I have is convergence vs divergence. When a new modality emerges, a normally convergent trend can be disrupted when a new divergent utility frontier emerges Here are a few examples: the rate of deaths in wars was roughly stable for hundreds of years until WW1 and WW2, due to the advent of tech so new that it upset many of the stable equilibria that minimized the excess death of wars. The rate of transportation deaths was very small all throughout history until the advent of the car. Even now, 130 years after its invention 1/100 deaths in the US are due to traffic fatalities. My general point is, the vast superintelligence of intra and international systems tends towards a banal equilibrium UNTIL a massively disruptive technology is introduced which massively incentivizes one side. The level of potential hazard and collateral damage is proportional to the benefit offered to the winning party. The moment the US invented the nuclear bomb, they used it on citizens, who all died solely because their deaths weren't quite 100% orthogonal to the US' interests, and the nuke has only not been used since because the equilibrium was balanced due to the technology's proliferation to opposing sides faster than it became necessary again.
- janalsncm 2y ago
  I don’t think that corporate power is in any kind of equilibrium, banal or otherwise, with non-corporate power. They are at odds, and for the last few decades due to a fairly stable international order and globalized trade corporations have grown more powerful than ever before. We think of Apple and Google as “American” companies but in reality they were just started here and have no real allegiance to anything but profits.

atleastoptimal 2y ago

The issue I have is convergence vs divergence. When a new modality emerges, a normally convergent trend can be disrupted when a new divergent utility frontier emerges Here are a few examples: the rate of deaths in wars was roughly stable for hundreds of years until WW1 and WW2, due to the advent of tech so new that it upset many of the stable equilibria that minimized the excess death of wars. The rate of transportation deaths was very small all throughout history until the advent of the car. Even now, 130 years after its invention 1/100 deaths in the US are due to traffic fatalities. My general point is, the vast superintelligence of intra and international systems tends towards a banal equilibrium UNTIL a massively disruptive technology is introduced which massively incentivizes one side. The level of potential hazard and collateral damage is proportional to the benefit offered to the winning party. The moment the US invented the nuclear bomb, they used it on citizens, who all died solely because their deaths weren't quite 100% orthogonal to the US' interests, and the nuke has only not been used since because the equilibrium was balanced due to the technology's proliferation to opposing sides faster than it became necessary again.

janalsncm 2y ago
I don’t think that corporate power is in any kind of equilibrium, banal or otherwise, with non-corporate power. They are at odds, and for the last few decades due to a fairly stable international order and globalized trade corporations have grown more powerful than ever before. We think of Apple and Google as “American” companies but in reality they were just started here and have no real allegiance to anything but profits.