4 ms·

> Once AI has begun to displace humans, existing feedback mechanisms that encourage human influence and flourishing will begin to break down. For example, state

> Once AI has begun to displace humans, existing feedback mechanisms that encourage human influence and flourishing will begin to break down. For example, states funded mainly by taxes on AI profits instead of their citizens' labor will have little incentive to ensure citizens' representation. This could occur at the same time as AI provides states with unprecedented influence over human culture and behavior, which might make coordination amongst humans more difficult, thereby further reducing humans' ability to resist such pressures. We describe these and other mechanisms and feedback loops in more detail in this work. States like the USA already have little incentive to represent their citizens. This has been studied. Despite most of them being workers. They represent the rich instead, who just have their asses sat on assets. This has been an issue for over a hundred years. So there is plenty of content (like what the AI likes) to pull from. AI that is not embodied in the world can simply be unplugged. Straightforward if we all own technology collectively. But massively complicated when you have capitalists who have every incentive to replace all human labor. (And replacing all human labor is only a problem because a tiny minority would end up dominating everyone else.) The authors seem more concerned with that hypothetical AI that would consume the universe on a directive to produce stamps (or whatever it was). Instead they could focus on the same issue that they are ostensibly concerned about but face it much more directly.

duvenaud 2y ago
Last author here. I agree that states already have little incentive to effectively represent their citizens. But they could have even less! What would it look like to face these issues but more directly? Ending capitalism and competition?
- keybored 2y ago
  > But they could have even less! Driven by the most direct, tangible cause of the capitalists pursuing their own narrow interests. And AI fits in there as well ($500B funding to AI says Trump). > What would it look like to face these issues but more directly? Socialism. It doesn’t matter that jobs are automated away under socialism since there is no capital/worker social relation. If jobs are automated everyone just works less. > Ending capitalism and competition? You put those two together for some reason. Capitalism and the state sector have lead to amazing improvements in the productive capacity of society overall. A lack of, at least from our First World perspective at least, doesn’t seem to be an issue. Instead the problem is (1) unsustainable growth (climate change) and (2) directing the productive capacity towards pro-social goals. So yes a change is long overdue. Productive competition happens under capitalism. And sometimes it doesn’t. There’s plenty of accusations of Big Tech being anti-competitive on this site.
  - duvenaud 2y ago
    I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elaborate what you think I, or anyone should do or advocate for more concretely? The reason I mentioned competition is that, even under a complete command economy, there is still internal competition for control, which I think would still lead to human disempowerment eventually for similar reasons. Though probably on a longer time scale, which might still be a win. The only way to avoid being outcompeted is to have a total state ruled by something sufficiently agentic to resist all attempts to even adjust its policies. Which sounds terrifying and hard to get right on the first try.
    - keybored 2y ago
      > I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elaborate what you think I, or anyone should do or advocate for more concretely? A typical/classic way to build socialism is to organize the working class.
smackeyacky 2y ago
If all labour is displaced, where will our capitalist overloads find more income? Can’t be government because we’re being taxed now, but won’t be once unemployed. We need jobs so we can continue to be fleeced.
- acatnamedjoe 2y ago
  That's only because a capitalist economy uses the circulation of currency and goods as a way to multiply wealth, while motivating the people who generate the wealth. Sufficiently advanced AI offers the potential for exponential wealth generation for our (former) capitalist overlords, without you or I needing to produce or consume anything.
  - keybored 2y ago
    I think that was a sarcastic jab at consumer capitalism.
acatnamedjoe 2y ago
Focusing on the state misses the point. The state is a relatively modern abstraction for society. Society, state-based or not, has always been dominated by class of elites and governed primarily in their interests. However, for all of human history those elites have needed workers, and in complex societies, they need LOADS of them. The elites have always needed to ensure that the working people are sufficiently fit, healthy, motivated and skilled to do the work required. For the last 500 years or so the elites have also found it convenient to maintain a mass of relatively affluent people with a reasonable amount of leisure time, who will purchase the products that make them rich. Thus the typical person in the world today finds themself able to exchange their labour for basic necessities and increasingly, consumer goods. Most people receive some form of protection from bodily violence and for their property - whether from the state or from some other arrangement with the elite class. Most people also have access to some form of education and healthcare (although of course the level of provision varies massively). Most people have some amount of leisure time, some level of autonomy over what they do with that time, and an increasing range of options for leisure activities. All of this happens because it is convenient for elites - it gets them what they want. AI presents us with a possible future where a small group of elites could generate infinite wealth, and would have absolutely no need of the working and middle classes. The benefits we currently enjoy (however meagre) would dry up. At best, we'd be ignored and left to scratch a subsistence living out of whatever is left of our natural environment by that point. At worst, one could imagine a scenario where AI-wielding elites compete against each other, and need access to as many natural resources as possible to stay competitive. Then you'd suspect we wouldn't just be ignored, our very existence would be an opportunity cost for the elite class. e.g. It's 2056 and Musk needs every square meter of solar panels he can get to ensure his AI army triumphs over Zuckerberg's. The plot of land where you've been quietly growing your potatoes and trying to stop your children dying of cholera doesn't get much sun, but it gets a bit - and that's more than enough for him to have you murdered (or, if he's feeling merciful, evicted to die of starvation).
- keybored 2y ago
  I don’t buy your history but we seem to agree on the conclusion. The “thus” is misplaced. Nothing was given from the elites. In two senses of the word: labor created that standard of living, elites took a lot of it, and then labor forced them to give a bit more of if back. And labor has always created that value. And the future when labor is displaced? Does the fully automatically manufactured “largesse” of the elites dry up because the elites made it and they don’t have to give it to anyone else? No to the first part, yes to the second. Labor first created the value. Then the automation. Then they let the elite steal it wholesale. So discussing the elites as having inherently something to give away is misplaced.

duvenaud 2y ago

Last author here. I agree that states already have little incentive to effectively represent their citizens. But they could have even less! What would it look like to face these issues but more directly? Ending capitalism and competition?

keybored 2y ago
> But they could have even less! Driven by the most direct, tangible cause of the capitalists pursuing their own narrow interests. And AI fits in there as well ($500B funding to AI says Trump). > What would it look like to face these issues but more directly? Socialism. It doesn’t matter that jobs are automated away under socialism since there is no capital/worker social relation. If jobs are automated everyone just works less. > Ending capitalism and competition? You put those two together for some reason. Capitalism and the state sector have lead to amazing improvements in the productive capacity of society overall. A lack of, at least from our First World perspective at least, doesn’t seem to be an issue. Instead the problem is (1) unsustainable growth (climate change) and (2) directing the productive capacity towards pro-social goals. So yes a change is long overdue. Productive competition happens under capitalism. And sometimes it doesn’t. There’s plenty of accusations of Big Tech being anti-competitive on this site.
- duvenaud 2y ago
  I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elaborate what you think I, or anyone should do or advocate for more concretely? The reason I mentioned competition is that, even under a complete command economy, there is still internal competition for control, which I think would still lead to human disempowerment eventually for similar reasons. Though probably on a longer time scale, which might still be a win. The only way to avoid being outcompeted is to have a total state ruled by something sufficiently agentic to resist all attempts to even adjust its policies. Which sounds terrifying and hard to get right on the first try.
  - keybored 2y ago
    > I appreciate your engagement, and I don't really have a plan myself to address these problems, but I don't really know what to do with "Socialism" as a recommendation. Care to elaborate what you think I, or anyone should do or advocate for more concretely? A typical/classic way to build socialism is to organize the working class.

smackeyacky 2y ago

If all labour is displaced, where will our capitalist overloads find more income? Can’t be government because we’re being taxed now, but won’t be once unemployed. We need jobs so we can continue to be fleeced.

acatnamedjoe 2y ago
That's only because a capitalist economy uses the circulation of currency and goods as a way to multiply wealth, while motivating the people who generate the wealth. Sufficiently advanced AI offers the potential for exponential wealth generation for our (former) capitalist overlords, without you or I needing to produce or consume anything.
- keybored 2y ago
  I think that was a sarcastic jab at consumer capitalism.

acatnamedjoe 2y ago

Focusing on the state misses the point. The state is a relatively modern abstraction for society. Society, state-based or not, has always been dominated by class of elites and governed primarily in their interests. However, for all of human history those elites have needed workers, and in complex societies, they need LOADS of them. The elites have always needed to ensure that the working people are sufficiently fit, healthy, motivated and skilled to do the work required. For the last 500 years or so the elites have also found it convenient to maintain a mass of relatively affluent people with a reasonable amount of leisure time, who will purchase the products that make them rich. Thus the typical person in the world today finds themself able to exchange their labour for basic necessities and increasingly, consumer goods. Most people receive some form of protection from bodily violence and for their property - whether from the state or from some other arrangement with the elite class. Most people also have access to some form of education and healthcare (although of course the level of provision varies massively). Most people have some amount of leisure time, some level of autonomy over what they do with that time, and an increasing range of options for leisure activities. All of this happens because it is convenient for elites - it gets them what they want. AI presents us with a possible future where a small group of elites could generate infinite wealth, and would have absolutely no need of the working and middle classes. The benefits we currently enjoy (however meagre) would dry up. At best, we'd be ignored and left to scratch a subsistence living out of whatever is left of our natural environment by that point. At worst, one could imagine a scenario where AI-wielding elites compete against each other, and need access to as many natural resources as possible to stay competitive. Then you'd suspect we wouldn't just be ignored, our very existence would be an opportunity cost for the elite class. e.g. It's 2056 and Musk needs every square meter of solar panels he can get to ensure his AI army triumphs over Zuckerberg's. The plot of land where you've been quietly growing your potatoes and trying to stop your children dying of cholera doesn't get much sun, but it gets a bit - and that's more than enough for him to have you murdered (or, if he's feeling merciful, evicted to die of starvation).

keybored 2y ago
I don’t buy your history but we seem to agree on the conclusion. The “thus” is misplaced. Nothing was given from the elites. In two senses of the word: labor created that standard of living, elites took a lot of it, and then labor forced them to give a bit more of if back. And labor has always created that value. And the future when labor is displaced? Does the fully automatically manufactured “largesse” of the elites dry up because the elites made it and they don’t have to give it to anyone else? No to the first part, yes to the second. Labor first created the value. Then the automation. Then they let the elite steal it wholesale. So discussing the elites as having inherently something to give away is misplaced.