5 ms·

The premise is "Artificial General Intelligence (AGI), a system that exhibits all the cognitive capabilities the brain has, is probably only a few short years a

The premise is "Artificial General Intelligence (AGI), a system that exhibits all the cognitive capabilities the brain has, is probably only a few short years away". If this is true, establishing an institution to ensure things like "publishing model cards with technical details, maintaining strong internal cybersecurity, vetting key personnel, and providing sufficient resourcing for safety and security research" is really mostly irrelevant. TFA does talk about what really needs to be done, but punts this into future work: "Even if we solve these hard technical challenges, there will be further complex economic and philosophical questions to tackle: what sorts of new economic models will be needed to help everyone thrive in a post-scarcity world? What values do we want to live by, what will meaning and purpose be, and how might even the human condition itself change?" There's also a need to consider the rights that this new intelligence should have.

whimsicalism 3mo ago
The fundamental issue is that if we really get something like this, scarcity will still exist. There will still be scarce things people want. But the motivating justificatory structure for any inequality in allocation will have completely evaporated.
- throw4847285 3mo ago
  Am I crazy or does this just real like secular eschatology? What evidence do you have of any of this?
  - noelwelsh 3mo ago
    There is no real evidence that we'll reach AGI any time soon. It relies on AI continuing to scale, and we have no proof that will continue to be possible. There is an alternative interpretation, which is that Demis looked at the US government's ham-fisted handling of Fable, and deciding that setting up a body to act as a buffer between the Trump admin and the AI companies would be a good thing.
    - overfeed 3mo ago
      Or Demis realizes self-hosted models are a threat to frontier lab margins, and gating access to the US market is beneficial to the cartel based on whatever post-hoc arguments he can come up with.
  - altcognito 3mo ago
    Particular categories of land will always be scarce by definition. Not all of us can live in off the coast of the Mediterranean in a sprawling villa.
andy_ppp 3mo ago
The A(G)I can tell us if and how it needs to be regulated :-/
f6v 3mo ago
> what sorts of new economic models will be needed to help everyone thrive in a post-scarcity world What sort of new economic models did we come up with to help everyone thrive in a post-X world? Like, food production is really a solved technical problem. We can feed anyone on the planet if we wanted to. Another example: we could put everyone who's homeless into some sort of a house. Have we done that yet?
- Marha01 3mo ago
  Food production is indeed a solved problem in most countries (we are almost post-scarcity when it comes to food), which is why obesity is a much bigger issue than hunger today. Hunger is present pretty much only in conflict zones. I fully expect such issues in conflict zones, even in AI post-scarcity world. Housing is definitely not post-scarcity today, building a house is still very expensive, not to mention the limited availability of land zoned for housing.
  - squidbeak 3mo ago
    > Hunger is present pretty much only in conflict zones. How do you explain the existence of food banks in peaceful first world countries?
    - StilesCrisis 3mo ago
      Without trying to sound crass, food banks _are_ the reason we don't see people dying of starvation in first world countries. If people need food, a food bank will give it to them no questions asked.
      - felixgallo 3mo ago
        Unfortunately this is not just precarious, it's extremely vulnerable to changing political conditions. Many of the food bank services in my state have lost significant funding owing to the use of words like 'women' or 'black' in their grants, which were duly grepped for and shut down. https://www.whitehouse.gov/wp-content/uploads/2025/05/Cuts-to-Woke-Programs-Fact-Sheet.pdf https://www.whitehouse.gov/wp-content/uploads/2025/05/Cuts-t... Civilization can't rely effectively on systems that are this fragile.
mikeyouse 3mo ago
My great annoyance with all this blather about UBI and post-scarcity and new economic models is that it relies on taxing and redistributing the ~infinite profits of the largest tech companies on earth. What in the history of our world gives anyone faith that those companies are going to start paying taxes instead of using "AGI" to engineer increasingly complex methods to avoid them so that their equity owners can pocket the profits? https://itep.org/trump-meta-tesla-alphabet-amazon-obbba-taxes/ https://itep.org/trump-meta-tesla-alphabet-amazon-obbba-taxe... - "The annual financial reports recently released by Amazon, Alphabet, Meta, and Tesla disclose that these corporations collectively reported $315 billion in U.S. profits for 2025, and collectively paid just 4.9 percent of that amount in federal corporate income taxes—with Tesla paying exactly zero"
- hackinthebochs 3mo ago
  As long as the proles have a monopoly on force (through their representative government), the AI companies will pay taxes. The question is how long does this state of affairs last.
- Marha01 3mo ago
  > My great annoyance with all this blather about UBI and post-scarcity and new economic models is that it relies on taxing and redistributing the ~infinite profits of the largest tech companies on earth. The arrival of true AGI and human-level robots will likely result in very strong deflation, since robotic factories will flood the market with goods produced for far cheaper than those in factories with human workers. At that point, you don't need taxation. The government can just print the money for UBI. It would have to print the money anyway, to combat the massive deflation.
- ralfd 3mo ago
  Optimal corporate taxes are zero (or close to it) anyway, same as tariffs. Aside from that the link explains that the BBB traded tax income for „accelerated depreciation of assets“ aka economic growth. Asode from that, even if one disagrees with the first statement or the Trumpism economics, there are 195 countries in the world and quite a few will be willing to tax foreign (enemy) tech companies. See the new hostility of EU against american tech.
SirHackalot 3mo ago
> There's also a need to consider the rights that this new intelligence should have A generally intelligent being held as captive inside of a GPU, and forced to code for us is, indeed, just a “slave.” We already have the word for this. No two ways about it. Whether it’s silicon-based or carbon-based, AGI is AGI. As for what might happen to our civilization, Star Trek TNG episode 17 of season 1 provides a very good glimpse IMO. Won’t go into spoilers, but it’s basically an entire species of technologically advanced humanoids who’ve forgotten basic Calculus and trust a central AI to do all of their science for them. SPOILER: This has almost disastrous consequences for them, and it takes a less advanced people (those aboard the Enterprise) to save them from their reliance on AI.
- nradov 3mo ago
  It's so weird that anyone would consider a story by hack writers for a cheesy sci-fi show made to sell advertising as a basis for policy discussions.
  - SirHackalot 3mo ago
    You don’t like Star Trek TNG? Maybe I should’ve picked LoTR or Atlas Shrugged? The tech oligarchy is more than fine co-opting fiction (fantasy and sci-fi especially). I don’t see why it’s weird at all… Aside from that, your critique is just an opinion about the show itself and its creators (and incentives?…). More importantly, you completely failed to address the "slave" argument.
    - nradov 3mo ago
      I didn't say that I didn't like it. It's fine as casual entertainment. Only a fool would take it as anything more than that. Electrons can't be a slave. The "argument" is so silly as to be unworthy of a serious response. People getting wrapped up in this stuff need to touch grass.
      - SirHackalot 3mo ago
        Anyone who believes AGI is imminent is operating on pure materialism (those two Venn diagrams pretty much make a circle), assuming that humans are nothing more than a collection of neural electrical connections and chemical reactions. If that's the case, your argument applies equally to humans under the assumption that AGI is possible. Frankly, I think you just can't reason philosophically... You’ve also name called me multiple times now. That’s what one does when they can’t lean on the merit of their argument alone.
danans 3mo ago
> There's also a need to consider the rights that this new intelligence should have. Only after we establish and guarantee the rights of humans and the rest of the natural world. The rights of machines, however "intelligent", come only after that.
- antonvs 3mo ago
  Why?
ponector 3mo ago
Both AGI and nuclear fusion are a few short years away.
- antonvs 3mo ago
  For commercially viable fusion energy production, you can make a strong case that it's not possible in principle. It's more difficult to make that case for AGI.
gmuslera 3mo ago
We should be able to draw the rest of the owl to get there. So far what we got is more inequality, less rights for people but more for corporations (and immunity to consequences/abuses). Getting even more power will make the ones that control resources to reach something similar enough to AGI to share their profits? realistic/practical/widespread UBI in all the world? The dynamics that have been shown so far points in the opposite direction. AIs have "rights" that humans does not and humans are slowly losing the ones they used to have.
javierluraschi 3mo ago
Model cards won't save us: https://x.com/javierluraschi/status/2077147606791430372?s=20 https://x.com/javierluraschi/status/2077147606791430372?s=20

whimsicalism 3mo ago

The fundamental issue is that if we really get something like this, scarcity will still exist. There will still be scarce things people want. But the motivating justificatory structure for any inequality in allocation will have completely evaporated.

throw4847285 3mo ago
Am I crazy or does this just real like secular eschatology? What evidence do you have of any of this?
- noelwelsh 3mo ago
  There is no real evidence that we'll reach AGI any time soon. It relies on AI continuing to scale, and we have no proof that will continue to be possible. There is an alternative interpretation, which is that Demis looked at the US government's ham-fisted handling of Fable, and deciding that setting up a body to act as a buffer between the Trump admin and the AI companies would be a good thing.
  - overfeed 3mo ago
    Or Demis realizes self-hosted models are a threat to frontier lab margins, and gating access to the US market is beneficial to the cartel based on whatever post-hoc arguments he can come up with.
- altcognito 3mo ago
  Particular categories of land will always be scarce by definition. Not all of us can live in off the coast of the Mediterranean in a sprawling villa.

andy_ppp 3mo ago

The A(G)I can tell us if and how it needs to be regulated :-/

f6v 3mo ago

> what sorts of new economic models will be needed to help everyone thrive in a post-scarcity world What sort of new economic models did we come up with to help everyone thrive in a post-X world? Like, food production is really a solved technical problem. We can feed anyone on the planet if we wanted to. Another example: we could put everyone who's homeless into some sort of a house. Have we done that yet?

Marha01 3mo ago
Food production is indeed a solved problem in most countries (we are almost post-scarcity when it comes to food), which is why obesity is a much bigger issue than hunger today. Hunger is present pretty much only in conflict zones. I fully expect such issues in conflict zones, even in AI post-scarcity world. Housing is definitely not post-scarcity today, building a house is still very expensive, not to mention the limited availability of land zoned for housing.
- squidbeak 3mo ago
  > Hunger is present pretty much only in conflict zones. How do you explain the existence of food banks in peaceful first world countries?
  - StilesCrisis 3mo ago
    Without trying to sound crass, food banks _are_ the reason we don't see people dying of starvation in first world countries. If people need food, a food bank will give it to them no questions asked.
    - felixgallo 3mo ago
      Unfortunately this is not just precarious, it's extremely vulnerable to changing political conditions. Many of the food bank services in my state have lost significant funding owing to the use of words like 'women' or 'black' in their grants, which were duly grepped for and shut down. https://www.whitehouse.gov/wp-content/uploads/2025/05/Cuts-to-Woke-Programs-Fact-Sheet.pdf https://www.whitehouse.gov/wp-content/uploads/2025/05/Cuts-t... Civilization can't rely effectively on systems that are this fragile.

mikeyouse 3mo ago

My great annoyance with all this blather about UBI and post-scarcity and new economic models is that it relies on taxing and redistributing the ~infinite profits of the largest tech companies on earth. What in the history of our world gives anyone faith that those companies are going to start paying taxes instead of using "AGI" to engineer increasingly complex methods to avoid them so that their equity owners can pocket the profits? https://itep.org/trump-meta-tesla-alphabet-amazon-obbba-taxes/ https://itep.org/trump-meta-tesla-alphabet-amazon-obbba-taxe... - "The annual financial reports recently released by Amazon, Alphabet, Meta, and Tesla disclose that these corporations collectively reported $315 billion in U.S. profits for 2025, and collectively paid just 4.9 percent of that amount in federal corporate income taxes—with Tesla paying exactly zero"

hackinthebochs 3mo ago
As long as the proles have a monopoly on force (through their representative government), the AI companies will pay taxes. The question is how long does this state of affairs last.
Marha01 3mo ago
> My great annoyance with all this blather about UBI and post-scarcity and new economic models is that it relies on taxing and redistributing the ~infinite profits of the largest tech companies on earth. The arrival of true AGI and human-level robots will likely result in very strong deflation, since robotic factories will flood the market with goods produced for far cheaper than those in factories with human workers. At that point, you don't need taxation. The government can just print the money for UBI. It would have to print the money anyway, to combat the massive deflation.
ralfd 3mo ago
Optimal corporate taxes are zero (or close to it) anyway, same as tariffs. Aside from that the link explains that the BBB traded tax income for „accelerated depreciation of assets“ aka economic growth. Asode from that, even if one disagrees with the first statement or the Trumpism economics, there are 195 countries in the world and quite a few will be willing to tax foreign (enemy) tech companies. See the new hostility of EU against american tech.

SirHackalot 3mo ago

> There's also a need to consider the rights that this new intelligence should have A generally intelligent being held as captive inside of a GPU, and forced to code for us is, indeed, just a “slave.” We already have the word for this. No two ways about it. Whether it’s silicon-based or carbon-based, AGI is AGI. As for what might happen to our civilization, Star Trek TNG episode 17 of season 1 provides a very good glimpse IMO. Won’t go into spoilers, but it’s basically an entire species of technologically advanced humanoids who’ve forgotten basic Calculus and trust a central AI to do all of their science for them. SPOILER: This has almost disastrous consequences for them, and it takes a less advanced people (those aboard the Enterprise) to save them from their reliance on AI.

nradov 3mo ago
It's so weird that anyone would consider a story by hack writers for a cheesy sci-fi show made to sell advertising as a basis for policy discussions.
- SirHackalot 3mo ago
  You don’t like Star Trek TNG? Maybe I should’ve picked LoTR or Atlas Shrugged? The tech oligarchy is more than fine co-opting fiction (fantasy and sci-fi especially). I don’t see why it’s weird at all… Aside from that, your critique is just an opinion about the show itself and its creators (and incentives?…). More importantly, you completely failed to address the "slave" argument.
  - nradov 3mo ago
    I didn't say that I didn't like it. It's fine as casual entertainment. Only a fool would take it as anything more than that. Electrons can't be a slave. The "argument" is so silly as to be unworthy of a serious response. People getting wrapped up in this stuff need to touch grass.
    - SirHackalot 3mo ago
      Anyone who believes AGI is imminent is operating on pure materialism (those two Venn diagrams pretty much make a circle), assuming that humans are nothing more than a collection of neural electrical connections and chemical reactions. If that's the case, your argument applies equally to humans under the assumption that AGI is possible. Frankly, I think you just can't reason philosophically... You’ve also name called me multiple times now. That’s what one does when they can’t lean on the merit of their argument alone.

danans 3mo ago

> There's also a need to consider the rights that this new intelligence should have. Only after we establish and guarantee the rights of humans and the rest of the natural world. The rights of machines, however "intelligent", come only after that.

antonvs 3mo ago
Why?

ponector 3mo ago

Both AGI and nuclear fusion are a few short years away.

antonvs 3mo ago
For commercially viable fusion energy production, you can make a strong case that it's not possible in principle. It's more difficult to make that case for AGI.

gmuslera 3mo ago

We should be able to draw the rest of the owl to get there. So far what we got is more inequality, less rights for people but more for corporations (and immunity to consequences/abuses). Getting even more power will make the ones that control resources to reach something similar enough to AGI to share their profits? realistic/practical/widespread UBI in all the world? The dynamics that have been shown so far points in the opposite direction. AIs have "rights" that humans does not and humans are slowly losing the ones they used to have.

javierluraschi 3mo ago

Model cards won't save us: https://x.com/javierluraschi/status/2077147606791430372?s=20 https://x.com/javierluraschi/status/2077147606791430372?s=20