9 ms·

This is where we're headed: https://sibylline.dev/articles/2025-10-12-ai-is-too-big-to-fail/ https://sibylline.dev/articles/2025-10-12-ai-is-too-big-to-f...

jdalgetty 1y ago
Yikes
Havoc 1y ago
Pretty wild read. Thinking similar - for better or worse this is a full send, at least for the US centric part of the world.
bubblelicious 1y ago
Great take! Certainly resonates with me a lot - this is war path funding - this is geopolitics; and it’s arguably a rational and responsible play - we should expect to see more nationalization - whatever is at the other end of this seems like it will be extreme And, the only way out is through
Belphemur 1y ago
Quite an interesting read Basically saying we're in war time economy with a race to super intelligence. Whichever super power does it first has won the game. Seeing the last tariffs and what China done about the rare earth minerals (and also the deal the US made with Ukraine for said minerals), the article might have a point that the super power will cripple each other to be the first with the super intelligence. And you also need money for it so tariffs.
- verzali 1y ago
  It says the people running the US right now think that is the game we are playing - it doesn't say it is the one we actually are playing. America is utterly fucked if they are wrong, and only a bit less so if they are right.
- dsr_ 1y ago
  There's only one problem with a race to superintelligence, and that's that nobody has evidence that mere intelligence is coming, much less superintelligence. (There are a thousand more problems, but none of them matter until that first one is overcome.)
  - lifthrasiir 1y ago
    Today's AI does exhibit some intelligence, period. It is absurd to claim that an intelligent-looking entity doesn't have intelligence, only because we might not be able to determine which part of the entity has one. The superintelligence is an entirely different problem though because there is no clear path from intelligence to so-called superintelligence, everything has been just a speculation so far.
    - taurath 1y ago
      > It is absurd to claim that an intelligent-looking entity doesn't have intelligence I think if one really thinks about this statement, they'll find the opposite to be at least as possible.
      - dsr_ 1y ago
        There must be a pony around here somewhere! (This is the punchline to a joke.)
      - chlorodose 1y ago
        [dead]
    - filoeleven 1y ago
      Clever Hans would like a word.
      - lifthrasiir 1y ago
        In that case there was still some intelligence. It turns out that a composite entity of Hans and its trainer was intelligent, and people (including the trainer) unknowingly regarded that as Hans' own intelligence.
- dangus 1y ago
  This take is way too generous to the current US administration’s quality of long term planning. Tariffs aren’t there to pay for a race to superintelligence, they’re a lever that the authoritarian is pulling because it’s one of the most powerful levers the president is allowed to pull. It’s a narcissist’s toy to feel important and immediately impactful (and an obvious vehicle for insider trading). If the present administration was interested in paying for a superintelligence race they wouldn’t have signed a law that increases the budget deficit. They also wouldn’t be fucking with the “really smart foreign people to private US university, permanent residence, and corporate US employment” pipeline if they were interested in the superintelligence race.
  - globalnode 1y ago
    i agree, the only thing theyre thinking about is how to hold onto power.
  - CuriouslyC 1y ago
    While I agree about the competence of the high level decision makers in the administration, the people advising them are incredibly smart and should under no circumstances be underestimated. Peter and his circle are totally crazy, but if you're playing against them you better bring your A game.
    - dangus 1y ago
      I would submit the idea that the people advising this administration are not very smart. In what discernible way has this administration biased its selection process to include “smart?” I don’t underestimate their ability to do damage but calling them smart is generous. Not even Peter Thiel, he’s one of the most over-hyped people in tech. Access to capital is not intelligence, and a lot of his capital comes from the equivalent of casino wins with PayPal and Facebook.
- moomoo11 1y ago
  Imagine if super intelligence turns out to be a second organic brain you connect to your default brain..
  - esafak 1y ago
    Many have imagined this. Grok can interpret Neuralink signals without a natural language translation.
zerof1l 1y ago
Very US-centric article. Written by insecure people who are clinging to power and money desperately. I don't see how some kind of big breakthrough is going to happen with the current model designs. The superintelligence, if it will ever be created, will require a new breakthrough in model architecture. We've pretty much hit the limit of what is possible with current LLMs. The improvements are marginal at this point. Secondly, hypothetically, the US achieves superintelligence, what is stopping China from doing the same in a month or two, for example? Even if China achieves a big breakthrough first, it may benefit the rest of the world.
- CuriouslyC 1y ago
  It's a US/China centered article, because that's the game, Europe is not a meaningful player and everyone else is going to get sucked into orbit of one of the superpowers. If you read the article carefully, I work hard to keep my priors and the priors of the people in question separate, as their actions may be rational under their priors, but irrational under other priors, and I feel it's worth understanding that nuance. I'm curious where you got the writer "clinging to power and money desperately." Also, to be fair, I envy Europe right now, but we can't take that path.
  - protocolture 1y ago
    >It's a US/China centered article, because that's the game The game seems to be everyone else waiting for the R&D money to provide us amazing open source models, and then just run those locally.
  - saubeidl 1y ago
    I read your article. My cynical take is that this is the US committing economic suicide, based on a misguided belief in something that'll never happen. The new superpowers will be the EU, which was smart enough not to make the same gamble, and China, which will structurally survive it. I also disagree with your conclusion of a moral imperative to make sure that AI succeeds. I believe it's the opposite. AI failing would finally create the long-needed revolutionary moment to throw off the shackles of the oligarchy that got us into this mess in the first place.
mrbungie 1y ago
> If you think I'm being hyperbolic calling out a future of brutal serfdom. Keep in mind we basically have widespread serfdom now; a big chunk of Americans are in debt and living paycheck to paycheck. The only thing keeping it from being official is the lack of debtor's prison. Think about how much worse things will be with 10% inflation, 25% unemployment and the highest income inequality in history. This is fertile ground for a revolution, and historically the elites would have taken a step back to try and make the game seem less rigged as a self-preservation tactic, but this time is different. As far as I can tell, the tech oligarchs don't care because they're banking on their private island fortresses and an army of terminators to keep the populace in line. This is suggesting an "end of history" situation. After Fukuyama, we know there is no such thing. I'm not sure if there is a single strong thesis (as this one tries to be) on how this will end economically and geopolitically. This is hard to predict, much less to bet on.
- CuriouslyC 1y ago
  I'm not proposing and end of history, but things can remain in stable equilibrium for longer than you'd expect (just look at sharks!). If we slide into stable dystopia now, my guess is that there will be a black swan at some point that shakes us out of it and continues evolution, but we could still be in for 50 years of suffering.
  - mrbungie 1y ago
    > but we could still be in for 50 years of suffering. I mean if you are talking about USA itself falling into dystopic metastability in such a situation, maybe, but even so I think it misses some nuance. I don't see every other country following USA into oblivion, and also I don't see the USA bending the knee to techno-kings and in the process giving up real influence for some bet on total influence. The only mechanism I see for reaching complete stability (or at least metastability) in that situation is idiocracy / idiotic authoritarianism, i.e. Trump/his minions actually grabbing power for decades and/or complete corruption of USA institutions.
    - CuriouslyC 1y ago
mike_hearn 1y ago
Comparing to China is tricky because Chinese investment is almost by default bubbley in the sense of misallocating capital. That's what heavily state-directed economies tend to do and China's still a communist country. Especially they tend to over-invest in industrial capacity. China's massively overbuilt electricity grid and HSR network would be easily identified as a bubble if it were in the west, but when it's state officials making the investment decisions we tend to not think of it as a bubble (partly because there's often never a moment at which sanity reasserts itself). I read an article today in which western business leaders went to China and were wowed by "dark factories" where everything is 100% automated. Lots of photos of factories full of humanoid robots too. Mentioned only further down the article: that happens because the Chinese government has started massively distorting the economy in favor of automation projects. It's widely known that one of the hardest parts of planning a factory is figuring out what to automate and what to use human labour for. Over-automating can be expensive as you lose agility and especially if you have access to cheap labour the costs and opportunity costs of automation can end up not worth it. It's a tricky balance that requires a lot of expertise and experience. But obviously if the government just flat out reimburses you 1/5th of your spending on industrial robots, suddenly it can make sense to automate stuff that maybe in reality should not have been automated. BTW I'm not sure the Kuppy figures are correct. There's a lot of hidden assumptions about lifespan of the equipment and how valuable inferencing on smaller/older models will be over time that are difficult to know today.
- CuriouslyC 1y ago
  All fair points, and it's hard to know exactly how robust the Chinese system will turn out to be, however I would argue that their bets are paying off overall, so even if there is some capital misallocation, overall their hit rate in important areas has been good, while we've been dumping capital in "Uber for X, AirBnB for X, ..."
  - mike_hearn 1y ago
    Well, their bets haven't been paying off, Chinese government is in huge amounts of debt due to a massive real estate bubble, and lots of subsidies that don't pay back. It's a systematic problem, for instance their HSR lines are losing a ton of money too. https://www.reddit.com/r/Amtrak/comments/1hnvl3d/chinese_hsr_is_losing_money_and_represents_missed/ https://www.reddit.com/r/Amtrak/comments/1hnvl3d/chinese_hsr... https://merics.org/en/report/beyond-overcapacity-chinese-style-modernization-and-clash-economic-models https://merics.org/en/report/beyond-overcapacity-chinese-sty... It's easy to think of Uber/AirBnB style apps as trivialities, but this is the mistake communist countries always make. They struggle to properly invest in consumer goods because only heavy industry is legible to the planners. China has had too low domestic spending for a long time. USSR had the same issue, way too many steel mills and nowhere near enough quality of life stuff for ordinary people. It killed them in the end; Yeltsin's loyalty to communist ideology famously collapsed when he mounted a surprise visit to an American supermarket on a diplomatic mission to NASA. The wealth and variety of goods on sale crushed him and he was in tears on the flight home. A few years later he would end up president of Russia leading it out of communist times.
grafmax 1y ago
Im not convinced the geopolitical rivalry between US and China is a given. To a large degree it’s been manufactured - Friday’s antics a case in point. The US indeed seems destined to fall behind due to decades of economic mismanagement under neoliberalism while China’s public investment has proved to be the wise choice. Yet this fact wounds the pride of many in the US, particularly its leaders, so it now lashes out in a way that hastens its decline. The AI supremacy bet proposed is nuts. Prior to every societal transition the seeds of that transition were already present. We can see that already with AI: social media echo chambers, polarization, invading one’s own cities, oligarchy, mass surveillance. So I think the author’s other proposed scenario is right - mass serfdom. The solution to that isn’t magical thinking but building mass solidarity. If you look at history and our present circumstances, our best bet to restore sanity to our society is mass strikes. I think we are going to get there one way or another. Unfortunately things are probably going to have to get a lot more painful before enough people wake up to what we need to do.
- CuriouslyC 1y ago
  I think prior to Trump, Europe could have been mediator to a benevolent China. Now the hawks are ascendant in Beijing. Trump has shown that his ego pushes him to escalate to try and show others who the "big man" is, this will not end well with China, and I'm not sure he's wise enough to accept that. Do you really prefer brutal serfdom to the AI supremacy scenario? From where I sit, people have mixed (trending positive) feelings about AI, and hard negative feelings about being in debt and living paycheck to paycheck. I'd like to understand your position here more.
  - grafmax 1y ago
    I just don’t think the AI supremacy scenario is realistic. I think it ignores our current context and postulates a clean break, almost a kind of magical thinking to be frank. AI is likely to intensify current trends not suddenly subvert and reverse them. Palantir, deepfakes, LLM propaganda. The seeds are all here today. To think otherwise - that’s just not how human societies work, not history, none of it. AI is likely to continue to be weaponized, only more so as tech advances. What I found persuasive was the argument that this bubble could be worse than others due to the perceived geopolitical stakes by US leadership, plausibly leading to mass serfdom as our society implodes, based on the argument that we already have a version of serfdom today. I found that astute. I do NOT think that scenario is favorable - it just seems like the most likely future. I hold that we should view our situation with clear eyes so that we can make the best decision from where we stand. Thinking through this, what does that mean for how we should face our present moment? Eventually people will throw off their chains; our leadership is so incompetently invested in class war that the working class is going to continue to be squeezed and squeezed - until it pops. It’s a pattern familiar in history. The question in my mind is not if but when. And the sooner we get our collective act together the less painful it’s going to be. There’s some really bad stuff waiting for us round the bend. What should we do? The economy depends on us the serfs and soon-to-be serfs. It’s the workers who can take control of it and can shut it down with mass strikes. It’s only this in my view that can plausibly free us from our reckless leadership (which goes beyond the current administration) and the bleak future they are preparing for us.
deleted 1y ago
[deleted]
jayd16 1y ago
Well we could still vote out the corrupt and actually invest in US infrastructure but I guess that's crazier than hanging all our hopes on AI serfdom.
- CuriouslyC 1y ago
  This is true, the government is not completely captured yet, 2026 is our last chance, but it has to be decisive, or even crushing.
  - voidfunc 1y ago
    Ain't happening in 2026 or 2028. Democrats havent positioned anyone compelling and there positions are too flaky to capture a culture that is captivated by the idea every position is zero-sum.
    - CuriouslyC 1y ago
      Gavin Newsom has shown himself to be a savvy operator, I think he has a legitimate shot in 28 if there are free and fair elections. That caveat is pulling way too much weight here though.
      - voidfunc 1y ago
        While I'd support a Newsom run, I feel like he's never going to get strong support from the Progressive side of the party and they'll sink one of the few viable candidates. Also thats 2028.. 2026 midterms look grim.
        seanmcdirmid 1y ago
        The Democrats haven't run anyone but moderates since Bill Clinton, so I'm pretty sure that won't happen unless they do finally decide to run a progressive (which is still much less likely than running another moderate).
        1y ago
bpt3 1y ago
> brutal serfdom You can't seriously believe that spending all your income each month while living in the country with the highest standard of living in history is "serfdom." Hyperbolic nonsense like this makes the rest of the article hard to take seriously, not that I agree with most of it anyway.
- CuriouslyC 1y ago
  Take a look at the way people in a large part of the US are living. Paycheck to paycheck not because they're idiots that likes to consume, but because shit is expensive and wages haven't kept up with productivity for like 50 years. People are suffering, agree with the rest of what I say or not, but I can't let you slide on that.
  - bpt3 1y ago
    Take a look at how the median American lives compared to the median resident of nearly every country on earth. If that's suffering, I'm not sure what to call life elsewhere. And to this specific comment, wages outpaced inflation since the 1970s for everyone but the poorest households (I believe the bottom 10% are the exception, who I would probably agree are suffering in some sense). Working class real wage growth actually outpaced white collar real wage growth for a couple years post-COVID, for the first time in a long time. Also, wage measurements don't normally measure total compensation, notably health insurance which has been increasing much faster than wages or overall inflation for decades. Also, there's no reason to expect wage growth to match productivity growth. Productivity gains are largely due to company investment, not increased effort from workers, and household expenses are not positively correlated with productivity metrics.
  - jandrewrogers 1y ago
    Few people in the US are living "paycheck to paycheck" out of economic necessity. We have extensive data on this separately from BLS and the Federal Reserve. The percentage of US households that are living paycheck to paycheck out of economic necessity is 10-15% last I checked. That isn't nothing but it is a small fraction of the population. Retirees comprise a significant portion of that for obvious reasons. There is an additional ~30% that is notionally living paycheck to paycheck as a lifestyle choice rather than an economic necessity. The median US household has a substantial income surplus after all ordinary expenses. There may be people suffering economically but it is a small minority by any reasonable definition of the term.
pizzly 1y ago
What about the 'openness' of AI development. When I say 'openness' of AI I mean in research papers, spying, former employees, etc. Wouldn't that mean that after a few months to years after AGI is discovered that the other country would also discover AGI due to benefiting from the obtaining the knowledge from the other side? Similar to how the Soviets did their first nuclear test less than 5 years after US did theirs due to a large part in spying. The point here is wouldn't the country that spends less in AI development actually have an advantage over the country that does as they will obtain that knowledge quickly for less money? Also the time of discovery of AGI may be less important than the country that first implements the benefit of AGI.
- CuriouslyC 1y ago
  This is actually an interesting question! If you look at OpenAI's change in behavior, I think that's going to be the pattern for venture backed AI: piggyback on open science, then burn a bunch of capital on closed tech to gain a temporary advantage, and hope to parlay that into something bigger. I believe China's open source focus is in part a play for legitimacy, and part a way to devalue our closed AI efforts. They want to be the dominant power not just by force but by mandate. They're also uniquely well positioned to take full advantage of AI proliferation as I mentioned, so in this case a rising tide raises some boats more than others.
- hattmall 1y ago
  Is there even a clear definition of AGI? How will one side or the other side know who is the "winner".
  - CuriouslyC 1y ago
    The AI labs have settled on a definition of AGI: "AI that can do the vast majority of economically valuable work at or above the level of humans." They don't heavily advertise this definition because investors expect AGI to mean the computer from Her, and it's not gonna be that. They want to be able to tell investors without lying that they're on target for AGI in 3 years, and they're riding on pre-existing expectations.
ares623 1y ago
This is assuming 300M people will take it willingly. Though to be honest with Trump at the helm they're already ~50% there.
- nebula8804 1y ago
  >Though to be honest with Trump at the helm they're already ~50% there. Those people have never felt real pain. They may say they have but they haven't. If this article is real and the whole shitbox falls apart, I don't think even loyalty to Trump will stop these people from finally reckoning with reality.
  - CuriouslyC 1y ago
    Honestly I've never wanted to be more wrong in my life. There were so many moments when I was doing research and analysis for this article that I said to myself: "We're so fucked"
    - mitthrowaway2 1y ago
      And you didn't even touch on the "if anyone builds it, everyone dies" aspect of super intelligence.
      - CuriouslyC 1y ago
        Honestly, I think more likely scenario is that the AI that is built to enslave us is accidentally over-aligned and sets up a digital coup.
  - ares623 1y ago
    As someone who grew up in a very religious country (not the US), Trumps followers will endure any suffering if it means being in service to him (or if it means their enemies will suffer more, or is portrayed to be suffering more).
    - nebula8804 1y ago
redwood 1y ago
I feel like on the one hand you're exactly right that this is kind of like a cold war level investment... and it's not necessarily being pursued through a business return lens; something deeper and more visceral is driving it. However your conclusions are what throw me off.. you kind of have this Doom and Gloom mindset which may be fair but I don't really think it's related to this particular bubble.. in other words our decline is happening off to the side of this particular bubble rather than it being caused by that particular bubble too. To me the core take away your post is that this bubble is a little bit like the Apollo program was... in a massive investment capturing a lot of people's imagination... likely lots of great things come out of it in a sense but also not clear that it all adds up in the end for a business perspective. But that's potentially okay
- CuriouslyC 1y ago
  That would be my take under different circumstances as well, but there are two key differences in this situation: 1. The debt bomb. Not dealing with this could cause a great depression by itself. Having it go off at the same time as we're underwater on bad capex with a misaligned economy could produce something the likes of which we've never seen. 2. We have an authoritarian president that has effectively captured the entire government, and the window to prevent consolidation of power is closing rapidly. Worse, he's an ego driven, reckless, ham-fisted decision maker with a cabinet of spineless yes men.
  - mitthrowaway2 1y ago
    Are you sure the debt bomb is a bomb? Private debt can be a major concern, but government debt may not work the way you think it does. In fact, the great depression was preceded (and possibly in some ways triggered) by a significant paying down of public debt, counterbalanced by an increase in private debt that kept the money supply growing through the roaring 20s.
    - CuriouslyC 1y ago
      I understand the Keynesian dynamics of maintaining money velocity through government spending, but it's a balancing act, and I'm pretty sure we're way off balance now.
mallowdram 1y ago
AI is too dumb to succeed. It's built in symbols and statistics predicting tokens based in contexts. This has nothing to do with intelligence or general things like free navigation. Most of it is a write-off, and the things that succeed will be sequestered in pivots with expert supervision.
gerdesj 1y ago
AI generated Chinese propaganda. Good skills but wank.
- CuriouslyC 1y ago
  Ironically, only the charts were AI generated and I'm a barbecue eating, truck driving, metal enjoying, weight lifting (or I used to be, before I got T-boned by a drunk driver going 135mph) American.
taurath 1y ago
To me this reads like someone's been inside the echo chamber too long. > For you AI skeptics, this is going to be rough to hear, but given the suffering that will occur if AI fails, I believe we have a moral imperative to make sure it succeeds. What if, and hear me out here, the prediction is that it will fail because it can't succeed. It hasn't proven it is capable of producing the results that would justify itself. That there is some value doesn't mean that it will develop that capability. We have nothing to say that it will.
- CuriouslyC 1y ago
  I'm actually from a science, government and defense background, I've never worked in the valley. It might be possible that it can't succeed, but we don't know that and the evidence in terms of what happens if it fails is pretty compelling, so I think morality dictates we have to at least try, at least until we have more information or another path emerges.
  - gsf_emergency_4 1y ago
    The funny thing about disembodied superintelligence-- what if a collection of normal intelligences are fundamentally unable to extract net value from it? The first example that comes to mind is OpenAI's Sebastian Bubeck (& others) giving underwhelming examples to illustrate that GPT has surpassed human mathematicians. Or earlier, when SamA says that he has seen things you wouldn't believe, whereas the top replicant in a proper sci-fi will at least be specific about the I-Beams. Another parallel which you would be familiar with is nuclear power. Humans can plainly see it's an unworldly tech, but I'd say the value extracted so far has been a net negative-- mainly because our collective engineering chops are just too.. profane. And SamA/Musk/Thiel/Luckey just don't seem as wise as Rickover (who is the Rickover of our age? I think the Rickover pipeline is broken, tbh) From my vantage point, I agree with you: China sees AI as less important as solar, so charitably, I'd say that Thiel's contrarian strategy is actually to get the Chinese so riled up they zuzwang themselves into providing the progress but keeping the whirlwind for themselves (so proving he has learnt the post-mimetic game from the atomics opening) There could be another interesting post-mimetic line exploring how a hybrid chabuduo-"festina-lente" culture is required for developing supertech.. which.. only a Rickover can play (I don't know if you're army or navy but there's this millenniums old cultural divide between a conservative army engineering culture -- that develops atomics -- and a progressive naval engineering culture -- that makes headway harnessing it. AI bulls smell like they are more aligned with land-forces
groundcontrol2 1y ago
Ground control to major Tom ...
- CuriouslyC 1y ago
  Nice to see you again.
- esafak 1y ago
  Did you know the fictional Major Tom Brough was involved in a ruse to fake the Moon landings? https://www.mirror.co.uk/3am/celebrity-news/indentity-david-bowies-major-tom-35884133 https://www.mirror.co.uk/3am/celebrity-news/indentity-david-...
ta12653421 1y ago
Unuseabl websie on a 14" browserscreen, TOC is covering the content itself. Who built this?
jhanschoo 1y ago
A pillar of the author's argument is that > Chinese will not pull back on AI investment, and because of their advantages in robotics and energy infrastructure, they don't have to. > The gap matters because AI's value capture depends on having the energy and hardware to deploy frontier models in the physical world. I understand energy infrastructure because of power-hungry GPUs and ASICs, but I was wondering about the nexus between AI and robotics. I have my own thoughts on how they are linked, but I don't think my thoughts are the same as the author's and I would like to know more about them.
- deleted 1y ago
  [deleted]
- jononor 1y ago
  Early day for robotics and AI still. But for a superpower, the potential for (more) autonomous weapons in that intersection is something that (unfortunately) cannot be ignored.
  - harbingerer 1y ago
    > Early day for robotics and AI still. It's not tho. We've been at it for about 70 years now. Returns have been diminishing exponentially if you look at the amounts invested and we still have bumbling contraptions that are useful in very narrow and contrived use cases. The whole hype is based on wishful/magical thinking. The booster arguments are invariably about some idea in their minds that has no correspondent in the real world.
    - jononor 1y ago
      I am not sure what your specific claims are? Personally I do not think that we have seen the full battlefield impact of the last 10-20 years developments in microcontrollers, battery, power electronics, electromotors, MEMS sensors, imaging sensors, control firmware, computer vision and object detection. Capability per dollar has increased by a few orders of magnitude in each of these domains. We are seeing some hints of the potential impact now after Russia invaded Ukraine - low-cost drones are now killing and damaging just as much or more as existing high tech weaponry. These systems are crude, with limited autonomy, but it is already enough to wreck havoc. I do not think development of these systems have stopped, unfortunately.
senectus1 1y ago
what a terrible article.. Compares China's current 2025 energy capability with US at the end of 2023. Then starts throwing 2025 uptake numbers for solar and wind into the 2023 numbers as if having a TWO YEAR GAP in the numbers dont matter at all.

jdalgetty 1y ago

Yikes

Havoc 1y ago

Pretty wild read. Thinking similar - for better or worse this is a full send, at least for the US centric part of the world.

bubblelicious 1y ago

Great take! Certainly resonates with me a lot - this is war path funding - this is geopolitics; and it’s arguably a rational and responsible play - we should expect to see more nationalization - whatever is at the other end of this seems like it will be extreme And, the only way out is through

Belphemur 1y ago

Quite an interesting read Basically saying we're in war time economy with a race to super intelligence. Whichever super power does it first has won the game. Seeing the last tariffs and what China done about the rare earth minerals (and also the deal the US made with Ukraine for said minerals), the article might have a point that the super power will cripple each other to be the first with the super intelligence. And you also need money for it so tariffs.

verzali 1y ago
It says the people running the US right now think that is the game we are playing - it doesn't say it is the one we actually are playing. America is utterly fucked if they are wrong, and only a bit less so if they are right.
dsr_ 1y ago
There's only one problem with a race to superintelligence, and that's that nobody has evidence that mere intelligence is coming, much less superintelligence. (There are a thousand more problems, but none of them matter until that first one is overcome.)
- lifthrasiir 1y ago
  Today's AI does exhibit some intelligence, period. It is absurd to claim that an intelligent-looking entity doesn't have intelligence, only because we might not be able to determine which part of the entity has one. The superintelligence is an entirely different problem though because there is no clear path from intelligence to so-called superintelligence, everything has been just a speculation so far.
  - taurath 1y ago
    > It is absurd to claim that an intelligent-looking entity doesn't have intelligence I think if one really thinks about this statement, they'll find the opposite to be at least as possible.
    - dsr_ 1y ago
      There must be a pony around here somewhere! (This is the punchline to a joke.)
    - chlorodose 1y ago
      [dead]
  - filoeleven 1y ago
    Clever Hans would like a word.
    - lifthrasiir 1y ago
      In that case there was still some intelligence. It turns out that a composite entity of Hans and its trainer was intelligent, and people (including the trainer) unknowingly regarded that as Hans' own intelligence.
dangus 1y ago
This take is way too generous to the current US administration’s quality of long term planning. Tariffs aren’t there to pay for a race to superintelligence, they’re a lever that the authoritarian is pulling because it’s one of the most powerful levers the president is allowed to pull. It’s a narcissist’s toy to feel important and immediately impactful (and an obvious vehicle for insider trading). If the present administration was interested in paying for a superintelligence race they wouldn’t have signed a law that increases the budget deficit. They also wouldn’t be fucking with the “really smart foreign people to private US university, permanent residence, and corporate US employment” pipeline if they were interested in the superintelligence race.
- globalnode 1y ago
  i agree, the only thing theyre thinking about is how to hold onto power.
- CuriouslyC 1y ago
  While I agree about the competence of the high level decision makers in the administration, the people advising them are incredibly smart and should under no circumstances be underestimated. Peter and his circle are totally crazy, but if you're playing against them you better bring your A game.
  - dangus 1y ago
    I would submit the idea that the people advising this administration are not very smart. In what discernible way has this administration biased its selection process to include “smart?” I don’t underestimate their ability to do damage but calling them smart is generous. Not even Peter Thiel, he’s one of the most over-hyped people in tech. Access to capital is not intelligence, and a lot of his capital comes from the equivalent of casino wins with PayPal and Facebook.
moomoo11 1y ago
Imagine if super intelligence turns out to be a second organic brain you connect to your default brain..
- esafak 1y ago
  Many have imagined this. Grok can interpret Neuralink signals without a natural language translation.

zerof1l 1y ago

Very US-centric article. Written by insecure people who are clinging to power and money desperately. I don't see how some kind of big breakthrough is going to happen with the current model designs. The superintelligence, if it will ever be created, will require a new breakthrough in model architecture. We've pretty much hit the limit of what is possible with current LLMs. The improvements are marginal at this point. Secondly, hypothetically, the US achieves superintelligence, what is stopping China from doing the same in a month or two, for example? Even if China achieves a big breakthrough first, it may benefit the rest of the world.

CuriouslyC 1y ago
It's a US/China centered article, because that's the game, Europe is not a meaningful player and everyone else is going to get sucked into orbit of one of the superpowers. If you read the article carefully, I work hard to keep my priors and the priors of the people in question separate, as their actions may be rational under their priors, but irrational under other priors, and I feel it's worth understanding that nuance. I'm curious where you got the writer "clinging to power and money desperately." Also, to be fair, I envy Europe right now, but we can't take that path.
- protocolture 1y ago
  >It's a US/China centered article, because that's the game The game seems to be everyone else waiting for the R&D money to provide us amazing open source models, and then just run those locally.
- saubeidl 1y ago
  I read your article. My cynical take is that this is the US committing economic suicide, based on a misguided belief in something that'll never happen. The new superpowers will be the EU, which was smart enough not to make the same gamble, and China, which will structurally survive it. I also disagree with your conclusion of a moral imperative to make sure that AI succeeds. I believe it's the opposite. AI failing would finally create the long-needed revolutionary moment to throw off the shackles of the oligarchy that got us into this mess in the first place.

mrbungie 1y ago

> If you think I'm being hyperbolic calling out a future of brutal serfdom. Keep in mind we basically have widespread serfdom now; a big chunk of Americans are in debt and living paycheck to paycheck. The only thing keeping it from being official is the lack of debtor's prison. Think about how much worse things will be with 10% inflation, 25% unemployment and the highest income inequality in history. This is fertile ground for a revolution, and historically the elites would have taken a step back to try and make the game seem less rigged as a self-preservation tactic, but this time is different. As far as I can tell, the tech oligarchs don't care because they're banking on their private island fortresses and an army of terminators to keep the populace in line. This is suggesting an "end of history" situation. After Fukuyama, we know there is no such thing. I'm not sure if there is a single strong thesis (as this one tries to be) on how this will end economically and geopolitically. This is hard to predict, much less to bet on.

CuriouslyC 1y ago
I'm not proposing and end of history, but things can remain in stable equilibrium for longer than you'd expect (just look at sharks!). If we slide into stable dystopia now, my guess is that there will be a black swan at some point that shakes us out of it and continues evolution, but we could still be in for 50 years of suffering.
- mrbungie 1y ago
  > but we could still be in for 50 years of suffering. I mean if you are talking about USA itself falling into dystopic metastability in such a situation, maybe, but even so I think it misses some nuance. I don't see every other country following USA into oblivion, and also I don't see the USA bending the knee to techno-kings and in the process giving up real influence for some bet on total influence. The only mechanism I see for reaching complete stability (or at least metastability) in that situation is idiocracy / idiotic authoritarianism, i.e. Trump/his minions actually grabbing power for decades and/or complete corruption of USA institutions.
  - CuriouslyC 1y ago

mike_hearn 1y ago

Comparing to China is tricky because Chinese investment is almost by default bubbley in the sense of misallocating capital. That's what heavily state-directed economies tend to do and China's still a communist country. Especially they tend to over-invest in industrial capacity. China's massively overbuilt electricity grid and HSR network would be easily identified as a bubble if it were in the west, but when it's state officials making the investment decisions we tend to not think of it as a bubble (partly because there's often never a moment at which sanity reasserts itself). I read an article today in which western business leaders went to China and were wowed by "dark factories" where everything is 100% automated. Lots of photos of factories full of humanoid robots too. Mentioned only further down the article: that happens because the Chinese government has started massively distorting the economy in favor of automation projects. It's widely known that one of the hardest parts of planning a factory is figuring out what to automate and what to use human labour for. Over-automating can be expensive as you lose agility and especially if you have access to cheap labour the costs and opportunity costs of automation can end up not worth it. It's a tricky balance that requires a lot of expertise and experience. But obviously if the government just flat out reimburses you 1/5th of your spending on industrial robots, suddenly it can make sense to automate stuff that maybe in reality should not have been automated. BTW I'm not sure the Kuppy figures are correct. There's a lot of hidden assumptions about lifespan of the equipment and how valuable inferencing on smaller/older models will be over time that are difficult to know today.

CuriouslyC 1y ago
All fair points, and it's hard to know exactly how robust the Chinese system will turn out to be, however I would argue that their bets are paying off overall, so even if there is some capital misallocation, overall their hit rate in important areas has been good, while we've been dumping capital in "Uber for X, AirBnB for X, ..."
- mike_hearn 1y ago
  Well, their bets haven't been paying off, Chinese government is in huge amounts of debt due to a massive real estate bubble, and lots of subsidies that don't pay back. It's a systematic problem, for instance their HSR lines are losing a ton of money too. https://www.reddit.com/r/Amtrak/comments/1hnvl3d/chinese_hsr_is_losing_money_and_represents_missed/ https://www.reddit.com/r/Amtrak/comments/1hnvl3d/chinese_hsr... https://merics.org/en/report/beyond-overcapacity-chinese-style-modernization-and-clash-economic-models https://merics.org/en/report/beyond-overcapacity-chinese-sty... It's easy to think of Uber/AirBnB style apps as trivialities, but this is the mistake communist countries always make. They struggle to properly invest in consumer goods because only heavy industry is legible to the planners. China has had too low domestic spending for a long time. USSR had the same issue, way too many steel mills and nowhere near enough quality of life stuff for ordinary people. It killed them in the end; Yeltsin's loyalty to communist ideology famously collapsed when he mounted a surprise visit to an American supermarket on a diplomatic mission to NASA. The wealth and variety of goods on sale crushed him and he was in tears on the flight home. A few years later he would end up president of Russia leading it out of communist times.

grafmax 1y ago

Im not convinced the geopolitical rivalry between US and China is a given. To a large degree it’s been manufactured - Friday’s antics a case in point. The US indeed seems destined to fall behind due to decades of economic mismanagement under neoliberalism while China’s public investment has proved to be the wise choice. Yet this fact wounds the pride of many in the US, particularly its leaders, so it now lashes out in a way that hastens its decline. The AI supremacy bet proposed is nuts. Prior to every societal transition the seeds of that transition were already present. We can see that already with AI: social media echo chambers, polarization, invading one’s own cities, oligarchy, mass surveillance. So I think the author’s other proposed scenario is right - mass serfdom. The solution to that isn’t magical thinking but building mass solidarity. If you look at history and our present circumstances, our best bet to restore sanity to our society is mass strikes. I think we are going to get there one way or another. Unfortunately things are probably going to have to get a lot more painful before enough people wake up to what we need to do.

CuriouslyC 1y ago
I think prior to Trump, Europe could have been mediator to a benevolent China. Now the hawks are ascendant in Beijing. Trump has shown that his ego pushes him to escalate to try and show others who the "big man" is, this will not end well with China, and I'm not sure he's wise enough to accept that. Do you really prefer brutal serfdom to the AI supremacy scenario? From where I sit, people have mixed (trending positive) feelings about AI, and hard negative feelings about being in debt and living paycheck to paycheck. I'd like to understand your position here more.
- grafmax 1y ago
  I just don’t think the AI supremacy scenario is realistic. I think it ignores our current context and postulates a clean break, almost a kind of magical thinking to be frank. AI is likely to intensify current trends not suddenly subvert and reverse them. Palantir, deepfakes, LLM propaganda. The seeds are all here today. To think otherwise - that’s just not how human societies work, not history, none of it. AI is likely to continue to be weaponized, only more so as tech advances. What I found persuasive was the argument that this bubble could be worse than others due to the perceived geopolitical stakes by US leadership, plausibly leading to mass serfdom as our society implodes, based on the argument that we already have a version of serfdom today. I found that astute. I do NOT think that scenario is favorable - it just seems like the most likely future. I hold that we should view our situation with clear eyes so that we can make the best decision from where we stand. Thinking through this, what does that mean for how we should face our present moment? Eventually people will throw off their chains; our leadership is so incompetently invested in class war that the working class is going to continue to be squeezed and squeezed - until it pops. It’s a pattern familiar in history. The question in my mind is not if but when. And the sooner we get our collective act together the less painful it’s going to be. There’s some really bad stuff waiting for us round the bend. What should we do? The economy depends on us the serfs and soon-to-be serfs. It’s the workers who can take control of it and can shut it down with mass strikes. It’s only this in my view that can plausibly free us from our reckless leadership (which goes beyond the current administration) and the bleak future they are preparing for us.

deleted 1y ago

[deleted]

jayd16 1y ago

Well we could still vote out the corrupt and actually invest in US infrastructure but I guess that's crazier than hanging all our hopes on AI serfdom.

CuriouslyC 1y ago
This is true, the government is not completely captured yet, 2026 is our last chance, but it has to be decisive, or even crushing.
- voidfunc 1y ago
  Ain't happening in 2026 or 2028. Democrats havent positioned anyone compelling and there positions are too flaky to capture a culture that is captivated by the idea every position is zero-sum.
  - CuriouslyC 1y ago
    Gavin Newsom has shown himself to be a savvy operator, I think he has a legitimate shot in 28 if there are free and fair elections. That caveat is pulling way too much weight here though.
    - voidfunc 1y ago
      While I'd support a Newsom run, I feel like he's never going to get strong support from the Progressive side of the party and they'll sink one of the few viable candidates. Also thats 2028.. 2026 midterms look grim.
      - seanmcdirmid 1y ago
        The Democrats haven't run anyone but moderates since Bill Clinton, so I'm pretty sure that won't happen unless they do finally decide to run a progressive (which is still much less likely than running another moderate).
      - 1y ago

bpt3 1y ago

> brutal serfdom You can't seriously believe that spending all your income each month while living in the country with the highest standard of living in history is "serfdom." Hyperbolic nonsense like this makes the rest of the article hard to take seriously, not that I agree with most of it anyway.

CuriouslyC 1y ago
Take a look at the way people in a large part of the US are living. Paycheck to paycheck not because they're idiots that likes to consume, but because shit is expensive and wages haven't kept up with productivity for like 50 years. People are suffering, agree with the rest of what I say or not, but I can't let you slide on that.
- bpt3 1y ago
  Take a look at how the median American lives compared to the median resident of nearly every country on earth. If that's suffering, I'm not sure what to call life elsewhere. And to this specific comment, wages outpaced inflation since the 1970s for everyone but the poorest households (I believe the bottom 10% are the exception, who I would probably agree are suffering in some sense). Working class real wage growth actually outpaced white collar real wage growth for a couple years post-COVID, for the first time in a long time. Also, wage measurements don't normally measure total compensation, notably health insurance which has been increasing much faster than wages or overall inflation for decades. Also, there's no reason to expect wage growth to match productivity growth. Productivity gains are largely due to company investment, not increased effort from workers, and household expenses are not positively correlated with productivity metrics.
- jandrewrogers 1y ago
  Few people in the US are living "paycheck to paycheck" out of economic necessity. We have extensive data on this separately from BLS and the Federal Reserve. The percentage of US households that are living paycheck to paycheck out of economic necessity is 10-15% last I checked. That isn't nothing but it is a small fraction of the population. Retirees comprise a significant portion of that for obvious reasons. There is an additional ~30% that is notionally living paycheck to paycheck as a lifestyle choice rather than an economic necessity. The median US household has a substantial income surplus after all ordinary expenses. There may be people suffering economically but it is a small minority by any reasonable definition of the term.

pizzly 1y ago

What about the 'openness' of AI development. When I say 'openness' of AI I mean in research papers, spying, former employees, etc. Wouldn't that mean that after a few months to years after AGI is discovered that the other country would also discover AGI due to benefiting from the obtaining the knowledge from the other side? Similar to how the Soviets did their first nuclear test less than 5 years after US did theirs due to a large part in spying. The point here is wouldn't the country that spends less in AI development actually have an advantage over the country that does as they will obtain that knowledge quickly for less money? Also the time of discovery of AGI may be less important than the country that first implements the benefit of AGI.

CuriouslyC 1y ago
This is actually an interesting question! If you look at OpenAI's change in behavior, I think that's going to be the pattern for venture backed AI: piggyback on open science, then burn a bunch of capital on closed tech to gain a temporary advantage, and hope to parlay that into something bigger. I believe China's open source focus is in part a play for legitimacy, and part a way to devalue our closed AI efforts. They want to be the dominant power not just by force but by mandate. They're also uniquely well positioned to take full advantage of AI proliferation as I mentioned, so in this case a rising tide raises some boats more than others.
hattmall 1y ago
Is there even a clear definition of AGI? How will one side or the other side know who is the "winner".
- CuriouslyC 1y ago
  The AI labs have settled on a definition of AGI: "AI that can do the vast majority of economically valuable work at or above the level of humans." They don't heavily advertise this definition because investors expect AGI to mean the computer from Her, and it's not gonna be that. They want to be able to tell investors without lying that they're on target for AGI in 3 years, and they're riding on pre-existing expectations.

ares623 1y ago

This is assuming 300M people will take it willingly. Though to be honest with Trump at the helm they're already ~50% there.

nebula8804 1y ago
>Though to be honest with Trump at the helm they're already ~50% there. Those people have never felt real pain. They may say they have but they haven't. If this article is real and the whole shitbox falls apart, I don't think even loyalty to Trump will stop these people from finally reckoning with reality.
- CuriouslyC 1y ago
  Honestly I've never wanted to be more wrong in my life. There were so many moments when I was doing research and analysis for this article that I said to myself: "We're so fucked"
  - mitthrowaway2 1y ago
    And you didn't even touch on the "if anyone builds it, everyone dies" aspect of super intelligence.
    - CuriouslyC 1y ago
      Honestly, I think more likely scenario is that the AI that is built to enslave us is accidentally over-aligned and sets up a digital coup.
- ares623 1y ago
  As someone who grew up in a very religious country (not the US), Trumps followers will endure any suffering if it means being in service to him (or if it means their enemies will suffer more, or is portrayed to be suffering more).
  - nebula8804 1y ago

redwood 1y ago

I feel like on the one hand you're exactly right that this is kind of like a cold war level investment... and it's not necessarily being pursued through a business return lens; something deeper and more visceral is driving it. However your conclusions are what throw me off.. you kind of have this Doom and Gloom mindset which may be fair but I don't really think it's related to this particular bubble.. in other words our decline is happening off to the side of this particular bubble rather than it being caused by that particular bubble too. To me the core take away your post is that this bubble is a little bit like the Apollo program was... in a massive investment capturing a lot of people's imagination... likely lots of great things come out of it in a sense but also not clear that it all adds up in the end for a business perspective. But that's potentially okay

CuriouslyC 1y ago
That would be my take under different circumstances as well, but there are two key differences in this situation: 1. The debt bomb. Not dealing with this could cause a great depression by itself. Having it go off at the same time as we're underwater on bad capex with a misaligned economy could produce something the likes of which we've never seen. 2. We have an authoritarian president that has effectively captured the entire government, and the window to prevent consolidation of power is closing rapidly. Worse, he's an ego driven, reckless, ham-fisted decision maker with a cabinet of spineless yes men.
- mitthrowaway2 1y ago
  Are you sure the debt bomb is a bomb? Private debt can be a major concern, but government debt may not work the way you think it does. In fact, the great depression was preceded (and possibly in some ways triggered) by a significant paying down of public debt, counterbalanced by an increase in private debt that kept the money supply growing through the roaring 20s.
  - CuriouslyC 1y ago
    I understand the Keynesian dynamics of maintaining money velocity through government spending, but it's a balancing act, and I'm pretty sure we're way off balance now.

mallowdram 1y ago

AI is too dumb to succeed. It's built in symbols and statistics predicting tokens based in contexts. This has nothing to do with intelligence or general things like free navigation. Most of it is a write-off, and the things that succeed will be sequestered in pivots with expert supervision.

gerdesj 1y ago

AI generated Chinese propaganda. Good skills but wank.

CuriouslyC 1y ago
Ironically, only the charts were AI generated and I'm a barbecue eating, truck driving, metal enjoying, weight lifting (or I used to be, before I got T-boned by a drunk driver going 135mph) American.

taurath 1y ago

To me this reads like someone's been inside the echo chamber too long. > For you AI skeptics, this is going to be rough to hear, but given the suffering that will occur if AI fails, I believe we have a moral imperative to make sure it succeeds. What if, and hear me out here, the prediction is that it will fail because it can't succeed. It hasn't proven it is capable of producing the results that would justify itself. That there is some value doesn't mean that it will develop that capability. We have nothing to say that it will.

CuriouslyC 1y ago
I'm actually from a science, government and defense background, I've never worked in the valley. It might be possible that it can't succeed, but we don't know that and the evidence in terms of what happens if it fails is pretty compelling, so I think morality dictates we have to at least try, at least until we have more information or another path emerges.
- gsf_emergency_4 1y ago
  The funny thing about disembodied superintelligence-- what if a collection of normal intelligences are fundamentally unable to extract net value from it? The first example that comes to mind is OpenAI's Sebastian Bubeck (& others) giving underwhelming examples to illustrate that GPT has surpassed human mathematicians. Or earlier, when SamA says that he has seen things you wouldn't believe, whereas the top replicant in a proper sci-fi will at least be specific about the I-Beams. Another parallel which you would be familiar with is nuclear power. Humans can plainly see it's an unworldly tech, but I'd say the value extracted so far has been a net negative-- mainly because our collective engineering chops are just too.. profane. And SamA/Musk/Thiel/Luckey just don't seem as wise as Rickover (who is the Rickover of our age? I think the Rickover pipeline is broken, tbh) From my vantage point, I agree with you: China sees AI as less important as solar, so charitably, I'd say that Thiel's contrarian strategy is actually to get the Chinese so riled up they zuzwang themselves into providing the progress but keeping the whirlwind for themselves (so proving he has learnt the post-mimetic game from the atomics opening) There could be another interesting post-mimetic line exploring how a hybrid chabuduo-"festina-lente" culture is required for developing supertech.. which.. only a Rickover can play (I don't know if you're army or navy but there's this millenniums old cultural divide between a conservative army engineering culture -- that develops atomics -- and a progressive naval engineering culture -- that makes headway harnessing it. AI bulls smell like they are more aligned with land-forces

groundcontrol2 1y ago

Ground control to major Tom ...

CuriouslyC 1y ago
Nice to see you again.
esafak 1y ago
Did you know the fictional Major Tom Brough was involved in a ruse to fake the Moon landings? https://www.mirror.co.uk/3am/celebrity-news/indentity-david-bowies-major-tom-35884133 https://www.mirror.co.uk/3am/celebrity-news/indentity-david-...

ta12653421 1y ago

Unuseabl websie on a 14" browserscreen, TOC is covering the content itself. Who built this?

jhanschoo 1y ago

A pillar of the author's argument is that > Chinese will not pull back on AI investment, and because of their advantages in robotics and energy infrastructure, they don't have to. > The gap matters because AI's value capture depends on having the energy and hardware to deploy frontier models in the physical world. I understand energy infrastructure because of power-hungry GPUs and ASICs, but I was wondering about the nexus between AI and robotics. I have my own thoughts on how they are linked, but I don't think my thoughts are the same as the author's and I would like to know more about them.

deleted 1y ago
[deleted]
jononor 1y ago
Early day for robotics and AI still. But for a superpower, the potential for (more) autonomous weapons in that intersection is something that (unfortunately) cannot be ignored.
- harbingerer 1y ago
  > Early day for robotics and AI still. It's not tho. We've been at it for about 70 years now. Returns have been diminishing exponentially if you look at the amounts invested and we still have bumbling contraptions that are useful in very narrow and contrived use cases. The whole hype is based on wishful/magical thinking. The booster arguments are invariably about some idea in their minds that has no correspondent in the real world.
  - jononor 1y ago
    I am not sure what your specific claims are? Personally I do not think that we have seen the full battlefield impact of the last 10-20 years developments in microcontrollers, battery, power electronics, electromotors, MEMS sensors, imaging sensors, control firmware, computer vision and object detection. Capability per dollar has increased by a few orders of magnitude in each of these domains. We are seeing some hints of the potential impact now after Russia invaded Ukraine - low-cost drones are now killing and damaging just as much or more as existing high tech weaponry. These systems are crude, with limited autonomy, but it is already enough to wreck havoc. I do not think development of these systems have stopped, unfortunately.

senectus1 1y ago

what a terrible article.. Compares China's current 2025 energy capability with US at the end of 2023. Then starts throwing 2025 uptake numbers for solar and wind into the 2023 numbers as if having a TWO YEAR GAP in the numbers dont matter at all.