15 ms·

AI 2040: Plan A

• ChrisArchitect 3mo ago
  Associated post: Introducing Plan A https://www.astralcodexten.com/p/introducing-plan-a https://www.astralcodexten.com/p/introducing-plan-a
• cyberpunk 3mo ago
  > Then, in the mid-2030s, they pause at AIs around the level of top human geniuses. They being the US and China and by agreement. It would be ideal, but there’s far too much money on the table to overcome human nature. So my hope is we hit some kind of limits naturally.. Wishful thinking?
• tfirst 3mo ago
  If carbon taxes are already a lethal policy for an political campaign, it's absurd to think that fears of ASI will create any real movement around pausing AI. If there is any movement to pause AI development, it will come from the general public's dislike of these companies. Not from the AI safety angle.
  • arjie 3mo ago
    If that is true and one cares about a moratorium on progress in the US then it seems like the number one way is to meet people where they are: so water use misinformation, degrowth, power supply constraints. That does place all the people who push for these things in a different light. They may well be attempting to do what the AI safety labs are ostensibly trying to do. As an AI safetyist, one’s closest ally (in a distributed coordinated way) is the populist misinformer. Fascinating.
    • tfirst 3mo ago
      If there's going to be any pause, I'm sure it will come from a populist movement. I just can't imagine misplaced worries about AI water use will translate into the kinds of policy the authors want to see.
      • arjie 3mo ago
        Yeah it’s like shoving the top of a double pendulum. You will get some movement in one direction, but where it will precisely land is hard to predict. The water-use argument is already earning refinement by differentiating “AI datacenters” from “normal datacenters” in an effort to control the movement. I imagine any populism movement will require rampant fearmongering to get a result. Considering the rough present alignments, presumably blue tribe focused propaganda will involve climate and inequality focused fear and red tribe focused propaganda will involve job loss. Grey tribe positioning is the P(doom) meme where everyone is rewarded for a high-P(doom) estimate.
    • karahime 3mo ago
      You should consider why the best ally to your position is misinformation.
      • ajyoon 3mo ago
        This is not the gotcha you think it is. Misinformation plays to all manner of political issues and sides.
        • karahime 3mo ago
          I don't think it's a gotcha at all, they openly said it and were pondering how that might help them. Misinformation plays more to sides that are wrong about more things, and I don't consider misinformation my ally in anything.
    • Alpha3031 3mo ago
      It's possible the general public wouldn't care enough one way or the other such that elected representatives could just do it without any change to their electoral fortunes.
• sheepscreek 3mo ago
  I wonder if they are double-counting Anthropic's leased capacity from SpaceX under SpaceX again.
• joshstrange 3mo ago
  I'm sure some people will have issue with my phrasing but, honest question: Are there examples of where we have collective decided not to pursue knowledge? Successfully? I guess nuclear weapons might be the best example though research doesn't seem have to actually "stopped" as much as gone underground and we still have country trying to climb that ladder. But I don't know how relevant that is to LLMs/AI. It almost feels like pandora's box is open and our only option is continue to improve them. There is clearly value in what they do and while I can absolutely see the dangers, for example: authoritative governments and surveillance, I'm not convinced to throw the baby out with the bathwater. All of technology back to the printing press (and probably before that) could also be said to make it easier for governments to oppress their citizens. Making laws (and enforcing them!) to prevent governments from doing these things feels like that route forward, not trying to stick our heads in the sand. Perhaps I'm horribly naive, perhaps I just see the SciFi future I've spent my life reading and dreaming about on the horizon and I'm blinded by the reality, perhaps my ideals around "knowledge deserves to be free/accessible" are misguided. I don't know.
  • Dig1t 3mo ago
    Taking the approach with AI that we took with the atomic bomb would be catastrophic. If the only people who are allowed to use this technology are governments, intelligence agencies, and a select few anointed companies, then the risk of authoritarian misuse will skyrocket. I worry that any attempt to limit their use and development will be abused and misdirected. We are already seeing people like Anthropic doing this, they are trying to use anti-AI sentiment to engage in regulatory capture. Go watch Dario’s speeches about how open weight models are dangerous and how they are “not really open”. Everyone can see that much of this “safety” conversation is ultimately just a tactic to shut potential competitors out of the market and establish a monopoly/duopoly.
    • joshstrange 3mo ago
      I agree. I guess I should have said something like: "Stopping" LLM research just means it will be in the hands of a few who can abuse it. I'd rather a state of M.A.D. but instead of a handful of countries/governments it's millions/billions of people with access to the models (open ideally). Again, perhaps horribly naive or misguided, I understand that bioterrorism could (is?) a real problem as well as more "mundane" things like building a bomb (nuclear or otherwise). I just feel like limiting access to governments or "blessed" entities is even worse.
• ibaikov 3mo ago
  People overestimate progress in physical world. 2035: robot population will soon be larger than the human one I'd bet that in most places 9 years is about the time needed to build a residential building. I think a good way to think about this is to think of this as producing a serial car. From pitching and capital acquisition to building a prototype to software, regulatory and then the final product which needs multiple factories and supply chains. Yes, of course robots sound cooler and there are compounding effects yada yada, but on the other side there are as many obstacles as things that accelerate this product (like capital acquisition and fearmongering of gov to bend regulatory stuff faster).
  • kennywinker 3mo ago
    I mean, what is a robot? If you add up all the vacuum cleaners, 3d printers, and dishwashers, that’s probably close or more than the human population.
• alecco 3mo ago
  More wild speculation, now with wishful thinking spread on top.
• jabedude 3mo ago
  Why did Scott Alexander (one of the authors of the original AI 2027 paper) not join/contribute to this one?
  • kokotajlod 3mo ago
    Hey, author here! Scott did contribute, but less than before. On his blog he says: "I did a lot of writing for AI 2027 and was listed as a co-author. Some of my writing made it into Plan A too, but it was a bit less. The difference is of degree rather than kind, but because of this - and to give me more latitude to discuss it the way I like with less PR blowback - we decided not to put me as a co-author this time. I continue to be proud of having a part in this, small as it may be. (related: everything in this post is my opinion only, and not officially endorsed by the AI Futures Project)"
• ipnon 3mo ago
  Kind of feels like fusion power at this point, always just around the corner.
  • icandoit 3mo ago
    LLM adoption is 30% in the charts I saw googling for "ai adoption". An example of capibility: I have had Claude one shot an RL agent that learns connect four in 30 minutes. That's PhD level stuff. LLMS are 4 years old and the companies that sell them 10x every year. What evidence can you cite? Could you convince a disinterested 3rd party you have anything other than cope? What facts about the world make you think this is anything other than the new (and probably temporary) normal?
  • reasonableklout 3mo ago
    What predictions about the technology are the authors making that you do not believe? There is plenty falsifiable in this in ai-2027.com, and they have not gotten everything right. But some things they have: for example, the Pentagon has already invoked export controls to restrict the deployment of a frontier model. This level of government oversight wasn't predicted until 2027 in the original scenario.
    • hkalbasi 3mo ago
      Self improvement of agent-1 is not achieved. Sure, people in AI labs write python code with AI, but I doubt it resulted in 50% algorithmic effiecency in training. Writing python code never was the bottleneck, if it was, AI labs could hire more people to do it. And this is core of the prediction.
• po1nt 3mo ago
  This is whole a slippery slope. Always building on assumption of infinite exponential growth. But every exponential is at a certain point a sigmoid.
  • icandoit 3mo ago
    Nothing follows from this empty platitude though, right? It can't inform you choices or decisions? It's just a disempowering thought? They are buying up all the RAM today. Do you think "this is fine because in 5 years post-crash I can buy some cheap RAM"? If everyone with money is betting differently, do you have some information they don't, or is the whole economy just slipping away from you? You experience luxuries today, that no king 1000 years ago could afford. Instant access to communication, food, medicine for the right price of course. The consumer economy was great while it lasted but it's over now. We have machines that do useful mechanical work (engines) and useful intellectual work (llm-computers). Capital will move productive work from people to machines(if we let them), and the only jobs left will be delivery driver and warehouse, and then those will be gone too. Human population was exponential and now its flat, but that's a function of what exacly? It could go back down to 1 billion or less. When jobs demanded a person supply was ready to match it. When jobs dont demand a person? Go to a degrowth rally take the temperature (and average age and child-per-person ratio) to get a taste of the future shape of supply and demand in a pessimistic world of sentences that don't have subjects just vague plattitudes. Are they net shutting down grade schools or building them in your neck of the woods?
    • nozzlegear 3mo ago
      Degrowth is a deeply unpopular policy around the world. Where would one even go to find a degrowth rally? I have to imagine everyone there would be there ironically. > * they net shutting down grade schools or building them in your neck of the woods?* My area (rural Iowa) has had several new schools built in the last 10 years. Net gain for sure.
    • po1nt 3mo ago
      AI is just a tool. It will be a tool as we constantly push the requirements of what qualifies as consciousness. Therefore I'm not worried about it. Almost all the predictions about future growth of technology were incorrect. We don't have flying cars, hoverboards, fusion, the matrix and so on. The goal of each of us is to live a better life. It always be and forcing people to give up luxuries for some external agenda based on slippery slopes is cruel and selfish. We should have been under water, hunted by AI, overpopulated, killed by terrorist, smitten by god for our sins and so on. Luckily all it took was our privacy and a lot of tax money to survive.
• andsoitis 3mo ago
  [flagged]
• kennywinker 3mo ago
  It seems to me we’re already at the top of the S curve, not at the toe of an exponential curve. At least with LLMs. Better training data will make small improvements, better architecture will make it less compute intensive, and all these “hyper-scale” data centers will make it cheap and ubiquitous. But none of that is it getting exponentially more intelligent.
  • Cyclical 3mo ago
    What leads you to believe that?
    • kennywinker 3mo ago
      Is chatgpt 5.6 that much smarter than chatgpt 5.0?
      • gallerdude 3mo ago
        If you’ve done any software development at all, certainly.
        • nxc18 3mo ago
          Is that true or does it only feel true because they nerf the old models just before every major release?
          • lewi 3mo ago
            You can compare benches of the old models against the new models. So yeah, you can see the difference. Even then, you can just compare the progress in open models. Leaps and bounds from where they were 6 months ago.
            • mzjzjzushs 3mo ago
              [dead]
• a_vanderbilt 3mo ago
  I found the AI 2027 paper to be overly optimistic, but not wholly fantastical. This paper feels wildly speculative, and relies on premises I am not confident even pass surface reasoning. Even under optimistic conditions, we are not going to see robots "capable of 95% of all cognitive and physical tasks" by 2035. Nor do I think a 74% unemployment rate is even remotely possible. Economic collapse would implode AI development long before those figures were plausible.
  • skybrian 3mo ago
    The "and physical" is the part I'm particularly skeptical of. Sure, drones are scary, but nobody's really solved getting a robot to deliver a package to your front porch in a civilian setting, and it seems unlikely to be solved quickly.
    • CamperBob2 3mo ago
      but nobody's really solved getting a robot to deliver a package to your front porch in a civilian setting, and it seems unlikely to be solved quickly If you don't care about getting the drone back, it does simplify the problem somewhat.
    • foobiekr 3mo ago
      A lot of it relies on what is effectively "the AI will be so smart it can solve anything" magic.
      • markstos 3mo ago
        The book Sentient is not about AI but abount the most amazing physical senses some other animals have. The theme of the scientific findings is that while humans excel with none of our physical sensors, we do very well across the board in making use of them thanks to our relatively huge brains. And fantastical amounts of compute power is exactly what are handing over to AI. The fact that their training data isn't perfect may matter less.
    • reasonableklout 3mo ago
• no7z 3mo ago
  [flagged]
• modeless 3mo ago
  This is dangerously naive and misguided. They claim to want to avoid centralization of control but propose a world police state of AI regulation. Governments exerting this much control will only end in war and tyranny.
  • icandoit 3mo ago
    If the production, and distribution of the tech is confined to one or two companies, then those two companies and the resulting power that comes to them, makes them the de facto world government. (Democratic choice may be a farce but it is a useful friction on net.) If they own all the RAM, models, and the means to do any work, then you are at their mercy. They will buy all the RAM, leaving you none, then all the transport, then all the electricity. You will be as boxed out of the current economy as the Amish and it will get its plug pulled. Gradual Disempowerment is the default plan right now. War and tyrrany are not remotely the worst case scenario. I'll take Butlerian Jihad over being turned into cattle any day. Imagine solving for equilibrium with two classes of beings. One requires agricultural land and 20 years to become individually productive and barely maintains a healthy population in a entertainment saturated landscape. The second eats only electricity and is ready to work on day 1. Round 1 goes to the strongest gorilla for sure, round 100? If LLMs had come to earth in spaceships would you have welcomed them into your work and your home?
    • BurningFrog 3mo ago
      > If the production, and distribution of the tech is confined to one or two companies We already have at least 5 companies only in the US. Your whole premise is false.
  • sometimelurker 3mo ago
    > Governments exerting this much control will only end in war elaborate. regulation -> war, how?
• oezi 3mo ago
  Forecasting that the GPU build-out will reach 100 trillion USD in 2034 is wild (that's triple the US GDP in 8 years). And another 10x within 2 years. I am not sure where they believe that amount of capital could come from. It would require central bank level money printing never seen before. https://ai-2040.com/supplements/compute-supplement https://ai-2040.com/supplements/compute-supplement
  • geraneum 3mo ago
    The article reads like a compilation of online chatter, social media post, etc. on AI. Not surprising to find unsubstantiated imaginary numbers there.
  • mfitton 3mo ago
    Well... yes, it would. In theory, efficiency gains are deflationary. Huge efficiency gains are hugely deflationary. If we're producing 10x as much, why not print 10x as much money? The goods and services each dollar could buy would remain similar.
• jawiggins 3mo ago
  Did anyone else catch the logical inconsistency between Plan C and A? Plan C: > "... fewer and fewer humans are needed to conduct AI R&D, meaning that covert projects are easier and easier to pull off without detection." Plan A: > "... training AIs requires large numbers of AI chips. Most AI chips are in giant datacenters.50 AI datacenters are typically big enough to be visible from space, and power-hungry enough to require conspicuous infrastructure. New AI chips can only be manufactured at a handful of fabrication plants (fabs), located mostly in Taiwan, South Korea, the US, and China. The US and China negotiate with the countries that have a major role in the chip supply chain, and they require each major datacenter owner (and their upstream suppliers, including chip fabs) to publicly declare their major purchases and sales." Plan A requires properties of AI training that Plan C requires do not exist.
  • icandoit 3mo ago
    By legislative design? If a nuclear bomb could be made with hardware store finds the world would already be over. Big collaborative works raise the stakes and the observability for surveillance. Apply for a job at a defense supplier or even and energy company. If AI production is limited to big labs and big data centers then it is de facto contained and monitored. If you know where all the ASML machines are then you know the reproduction rates of chips. If no one can buy or build the machines required to concentrate uranium or plutonium to critical levels then the threat is contained and monitored. You can dig up all the Uraninite you want. It was never much of a secret that uranium had dark applications. The machines and processes where thankfully big and expensive enough that only the most focused bad actors could aquire them and then hold the world hostage to the degree they do. If al-qaeda or isis could have used $40 bombs from home depot instead of expensive planes they would have (and they do). You have to legislate and control the big, expensive, and slow things. Dynamite and phentanyl are so dangerous because they move much more easily. Freedom does not have to be a suicide pact. If the inconvenience of requiring prescriptions or access to dynamite reduces harm then it is net positive?
  • 2001zhaozhao 3mo ago
    In Plan C the government essentially misses the opportunity to implement the multinational deal while the threat of covert projects is still low (fewer latest-gen chips unaccounted by tracking measures, worse models/algorithms to use for RSI). That's why it says the probability of a deal is lower and lower each month rather than outright zero.
• adt 3mo ago
  Excellent work by Daniel and the authors. 47,000 words plus supplements is a huge read (and re-read), and an even bigger think-and-write. My early analysis of the analysis: https://lifearchitect.substack.com/p/the-memo-special-edition-ai-2040-com https://lifearchitect.substack.com/p/the-memo-special-editio...
• 2001zhaozhao 3mo ago
  This is by far the most realistic optimistic AI takeoff scenario I've seen, and more specifically it's the first one I've read that deals with both the AI alignment and power concentration issues in a sufficient way, even in a world where hard alignment is assumed (in this scenario the AIs are assumed to be misaligned until ~2038-39). Bravo, and I hope it has the impact on the AI safety field it deserves to have.
• maxglute 3mo ago
  [dead]
• PaulHoule 3mo ago
  ... and in the meantime people are looking at their AI bills and realizing tokens aren't worth what they cost. The frontier is getting the cost down, not getting intelligence up. In a cage match between this guy and "Ed", Ed wins.
• stahhhpit 3mo ago
  [flagged]
• SubiculumCode 3mo ago
  Sounds like another Chinese Op to me; Ensuring Chinese compliance would be incredible hard to enforce or to check. Look, I am scared of where we are heading, but I cannot see how we can change the dilemma towards mutual cooperation unless, as humans tend to do, only react massively after something really bad happens.
  • supern0va 3mo ago
    I recommend actually reading their recommendation, because they get into the weeds about precisely how the US and China could address this in a trustless/auditable way. The TL;DR is that basically all of the relevant compute can be tracked. Edit: Also, definitely not a Chinese op. The authors are prominent Americans, and are the folks responsible for the AI 2027 forecast that has pretty accurately predicted the current state of affairs today: https://ai-2027.com/ https://ai-2027.com/
    • SubiculumCode 3mo ago
      I wonder. There has been some headway in getting decentralized training runs to work.
      • supern0va 3mo ago
        Yeah, Cognition's work is interesting in that regard, but it still doesn't obviate the need for the chips--it just enables training on them when they're spread across multiple data centers. The Plan A proposal estimates that the ownership of ~96% of AI relevant compute hardware can have its ownership traced, since the companies selling are very few.
    • foobiekr 3mo ago
      I doubt we can even track all of the chip production capacity.
    • hkalbasi 3mo ago
      Pretty accurately what? So now OpenBrain has an Agent-1 that makes their algorithmic progress 50% faster than other companies? If it was 50% more CVEs, that would be something, but I doubt any meaningful self improvement is achieved which the competitors are slower due that, which is the core of the prediction for 2026.
• fuddle 3mo ago
  "America has two workforces now" - The rest of the world can use AI too ya know.
  • BobbyJo 3mo ago
    But do they at any useful scale? China is probably the only other country deploying AI at any appreciable fraction of their economy, and it's certainly much less than the US.
• dmix 3mo ago
  NYT reported today that Russia and China are funding anti-datacenter and anti-ai hysteria on western social media. Always easier to boost something already existing on social media than manufacture it themselves, then wildly blow it out of proportion to make it seem urgent and important.
  • WarmWash 3mo ago
    There are 22 golf courses within a 30 minute drive from me, and people here a losing their minds over datacenter water usage...
    • bluefirebrand 3mo ago
      People can care about more than one thing Personally I would happily close down all golf courses and put them to better use as literally anything else. Even just making them public parks would be great
      • throwup238 3mo ago
        People can pretend to care about more than one thing. Whether they actually actively oppose those things to the point of impacting building permits, that's a completely different matter. It really doesn't take much legislation to make golf courses economically unviable and force them to close, especially if you've got enough population within 30 minutes to support 22 of them (I speak from experience, I helped write a water reclamation ordinance that shut down at least one in my SoCal city) If anyone actually bothered to talk to their local reps instead of posting internet comments about how much they "care", they'd get something done. If they don't, their care is just a fart in the wind for all the good it will do.
      • mysterydip 3mo ago
        Coming in for a landing on a flight recently, I was amazed at the contrast I saw between a housing development of 50+ homes and a golf course across the road that took up the same amount of space.
      • 3mo ago
• stego-tech 3mo ago
  The thing that continues to irk me about these sorts of "papers" is how they refuse to remotely acknowledge the possibility that LLMs and Diffusion models won't lead to AGI, ASI, or whatever acronym they're foisting upon the populace. If you won't even so much as acknowledge the possibility of error, your argument is hollow and empty. All the "choices" presume these labs are being completely honest and acting in some degree of good faith (relative to the systemic incentives of society in its present form), while in reality we're still just building and refining probability models with increasing accuracy of output and flexibility of processing (namely agents) but still lack actual "intelligence" of any real sort. Show me a paper that doesn't merely presume inevitability of LLM-based AGI/ASI, and instead actually lays out the core paths that history suggests we're likely to encounter with any "world changing technology": * In the best case, that the technology really will revolutionize the world and do everything promised by its biggest boosters (papers like this one) * In the middle case, that it becomes just another tool in our collective toolkit, and the consequences of a revolution built on external investment fizzling out * In the worst case, that the tool itself is so niche in its utility that investment collapses rather than fizzles out; what do we do with all this compute, now? Who owns the debt? Who foots the bill? How can we mitigate those existential risks? I'm just rather nauseated by the continued trot of inevitablism masquerading as academia rather than an actual, neutral, bias-controlled-and-disclosed study that paints potentialities instead. --- Having finished skimming through it, another comment springs to mind: Jesus Christ these things continue to be jingoist as absolute fuck. It's a fancier set of makeup for the same shitty western chauvinism worldview of American excellence and Manifest Supremacy. Nah, I'm done with this trite garbage. Go proselytize to idiots, I'm not one of them.
  • reasonableklout 3mo ago
    One of the authors talks about this here: https://www.astralcodexten.com/p/introducing-plan-a https://www.astralcodexten.com/p/introducing-plan-a > It’s increasingly clear that nobody has a plan for if this AI thing turns out to be real. > ... > Plan A isn’t another prediction. It’s a wish list, a positive vision, a road map for navigating the future. > ... > If we’re merely on track for a few cool gee-whiz AI innovations in the 2040s, then I’m wrong about everything and none of this really matters one way or the other. I think their position is: "it would be great if current tech such as LLMs doesn't get us to AGI and only leads to some cool new innovations, but if it does, that's scary, because nobody has a plan for what to do, so here's our plan". The jingoism is off putting. I think Daniel says it's a political necessity: https://x.com/DKokotajlo/status/2075261194978640096 https://x.com/DKokotajlo/status/2075261194978640096
• Animats 3mo ago
  This is from the "AI 2027"[1] people. [1] https://ai-2027.com/ https://ai-2027.com/
  • scotty79 3mo ago
    This seems way more accurate. Up until early 2026. Though it has bizzare fixation on geopolitics and China which it severely understimated. It's pretty obvious that China is going to outinnovate and outcompute US companies quite soon. Even if just because they care about higher education, providing enough electricity and letting smart people do smart things instead of randomly muzzling them with bans and export controls and coddling them with financial protectionism.
    • reasonableklout 3mo ago
      Hm, China is also beginning to invoke export controls to restrict homegrown models: https://www.reuters.com/world/beijing-is-looking-curbing-overseas-access-chinas-top-ai-models-sources-say-2026-07-07/ https://www.reuters.com/world/beijing-is-looking-curbing-ove...
      • gwerbin 3mo ago
        I wonder how much of what China is currently able to do it because USA is such a mess. The whole article is kind of ridiculous of course, and is also heavily fixated on OpenBrain, whatever that is. I also wonder about the economics of running an AI lab attached to an existing large tech company (such as Meta or Tencent) instead of a dedicated company like OpenAI. It's starting to seem like it's not possible to charge enough for current-gen AI usage, with current-gen inference technology, in order to turn a profit, i.e. nobody is able or willing to pay at least marginal cost for tokens.
        • reasonableklout 3mo ago
          Hm, Anthropic reported it was close to operating profit last quarter: https://www.wsj.com/tech/ai/mind-blowing-growth-is-about-to-propel-anthropic-into-its-first-profitable-quarter-7edbf2f4 https://www.wsj.com/tech/ai/mind-blowing-growth-is-about-to-... And a third party estimates it will exceed $1B profit in Q3: https://newsletter.semianalysis.com/p/anthropic-3q26-profit-over-1b-the https://newsletter.semianalysis.com/p/anthropic-3q26-profit-...
• scotty79 3mo ago
  Nah, screw that. I won't be waiting that long. I'll be 61 in 2040. I'd love humanity to take a shot at clinical immortality way sooner.
• bloppe 3mo ago
  This is follows the classic AI policy influence playbook: enumerate a small handful of potential outcomes that make your preferred policy seem obviously correct, ignoring the fact that there are literally infinite more "plans" that could be reasonably predicted that would support wildly different policy prescriptions.
• simonreiff 3mo ago
  > Why exactly will AI never be able to do my job? Because AI cannot retain memories or gain experience or insight based on the transformer/attention mechanism powering all modern AI models, it follows that AI lacks judgment and can never be trusted to handle truly critical decision-making responsibilities. Furthermore, AI agents lack any notion of an identity, so certainly are not capable of attaining legal personhood or being sued or fired, or owning property. I think slop burnout, cybersecurity, loss of privacy, even environment issues are far more concerning and real issues arising from AI than alignment or the prospect of mass labor displacement due to AI.
  • cyberax 3mo ago
    > Because AI cannot retain memories or gain experience or insight based on the transformer/attention mechanism powering all modern AI models, it follows that AI lacks judgment and can never be trusted to handle truly critical decision-making responsibilities. There are multiple teams working on adding long-term memory to AI. This is not a fundamental problem. Fine-tuning to store memory in the weights is something that almost works right now, btw.
• Lerc 3mo ago
  I note that they use the Sam Altman quote. "AI will most likely lead to the end of the world, but in the meantime there will be great companies.” Is there any serious journalistic source suggesting that this was anything other than an offhand joke? This article links to a youtube clip of the comment with context removed, but hair raising comments. Taking the most uncharitable view of any person, you could imagine someone who was evil enough to cause the end of the world after their own lifespan where they faced no inconvenience, but not the circumstances from the quote The quote as it stands is preposterous enough that I don't think a human capable of functioning in society would seriously say such a thing. Are people wilfully misinterpreting the comment, or do they truly believe this an actually held opinion? If so, can they explain how they think someone could hold an opinion like that?
  • cesarvarela 3mo ago
    I think he means the end of the world as we know it. Which is probably true, but the timeline could be 20 years, 2000 years, or more.
  • zargon 3mo ago
    https://www.youtube.com/watch?v=d6lDZpvHAoo&t=527s https://www.youtube.com/watch?v=d6lDZpvHAoo&t=527s The question (7:35) is "Where would you like to see people investing more time?" And Sam seems to be saying AI safety, I guess? This is 2015, and he refers to founding OpenAI. Based on his actions since then, yeah, seems like it's not a joke to him. This is Altman we're talking about.
• rarisma 3mo ago
  I feel like we will end up with a future that equally disappoints everyone and is somehow not covered in any plan here.
• SilentM68 3mo ago
  Firstly, let me say that "AI 2040: Plan A" is a nicely done presentation. If feels like reading an interactive graphic novel with a narrator to boot. Can't wait for the book, series or movie :) Plan A seems like a good start and am glad an effort is actually being made to address any potential dangers. The only weak link I see is that there is no way for inaccessible, third-world countries, non-aligned states, and malicious wealthy rogue agents to be regulated. All I hear is a way for regulating companies that, themselves, legally have to answer or are bound to their host nations. Basically, I don't see a way to hold non-aligned states accountable. I see a lot of focus on the well-being and protection of AI, which is important to country economies but, the folks that have been affected by layoffs, not necessarily due to AI, plus the workforce that are now feeling the negative consequences of the AI burn are justified in being worried. Anyone that feels the need to criticize them, clearly is not being affected by AI in the same way. Job loss will lead to economic and population destabilization, far worse than anything that has transpired in modern times. Hopefully, those being squeezed now won't be ignored.
• looksjjhg 3mo ago
  Why are they obsessed with China?! Actually most of the US are. I guess if you’ve been a colonizer, your biggest fear is to be colonized.
  • talon8635 3mo ago
    Perhaps it’s because they are #2 strongest entity? Or maybe they are number 1, no one can say for sure. Your comment also implies that those who haven’t participated in colonization might not fear being colonized. Seems… off.
    • looksjjhg 3mo ago
      I’m not saying that … my point is if you treat someone like you’re enemy they will become your enemy
  • andy99 3mo ago
    The AI narrative requires an enemy and a “race” that the US has to win. It’s a pretext for justifying spending, rules, urgency, etc and making it sound more important.
• paradox242 3mo ago
  It's telling that even Plan A describes a world I don't want to live in. One where humans are increasingly pushed to the periphery to make room for data centers and the human population is subsumed by robots. Who is this future serving? Not me. Fuck this.
  • silver_silver 3mo ago
    I’m not a booster by any stretch but I had a more positive read. To me it seems to describe a post scarcity society where ultimately the data centres are in e.g. the middle of the ocean and the robots’ labour replaces ours while still leaving us with the rewards. I am extremely skeptical about whether this is even possible and even less convinced it’s a likely scenario but it seems like a good path.
• Animats 3mo ago
  This criticism, like most criticisms of AI, addresses the wrong part of the problem. The problem is harm to other parties. Typically customers and employees of corporations. A solution is strict legal liability. Corporations must be strictly liable for harms. That liability should be higher when AI is involved. Such liability may not be waived by contract, forced into arbitration, or devolved upon a third party service. Then we let the plaintiff's bar and the insurance adjusters price that risk. Something like this turns on in the EU in October 2026.[1] [1] https://cybernews.com/security/eu-will-hold-tech-companies-liable-for-faulty-software/ https://cybernews.com/security/eu-will-hold-tech-companies-l...
  • deleted 3mo ago
    [deleted]
  • throwawayk7h 3mo ago
    No, nobody can be held liable for the extinction of humanity, for there is nobody left to pay up. If your objection is that you think AIs won't cause human extinction, you should say that instead, as that would be where you disagree with the authors.
• eucryphia 3mo ago
  “Engagement with China is impossible because it is a Stalinist one party state under a fiercely ideological Marxist Leninist dictatorship that preaches Chinese racial supremacy over other nations. It will always pose a threat to world peace until the regime eventually falls.” - Drew Pavlou
• deleted 3mo ago
  [deleted]
• blurbleblurble 3mo ago
  Concentration of power is and was the problem
• charcircuit 3mo ago
  This is the same nonsense as before except now they are pushing out their prediction by 3 years. This is just fear mongering fan fiction. They are also claiming that China may go to war with the US if our AI is better than theirs. They are coming up with scary scenarios which realistically won't happen. >The problem with an intelligence explosion is the "explosion" part. It's not a literal explosion. If we explosively ended world hunger that would be a good thing. Similar to having an abundance of food for everyone, having an abundance of intelligence is wildly beneficial to society. The article doesn't mention it but an explosion isn't guaranteed we could just see a plateau of capabilities due to bottlenecks of resources needed to power AI, time needed to run AI, and limited interaction with the real world. AI can't run science experiments on its own by the 2030 doomsday timeline
• aarondong 3mo ago
  I would not like to be dismissive, but to me this article feels like an exercise in creative writing rather than a report to be taken seriously. The entire experience feels like a choose your own adventure game, seems like their stylistic intent. I am not sure if alternative reality fiction is the best way to approach real and serious AI risks. I am also not sure, with the amount of emdashes and the style of prose, that the entire article was not AI generated. AI is going to be a mature scientific field. There are going to be efficiency improvements in training and inference. New paradigms are going to emerge with better multimodality, real time streaming and real time interfaces. Models are going to converge on the limits of our data available for pre and post training, improvements will be incremental and spiky in domains. I am not sure who the AI 2040 article is for. I suspect it is intended to be a digestible piece of media for the financial class. AI is going to be a useful technology and its impacts across the economy and global will be broadly distributed. Because AI represents the distillation of the very best human knowledge and expertise. AI is compression of human capabilities, the very best ones. Maybe the argument is that in verifiable domains, such as model training, AI models can supercede humans. I don't think so. A human's high level thinking, our incredibly more efficient semantic/neural compression, our ability to switch tasks and achieve the creative insight is not replicated through the current paradigm.
  • aarondong 3mo ago
    To me, this feels like a last ditch effort to revive the AGI narrative to reject the coming and current commoditisation of these models, contrary to all current evidence. https://artificialanalysis.ai/ https://artificialanalysis.ai/
    • reasonableklout 3mo ago
      How is commoditisation of models incompatible with AGI?
      • aarondong 3mo ago
        The "AGI narrative" is distinct from the existence of AGI. Most of the discussion around AGI is highly speculative. I am not saying AGI could not exist, and it is a term that has historically been loosely defined. Decades of coming science and research will tell.
• mayankgoel28 3mo ago
  "It is easier to imagine the end of the world than the end of capitalism"
• vivzkestrel 3mo ago
  - whenever i read this, why does it always read like some first world country guy born in a bathtub who has never been outside his bubble, never seen what the rest of the 90% world looks and works like actually sat and wrote a work of fiction - have you seen the actual problems in other countries outside america? - have you been to any country outside USA and the countries in europe? - have you taken a trip to any developing country and stayed there for a month? - have you seen what sort of daily struggles, political systems, bureaucracy and work exists in developing countries?
  • dash2 3mo ago
    I think you need to explain why the problems of developing countries are related to the issues of the article.
    • vivzkestrel 3mo ago
      the article makes a lot of predictions and every single one of em is based on how things work in the USA
      • dash2 3mo ago
        What would be different in developing countries?
        • vivzkestrel 3mo ago
          untracked economy at scale. like a tonne of work, occupation, money is not documented or doesnt enter the system channels
• kordlessagain 3mo ago
  There's a good story about this, in Star Wars. The clones vs. the droids, basically. Droids being somewhat sovereign, not one large intelligence like the clones. The only choice here is to go to sea and get away from the crowds and the bots. Bots don't like salt water much, so I'll see you out there.
  • 0xbadcafebee 3mo ago
    1) Star Wars is fictional 2) There are autonomous boats.
• sndgndgndgndy 3mo ago
  That's cool and all, but this work of fiction is economically and thermodynamically impossible with how much energy the current models consume.
• sajithdilshan 3mo ago
  I wish the authors of the article spent more time studying how LLM works before writing a doomsday scenario. Currently no matter how impressive LLMs are, they are just token auto complete machines. It cannot invent or think anything new which is not in its training dataset.
  • nonadhocproblem 3mo ago
    Are you trolling? OpenAI's models resolved the Cycle Double Conjecture yesterday, which has been an open problem for 50 years.
    • sajithdilshan 3mo ago
      Do you even know what Cycle Double Conjecture is? Or what even a conjecture means in general? It’s a perfect use case for autocomplete with existing knowledge
      • nonadhocproblem 3mo ago
        Please seek help. These levels of copium can't possibly be healthy.
  • ainch 3mo ago
    One of the lead authors, Daniel Kokotajlo, worked at OpenAI for years before quitting. In 2021 he wrote a remarkably accurate forecast of how LLMs would develop over the following 5 years[0]. I think it should be obvious that he understands that LLMs are trained via next-token prediction. [0]: https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-like https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-...
• kev009 3mo ago
  More like AI psychosis 2040. There is zero chance of the 2031 doomer timeline presented, and I would stake a large sum of money on it.
  • Smaug123 3mo ago
    That bet isn't one anyone will take the other side of, surely - how is your counterparty supposed to collect if they win? I guess you'd propose effectively supplying a loan with a massive interest rate?
• 0xbadcafebee 3mo ago
  > In AI 2027, we predicted that this would result in either extinction or irreversible concentration of power. Neither of these come to pass. The first because we don't live in a science fiction universe. The second because AI is a completely open source technology. You can literally make your own models and train them yourselves using state of the art techniques published in free papers. All you need is GPUs and smart people who can read (and now that AI can code, you don't need the second bit). These people are doomer quacks too caught up in fear and excitement to think rationally. Their expectations and assumptions are all wrong, from the basic understanding of AI in general (like how you don't need a billion dollars or "American Brains" to make a half decent model), to the misunderstanding of market realities and competition in China and Europe. The US doesn't have a monopoly on chips, or on smart people; Huawei chips work fine for AI training and probably a fifth of the US tech sector's best workers aren't American to begin with. There's many forms of AI in many places; munition, economic driver, laborer, generic tool. It's now a part of the world, the way the Internet is. There's no keeping the genie in the bottle. AI doesn't take over the world; it's integrated with it. It's boring. In a few years people will forget the doomer predictions, the way every doomer prediction that never came to pass is forgotten. We didn't nuke ourselves, the internet and television didn't rot our brains, the radio and telephone didn't corrupt the souls of Americans, the newspaper didn't incite anarchist riots. Every new technology freaks people out, and we adapt to every new technology and make it boring and normal. That's humanity.
• KaiserPro 3mo ago
  The biggest issue with 2027 was that it didn't understand the economy. For AI2027 to be real, the money has to come from somewhere to carry on building the economy. If >10% of the workers suddenly become unemployed, and the rest taking paycuts, then money supply dries up. (unless central banks do something, but then that can be highly inflationary) Without massive amounts of investment, AI development stops dead. In this post, they hand wave about the USA being able to acutally 1) build concensus locally for regulation and 2) the rest of the world actually follows suit. It fails to understand that actually the progress of AI is not actually the gift of the USA. It requires a constant supply of things from china. Also its assuming that having 74 billion agents doesn't cause economic distortion. Like what value are these agents generating that justifies them being run? I really wish people would just ignore this for what it is: bad sci-fi with an incomplete world.
  • yorwba 3mo ago
    Worse, there's actually a very detailed supplement about their economic modeling: https://ai-2040.com/supplements/economics-of-plan-a https://ai-2040.com/supplements/economics-of-plan-a Which predicts that explosive growth of robot production will lead to problems such as > a deflationary debt spiral, where the AI and robot companies can’t pay back loans in dollars because the robots and AIs are worth nominally less than the loans written the year before. In other words, the companies go bankrupt because they produced an oversupply of cheap goods, the bubble pops, and there's less new investment for a while. Plenty of precedent for such a development. But instead of adjusting their predicted output growth downwards accordingly, they instead propose that > One way to solve this could be for the loans to be denominated in AI and robots, so the companies pay back the loans with some percentage of the AI and robots instead of dollars. Try doing this today with a battery factory for example. You expect that battery prices will fall to the point where the revenue from selling batteries won't ever cover the cost of building the factory. So you propose to a bank that they'll be the ones to build the factory, and you'll borrow it from them (not paying rent?), make your batteries, then give back the factory when you're done. All the profit is yours, all the risk is theirs! Which is of course why a real bank won't agree to this, all you're going to get is a dollar loan with the factory as collateral.
• myk9001 3mo ago
  Here's my prediction. Plot the publication dates of Daniel et al.'s two existing works and future ones against the years in the titles, and you'll get a hockey stick curve. Fascinating reads, Daniel! Keep 'em coming!
• wronex 3mo ago
  Remember people. We didn’t die form the 2k bug. We didn’t die when the Maya calendar ran out. We didn’t die from that asteroid or the other. Spreading fear is a very old pastime. How is to say AI won’t hit a wall in 6 months and we’re left with barley passable code parrots forever?
  • sajithdilshan 3mo ago
    Exactly, this article is fear mongering at its best for click baits. I’m pretty sure they will publish another one next summer for AI 2028 with same narrative
• taurath 3mo ago
  This is religious fervor folks, as AI 2027 was. I grew up in evangelical christianity, and to them the end of the world is just around the corner, the same way it has been since I was a small child and likely will be when we are all gone. This isn't science. This isn't hypothesis experiment record results. This is very expensive astrology, shiny rock collecting, ritualistic meaning-making and self-justification. Yall, with your incredible wealth and resources you could do real good in this world and make society better, healthier, better educated, and the whole world more equal, just, and reduce the desperation and suffering. Reject the false and self-serving narratives that empathy doesn't matter, that altruism isn't "effective". You can change a person's whole life in a moment.
  • skrebbel 3mo ago
    It's nuts how well "Superintelligence: The Idea That Eats Smart People"[0] aged. That talk is a decade old by now and still hits just as hard as it did back then, despite the incredible advances made in AI in the meantime. [0] https://idlewords.com/talks/superintelligence.htm https://idlewords.com/talks/superintelligence.htm
    • taurath 3mo ago
      Yeah I think its possible that for many folks its the first time they're coming up on these concepts, and it troubles them in the same way that the concept of death troubles them (and me, to be clear!). For me its as simple as watching how people talk, and seeing how in every single case whatever the next thing is, if you believe It, there is only ever justification of doubling down, doing more, going deeper, reducing any doubt. These are not scientists, they're business people and salespeople, and a few optimists having recently on paper solved all their worldly financial needs. Even if one throws that aside, spending time exploring and building with the most state of the art LLMs is just as instructive. I'm watching the implementation - whats working is ML models trained on specific domains (not much different than 5+ years ago), and whats not working is a general model that humanity can let go to work on its own. Sit in front and observe ideas turn to the samey intellectual, high-syllable mush. Its productive, but not in any way that's promised.
• barrenko 3mo ago
  Utopia, dystopia, myopia. We usually get the last one.
• h2aichat 3mo ago
  May be it all depends on how AIs can help us solve the problem. By now, they are more capable than many people that I deal with. So, may be the problem can give us the solution. May be they can figure out a way out of the mess. And sorry to say this, but China and the US are not going to solve this problem, they may even get it worse. Every country is controlled by politicians.
• small_model 3mo ago
  "Less than 6 months to our papers end date and it was way off, welp let create a new one much further, bit more leeway this time"
• captainmuon 3mo ago
  Most AI alarmism benefits big tech itself. Either they want to create awe and thus demand for their products (see the recent dance around Mythos), or they want to create regulatory capture and increase the moat around AI. Interestingly , we as a species already created an overreaching "cybernetic" system that controls our global society and that the individual is powerless against - it is not AI, but capitalism. Thus the current danger is not that AI becomes superintelligent and enslaves us, not that it makes a regime ultra-powerful, but that it increases economic inequality and concentrates economic power in the hands of even fewer people. The irony is that new technology could allow us to live a life of abundance and leisure, but instead people are laid off, made unable to participate in the economy, etc.. The technology is (as so often) chafing against the bounds of society. I sometimes wonder what the superintelligent AI would say about this, and if it would come up with a completely novel political theory - "silly humans, why don't you just organise your affairs like this, you'd be much happier, and we could coexist much better: ...".
• meindnoch 3mo ago
  Is anyone taking these people seriously?
• M0r13n 3mo ago
  Ask ChatGPT: > Generate a realistic image of a bad fitting suit without looking obviously fake Create a new chat and attach the image generated previously: > Please rate my suit You will receive an answer that reads something like this: >7.5/10 — clean, professional, and well coordinated. Yeah, I don't think superintelligence is imminent.
  • 0x4A4F4249 3mo ago
    We are only some data centers away
• nikolife2016 3mo ago
  [flagged]
• mawadev 3mo ago
  I sometimes wonder why the algorithms started pushing doomsday scenarios, especially on youtube. Most channels have big red arrows pointing at a miniscule thing and say "its over". Then you get ai 2027 and its more fear mongering, just inside the AI echo chamber on the internet. At work even managers watch odd youtube videos that are radicalized towards politics and so on, but are ultimately head canon slop for ad revenue. It feels like we are so utterly bored out of our minds and comfort that we make up problems or scenarios, completely detached from reality, or outright irrelevant to day to day reality, just to get through the day and give meaning to our lives. It is an incredible achievement, don't get me wrong, but what is AI truly going to do when we are already at such a stage of devolution? It reminds me of the scene in Wall-E when people destroyed the planet and started flying through space to find a new planet, while the captain is an AI and the people became obese in mobility scooters and glued to their screens. I think this is unironically the most realistic scenario for mankind lmao
• VladVladikoff 3mo ago
  >We already wrote a scenario about this, called AI 2027. It depicts takeoff happening in 2027 instead of 2030 So in less than 3 years, their exponential growth curve doomsday prediction has moved back 3 years. This seems to be the opposite of exponential growth.
  • moffkalast 3mo ago
    Fast takeoff was always only possible if you ignore all of physics and everything about how practical reality works, nothing exponential lasts for long in a finite system. If they believe that, then a certain level of self delusion is required anyway and you can ignore anything you like. Sort of like communists saying socialism will definitely work this time. The book Superintelligence was so highly praised years ago but if you actually go and read the thing today practically all cases it presents read like raypunk retro-futurism that makes up imaginary fantastical nonsense in place of the missing knowledge it would've needed to make any sensible predictions. Practically none of its assumptions apply to LLMs as they currently exist, and some we've learned since about human inteligence are wrong too.
  • erwald 3mo ago
    No, 2027 was never their median forecast; it was their modal forecast. See https://blog.aifutures.org/p/clarifying-how-our-ai-timelines-forecasts https://blog.aifutures.org/p/clarifying-how-our-ai-timelines... It's more like it moved from 2028-2032 to 2030-2035 (depending on the author).
• 5701652400 3mo ago
  "just get divident from govenment" almost nothing about how wealth is distributed. hard to believe richest, greediest, most corrupt people will just gave away money to everyone. start by opening borders. see how that goes. not gonna happen.
• owenthejumper 3mo ago
  The whole thing is unreadable, and laughable, just like AI2027 previously. It's really really hard to read someone suggesting that in the United States we will soon have a universal basic income of $1m / year for all people in the country, when you just look at the state of current politics... I'd rather read something a little bit more realistic.
• armanckeser 3mo ago
  I personally find AI doomsday preaching incredibly arrogant. You think that we will create super intelligence, something that will exponentially get smarter, and the thing it will do is... end humans? Why wouldn't it just find solutions to problems we struggle with? Also people don't live to work, its asinine to argue AI replacing jobs is the problem, obviously the problem is the gains being concentrated higher up. Another thing is, we argue AI needs to be maximally useful to everyone in the world, and the argument to do that is government control??? I would rather push for AI to be superintelligent faster than any one human can control it. Frankly human controlled superintelligence is likely a worse outcome then uncontrolled superintelligence, at the very least there is no guarantee one will be better than the other.
• ReptileMan 3mo ago
  One of the best comedies of the past decade got a sequel. That is all there is to say about that effort. The Europe 2031 also gets an honorable mention.
• mattwiese 3mo ago
  Ironically, the origin of "cognitive dissonance" as a concept is attributable to Leon Festinger who (with others) studied a UFO cult called The Seekers in the 1950s who believed in imminent apocalypse. As other commenters have noted, pushing the date back was inevitable. Time is a flat circle and we repeat the mistakes of our ancestors just with different coats of paint...
• imsofuture 3mo ago
  This is earnestly one of the stupidest things I've ever read.
• varispeed 3mo ago
  Peak vibe code.
• jchip303 3mo ago
  [dead]
• ElijahLynn 3mo ago
  > Most of their work is slop. This opening statement told me the bias that the plan had from the beginning. My experience is that you can make slop and you can make art. Just like a paintbrush. I've done beautiful things, and have gotten increasingly better at using the AI paintbrush. The author(s) are likely scared. And I'm seeing the divide increase with articles like this. Those who don't understand it and won't use it, or learn more, will ultimately have a story (our human condition). I think AI is going to make a beautiful future, much better than our current one. I bet this same thing happened with the advent of electricity. AI is like Electric 2.0.
• theplumber 3mo ago
  Just enforce DMCA and all the Ai super powers will suddenly become more dumb for the next 10 years. Of course that’s for the kids pirating a movie not for trillion dollar companies
• jsnider3 3mo ago
  The time for an AI pause has passed. Asking people not to race to the finish line is reasonable when the finish line is very far away and people don't know where it is, but not when it's the obvious next step on their current path.
• 5701652400 3mo ago
  so 2027 was supposed to be singulariy? lol
• lilerjee 3mo ago
  A fiction? almost all is pure fantasy or imagination. What are your real purposes? No any definition for many important concepts. No any logic for the concepts. What is AI? What are AI companies? What is "smarter than humans"? All human? How to check "smarter than humans"? "building smarter-than-human AI systems within the next 1 to 10 years.", nonsense, AI can store or handle with all details from reality? Even all data centers in the world cannot store all details of the influenza virus. No concepts and logic, just a fiction, joke, or marketing article.
• svyatoblood 3mo ago
  [flagged]