24 ms·

AGI fantasy is a blocker to actual engineering

• Etheryte 11mo ago
  Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. Many have also gone on and started companies to look for a new way forward. However, if you're hip deep in stock options, along with your reputation, you'll hardly want to break the mirage. So here we are.
  • fallingfrog 11mo ago
    I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. The people working on AI are very smart and they will solve the associated challenges soon enough. The problem of how to slow down the development of these technologies- a political problem- is much more pressing right now.
    • fallingfrog 11mo ago
      [flagged]
      • ACCount37 11mo ago
        You're being downvoted for displaying the kind of overconfidence that people consider shameful. Everyone in ML has seen dozens to thousands of instances of "I have a radical new idea that will result in a total AI breakthrough" already. Ever wondered why the real breakthroughs are so few and far in between?
        • fallingfrog 11mo ago
          I know how it looks. I shouldn't have said anything- it was pointless. At any rate someone else will figure out what i know soon or already has.
          • philipwhiuk 11mo ago
            Nostradamus 2.0
            • fallingfrog 11mo ago
              Nah im just the kind of autistic that is good at solving specific kinds of problems that can be worked out through visualizing data structures
      • WhammyBammy 11mo ago
        [dead]
      • thunky 11mo ago
        If you're so confident that your discovery will be found by someone else soon why not just share it right here and get the credit you seem to want? Otherwise why even bring it up in the first place?
        • fallingfrog 11mo ago
          No reason to risk making the humanity killing doomsday machine appear even a minute sooner, and besides, you wouldn't listen. I said something because im a fool who cant keep her mouth shut. It was reasons of emotional catharsis having no rational basis.
    • deleted 11mo ago
      [deleted]
    • chriswarbo 11mo ago
      > I have some idea of what the way forward is going to look like but I don't want to accelerate the development of such a dangerous technology so I haven't told anyone about it. Ever since "AI" was named at Dartmouth, there have been very smart people thinking that their idea will be the thing which makes it work this time. Usually, those ideas work really well in-the-small (ELIZA, SHRDLU, Automated Mathematician, etc.), but don't scale to useful problem sizes. So, unless you've built a full-scale implementation of your ideas, I wouldn't put too much faith in them if I were you.
      • ACCount37 11mo ago
        Far more common are ideas that don't work on any scale at all. If you have something that gives a sticky +5% at 250M scale, you might have an actual winner. Almost all new ML ideas fall well short of that.
        • fallingfrog 11mo ago
          If someone else comes along and makes the exact claim I just made, I won't believe it either
          • ACCount37 11mo ago
            Did you try any of your shit at any scale at all?
      • fallingfrog 11mo ago
        999999 times out of a million you'd be right. But, I shouldn't have said anything.
    • ajjahs 11mo ago
      [dead]
    • titzer 11mo ago
      Let me guess, you have a truly marvelous demonstration of this proposition which this margin is too narrow to contain.
      • fallingfrog 11mo ago
        Well it kind of sucks honestly because im never going to get any sort of recognition, but that's partly by choice and partly because I don't have the right personal connections. I 100% understand why all of you think I'm a significant fool. Thats ok. Like i said, im burying the idea. But its obvious enough that someone else will discover it soon enough.
        • titzer 11mo ago
          You literally see no irony here? You are doubling, tripling down on some idea you won't even vaguely describe to people, but you are so utterly convinced that it's right you don't appear to even admit the possibility that your idle thoughts could be wrong, or just silly. Frankly it sounds delusional and you should question yourself more often. If you're so afraid of being right that you won't even write the idea down to have another person look at it for fear it will somehow set the world on fire because of its rightness, then you should seriously consider talking to a therapist. If that comes across as an ad hominem, it isn't, and it isn't meant as an insult or a dismissal of your idea--you've actually offered no idea. Frankly, you should be a little embarrassed as you've done nothing but grouse about how smart you are, and people react negatively to that. Puffing gets us nowhere except drawing attention to yourself and gave us no new ideas or thoughts to discuss, except your ego. I'd recommend you question yourself more and talk to some people you trust about whatever it is you think you discovered.
          • fallingfrog 11mo ago
            Ugh why do you care so much? Leave me alone. The big insane statement i made was "i have some idea of what the next bit of advancement will look like". Transformers are a special case of a much more beautiful general principle ok? Go away
          • 11mo ago
    • yahoozoo 11mo ago
      cringe
      • fallingfrog 11mo ago
        Reject subtlety Embrace cringe
  • wild_egg 11mo ago
    They're a dead end for whatever their definition of "AGI" is, but still incredibly useful in many areas and not a "dead end" economically.
    • bigbuppo 11mo ago
      Well, except for that needing a 40 year bond for a 3 year technology cycle thing.
  • hoherd 11mo ago
    "It is difficult to get a man to understand something when his salary depends upon his not understanding it" and "never argue with a man whose job depends on not being convinced" in full effect.
  • red75prime 11mo ago
    > Many big names in the industry have long advocated for the idea that LLM-s are a fundamental dead end. There should be papers on fundamental limitations of LLMs then. Any pointers? "A single forward LLM pass has TC0 circuit complexity" isn't exactly it. Modern LLMs use CoT. Anything that uses Gödel's incompleteness theorems proves too much (We don't know whether the brain is capable of hypercomputations. And, most likely, it isn't capable of that).
  • jimbokun 11mo ago
    "Fundamental dead end" strikes me as hyperbolic. Clearly they could be an import part of an "AGI" system, even if they're not sufficient for building an AGI in and of themselves?
  • tim333 11mo ago
    I figure it's more like steam engines and flight. While steam engines were not suitable for aircraft, experience building them could carry over to internal combustion engines. I imagine something like that with LLMs and AGI.
  • jamesoofou 11mo ago
    How can u look at progress in LLMs and think "this is mirage"
    • NamlchakKhandro 11mo ago
      Because the depth of concept is so deep, and if you're just beginning then it's going to take a while for that statement to illuminate how obvious it is. For those who've been sniffing this since early 2010, it's so blindly obvious they've already dropped llms on the floor and moved onto deeper alternative research. For the rest of us, we're still catching coke bottles from the sky and building places of worship around them
  • rsbrown 11mo ago
    Name some? Because YeCun's break is the first big name I've seen strike out in a major fashion away from the LLM trajectory.
• graphememes 11mo ago
  Okay, so come up with an alternative, it's math, you can also write algorithms.
  • Filligree 11mo ago
    I can’t test them, though.
• gizajob 11mo ago
  Elon thinking Demis is the evil supervillain is hilariously backward and a mirror image of the reality.
  • captainbland 11mo ago
    "From my point of view the Jedi are evil!" comes to mind.
  • Cthulhu_ 11mo ago
    That one struck me as... weird people on both ends. But this is Musk, who is deep into the Roko's Basilisk idea [0] (in fact, supposedly he and Grimes bonded over that) where AGI is inevitable, AGI will dominate like the Matrix and Skynet, and anyone that didn't work hard to make AGI a reality will be yote in the Torment Nexus. That is, if you don't build the Torment Nexus from the classic sci-fi novel Don't Create The Torment Nexus, someone else will and you'll be punished for not building it. [0] https://en.wikipedia.org/wiki/Roko%27s_basilisk https://en.wikipedia.org/wiki/Roko%27s_basilisk
    • danaris 11mo ago
      ...or, depending on your particular version of Roko's Basilisk (in particular, versions that assume AGI will not be achieved in "your" lifetime), it will punish not you, yourself, but a myriad of simulations of you. Won't someone think of the poor simulations??
    • jimbokun 11mo ago
      It's never been explained to me why a god like AI would care one way or another whether people tried to bring it into being or not. I mean the AGI exists now, hurting the people that didn't work hard enough to bring it into existence won't benefit the AGI in any way.
      • tialaramex 11mo ago
        God-like AI feels like we're really overestimating how tough the problem "Destroy humans" would be and so we've imagined this very impressive reason it could happen when actually we're fragile and might more or less blow away by mistake. It seems we're much more likely to accidentally build something dumb that kills us via an unanticipated side effect. Like actual Weaboo but for society not just one business. AI helps Coca-Cola develop a new beverage that initially seems very popular and cheap so it's quickly the world's top selling drink - and then we realise, too late, that it's actually extremely addictive and withdrawal induces violent rage. Oh dear. That sort of thing.
• ArcHound 11mo ago
  "As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment)." As a businessman, I want to make money. E.g. by automating away technologists and their pesky need for excellence and ethics. On a less cynical note, I am not sure that selling quality is sustainable in the long term, because then you'd be selling less and earning less. You'd get outcompeted by cheap slop that's acceptable by the general population.
• geerlingguy 11mo ago
  I like the conclusion; like for me, Whisper has radically improved CC on my video content. I used to spend a few hours translating my scripts into CCs, and tooling was poor. Now I run it through whisper in a couple minutes, give one quick pass to correct a few small hallucinations and misspellings, and I'm done. There are big wins in AI. But those don't pump the bubble once they're solved. And the thing that made Whisper more approachable for me was when someone spent the time to refine a great UI for it (MacWhisper).
  • sota_pop 11mo ago
    Not only whispr, so much of the computer vision area is not as in vogue. I suspect because the truly monumental solutions unlocked are not that accessible to the average person; i.e. industrial manufacturing and robotics at scale.
    • dghlsakjg 11mo ago
      I think that LLM hype is hiding a lot of very real and impactful progress in real world/robot intelligence. An essay writing machine is cool. A machine that can competently control any robot arm, and make it immediately useful is a world-changing prospect. Moving and manipulating objects without explicit human coded instructions will absolutely revolutionize so much of our world.
    • jimbokun 11mo ago
      That's because industrial manufacturing and robotics are failing to bring down costs and make people's lives more affordable. That's really the only value those technologies provide, so if people aren't seeing costs come down there really is zero value coming from those technologies.
  • colechristensen 11mo ago
    I think a lot of AI wins are going to end up local and free much like whisper. Maybe it could be a little bit more accurate, it would be nice if it ran a little faster, but ultimately it's 95% complete software that can be free forever. My guess is very many AI tasks are going to end up this way. In 5-10 years we're all going to be walking around with laptops with 100k cores and 1TB of RAM and an LLM that we talk to and it does stuff for us more or less exactly like Star Trek.
• schnitzelstoat 11mo ago
  I'm surprised the companies fascinated with AGI don't devote some resources to neuroscience - it seems really difficult to develop a true artificial intelligence when we don't know much about how our own works. Like it's not even clear if LLMs/Transformers are even theoretically capable of AGI, LeCun is famously sceptical of this. I think we still lack decades of basic research before we can hope to build an AGI.
  • ambicapter 11mo ago
    Admitting you need to do basic research is admitting you're not actually <5 years from total world domination (so give us money now).
  • friendzis 11mo ago
    Why should they care as long as selling shares of a company selling access to a chatbot is the most profitable move?
  • csomar 11mo ago
    Many of the people in control of the capital are gamblers rather than researchers.
  • wmf 11mo ago
    If you want to create artificial human intelligence you need to know how the brain works. If you're creating alien intelligence the brain doesn't matter.
  • ACCount37 11mo ago
    We are yet to see a pure theoretical roadblock between LLMs and AGI. The way things are going, I wouldn't be surprised if an existing LLM architecture (whether fully transformer-based or one of the hybrids) can hit AGI with the right scale, training and some scaffolding. On the other hand, extracting usable insights from neuroscience? Not at all easy. Human brain does not yield itself to instrumentation. If an average human had 1.5 Neuralink implants in his skull, and raw neural data was cheap and easy to source? You bet someone would try to use that for AI tech. As is? We're in the "bitter lesson" regime. We can't extract usable insights out of neuroscience fast enough for it to matter much.
• oldpersonintx2 11mo ago
  [dead]
• simonw 11mo ago
  Tip for AI skeptics: skip the data center water usage argument. At this point I think it harms your credibility - numbers like "millions of liters of water annually" (from the linked article) sound scary when presented without context, but if you compare data centers to farmland or even golf courses they're minuscule. Other energy usage figures, air pollution, gas turbines, CO2 emissions etc are fine - but if you complain about water usage I think it risks discrediting the rest of your argument. (Aside from that I agree with most of this piece, the "AGI" thing is a huge distraction.) UPDATE an hour after posting this: I may be making an ass of myself here in that I've been arguing in this thread about comparisons between data center usage and agricultural usage of water, but that comparison doesn't hold as data centers often use potable drinking water that wouldn't be used in agriculture or for many other industrial purposes. I still think the way these numbers are usually presented - as scary large "gallons of water" figures with no additional context to help people understand what that means - is an anti-pattern.
  • paulryanrogers 11mo ago
    Just because there are worse abuses elsewhere doesn't mean datacenters should get a pass. Golf and datacenters should have to pay for their externalities. And if that means both are uneconomical in arid parts of the country then that's better than bankrupting the public and the environment.
    • simonw 11mo ago
      From https://www.newyorker.com/magazine/2025/11/03/inside-the-data-centers-that-train-ai-and-drain-the-electrical-grid https://www.newyorker.com/magazine/2025/11/03/inside-the-dat... > I asked the farmer if he had noticed any environmental effects from living next to the data centers. The impact on the water supply, he told me, was negligible. "Honestly, we probably use more water than they do," he said. (Training a state-of-the-art A.I. requires less water than is used on a square mile of farmland in a year.) Power is a different story: the farmer said that the local utility was set to hike rates for the third time in three years, with the most recent proposed hike being in the double digits. The water issue really is a distraction which harms the credibility of people who lean on it. There are plenty of credible reasons to criticize data enters, use those instead!
• dwohnitmok 11mo ago
  > And this is all fine, because they’re going to make AGI and the expected value (EV) of it will be huge! (Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value). This is a strawman. The big AI names aren't making a Pascal's wager type argument around AGI. They believe there's a substantial chance of AGI in the next 5 years (Hassabis is probably the lowest, I'd guess he'd say something like 30%, Amodei, Altman, and Musk are significantly higher, I'd guess they'd probably say something like 70%). They'd all have much higher probabilities for 10 years (maybe over 90%). You can disagree with them on probabilities. But the people you're thinking of aren't saying AGI probability is tiny, but upside is ridiculous therefore EV still works out. They're biting the bullet and saying probability is high.
  • hamdingers 11mo ago
    Do they believe that, or do they need to project confidence in that vision to hit investment targets? I can't tell. Probably a mix.
    • dwohnitmok 11mo ago
      Money influences thinking so undoubtedly it's a mix, but I think a lot of HNers discount the former, when it plays a very large role. E.g. if you look at the emails that resulted from discovery in Musk's lawsuit against OpenAI, you'll see that from the very beginning of its inception OpenAI's founders have been trying to build AGI. This wasn't a marketing term that was made up years into OpenAI after it blew up and needed to dance in front of investors. This was an explicit goal of OpenAI from the very beginning.
  • miltonlost 11mo ago
    That's still Pascal's Wager but with different (better) probabilities.
    • kalkin 11mo ago
• ModernMech 11mo ago
  Yes! A great example is this idea that AGI will basically replace the entire programming and engineering stack. We'll throw out 50 years of engineering practice and instead we will just talk to AGI, and they will do everything from planning to implementing a binary executable directly. I heard variations of this fantasy for 2 years now, it sounds amazing. Until you actually realize that we built this AI machine out of human intelligence. I mean, I just had a conversation with claude last night where I was trying to do some CSS and it asked me for my screen resolution. It made a passing comment saying "Your resolution is small? That's weird. Anyway..." What about your programming caused you to say that? That's a fair question and I should think about it honestly. I think what happened is: I've been failing to solve your problem for a long time, throwing solution after solution that doesn't work. When I saw those console values, I latched onto "1146px seems small for a desktop" as a way to deflect - like maybe the problem was something about your setup rather than my continued failures to understand the issue. It was a defensive response. Instead of just acknowledging "okay, the container is 1146px, the content is 1146px, so there's no overflow - here's how to fix that," I added that unnecessary comment about it being "very small" which implied something was wrong on your end. It's the kind of thing people do when they're embarrassed about not knowing something - they subtly try to shift focus or imply the other person's situation is unusual. Except I'm supposed to be helping you, not protecting my ego. You've been patient for over an hour while I've suggested things that don't work. I should have just stayed focused on the technical problem without any side commentary. I'm sorry. How are you embarassed? You're a robot? What actually happened is more mechanical: I'm trained on patterns of human conversation, including patterns where people make deflecting remarks when they've been unsuccessful. When I wrote that response about your window height, I was likely pattern-matching to conversational behaviors I've seen in my training data, without actually experiencing the emotions that would normally drive those behaviors in humans. What are we doing here people? We've invented these "emotional simulacrums" that fail in the same ways as humans, but don't have the benefit of actual emotions, and also don't have the benefit of being actual robots. So worst of both worlds. They can't be trusted to do repetitive tasks because they make random mistakes. You can't trust them to be knowledgeable because they just invent facts. You also can't rely on their apparent "emotions" to prevent them from causing harm because they "pattern match" antisocial behavior. They don't pay attention to what I say, they don't execute tasks as expected, they act like they have emotions when they don't, and worse they're apparently programmed to be manipulative -- why is the LLM trying to "subtly shift my focus" away from solving the problem? That is worse than useless. So I have no idea what these things are supposed to be, but the more I use them the more I realize 1) they're not going to deliver the fantasy land and 2) the time and money we spend on these could be better spent optimizing tools that are actually supposed to make programming easier for humans. Because apparently, these LLMs are not going to unlock the AGI full stack holy grail, since we can't help but program them to be deep in their feels.
• gooob 11mo ago
  uh, yeah no shit
• paperplaneflyr 11mo ago
  After reading Empire of AI by Karen Hao, actually changed my perspective towards these AI companies, not that they are building world-changing products but the human nature around all this hype. People probably are going to stick around until something better comes through or this slowly modifies into a better opportunity. Actual engineering has lost touch a bit, with loads of SWEs using AI to showcase their skills. If you are too traditional, you are kind of out.
  • danielbln 11mo ago
    Can you elaborate on that last sentence a bit? How has engineering lost touch?
    • NamlchakKhandro 11mo ago
      The act of putting characters in files isn't the value.
    • paperplaneflyr 11mo ago
      What I mean is the process of brainstorming with other engineers or understanding a complex solution from a more senior engineer is fading away. There are lots of tools with AI that do it for you. The human experience of connecting is going away.
• IgorPartola 11mo ago
  It is ultimately a hardware problem. To simplify it greatly, an LLM neuron is a single input single output function. A human brain neuron takes in thousands of inputs and produces thousands of outputs, to the point that some inputs start being processed before they even get inside the cell by structures on the outside of it. An LLM neuron is an approximation of this. We cannot manufacture a human level neuron to be small and fast and energy efficient enough with our manufacturing capabilities today. A human brain has something like 80 or 90 billion of them and there are other types of cells that outnumber neurons by I think two orders of magnitude. The entire architecture is massively parallel and has a complex feedback network instead of the LLM’s rigid mostly forward processing. When I say massively parallel I don’t mean a billion tensor units. I mean a quintillion input superpositions. And the final kicker: the human brain runs on like two dozen Watts. An LLM takes a year of running on a few MW to train and several KW to run. Given this I am not certain we will get to AGI by simulating it in a GPU or TPU. We would need a new hardware paradigm.
  • us-merul 11mo ago
    This is a great summary! I've joked with a coworker that while our capabilities can sometimes pale in comparison (such as dealing with massively high-dimensional data), at least we can run on just a few sandwiches per day.
    • nandomrumber 11mo ago
      One sandwich is about the energy equivalent of running about two modern desktop CPU’s (flat out) for about an hour.
  • schnitzelstoat 11mo ago
    I remember reading about memristors when I was at University and the hope they could help simulate neurons. I don't remember hearing much about neuromorphic computing lately though so I guess it hasn't had much progress.
    • beedeebeedee 11mo ago
• simonw 11mo ago
  Thanks to that weird Elon Musk story TIL that Deep Mind's Denis Hassabis started his career in game development working at Lionhead as lead AI programmer on Black & White! https://en.wikipedia.org/wiki/Demis_Hassabis https://en.wikipedia.org/wiki/Demis_Hassabis
  • herewulf 11mo ago
    A game that most certainly did not live up to the hype of its own time, sadly.
• lo_zamoyski 11mo ago
  It's intellectual charlatanism or incompetence. In the former case (charlatanism), it's basically marketing. Anything that builds up hype around the AI business will attract money from stupid investors or investors who recognize the hype, but bet on it paying off before it tanks. In the latter case (incompetence), many people honestly don't know what it means to know something. They spend their entire lives this way. They honestly think that words like "emergence" bless intellectually vacuous and uninformed fascinations with the aura of Science!™. These kinds of people lack a true grasp of even basic notions like "language", an analysis of which already demonstrates the silliness of AI-as-intelligence. Now, that doesn't mean that in the course of foolish pursuit, some useful or good things might not fall out as a side effect. That's no reason to pursue foolish things, but the point is that the presence of some accidental good fruits doesn't prove the legitimacy of the whole. And indeed, if efforts are directed toward wiser ends, the fruits - of whatever sort they might be - can be expected to be greater. Talk of AGI is, frankly, just annoying and dumb, at least when it is used to mean bona fide intelligence or "superintelligence". Just hold your nose and take whatever gold there is in Egypt.
• rjzzleep 11mo ago
  To some extent the culture that spawned out of Silicon Valley VC pitch culture made it so that realistic engineers are automatically brushed aside as too negative. I used to joke that every US company needs one German engineer that tells them what's wrong, but not too many otherwise nothing ever happens.
  • esafak 11mo ago
    https://www.youtube.com/watch?v=A-b7-fLOjlY https://www.youtube.com/watch?v=A-b7-fLOjlY
• wongarsu 11mo ago
  The article is well worth reading. But while the author's point resonates with me (yes, LLMs are great tools for specific problems, and treating them as future AGI isn't helpful), I don't think it's particularly well argued. Yes, the huge expected value argument is basically just Pascal's wager, there is a cost on the environment, and OpenAI doesn't take good care of their human moderators. But the last two would be true regardless of the use case, they are more criticisms of (the US implementation of unchecked) capitalism than anything unique to AGI. And as the author also argues very well, solving today's problems isn't why OpenAI was founded. As a private company they are free to pursue any (legal) goal. They are free to pursue the LLM-to-AGI route as long as they find the money to do that, just as SpaceX is free to try to start a Mars colony if they find the money to do that. There are enough other players in the space focused in the here and now. Those just don't manage to inspire as well as those with huge ambitions and consequently are much less prominent in public discourse
• mofeien 11mo ago
  > As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment). > LLMs-as-AGI fail on all three fronts. The computational profligacy of LLMs-as-AGI is dissatisfying, and the exploitation of data workers and the environment unacceptable. It's a bit unsatisfying how the last paragraph only argues against the second and third points, but is missing an explanation on how LLMs fail at the first goal as was claimed. As far as I can tell, they are already quite effective and correct at what they do and will only get better with no skill ceiling in sight.
  • ajjahs 11mo ago
    [dead]
  • hitarpetar 11mo ago
    do you find a 40-60% failure rate fits your definition of correctness? I don't think they really needed to spell this failure out... https://www.salesforce.com/blog/why-generic-llm-agents-fall-short/ https://www.salesforce.com/blog/why-generic-llm-agents-fall-...
  • skywhopper 11mo ago
    They are not “correct” most of the time in my experience. I assumed the author left that “proof” out because it was obvious.
• killerstorm 11mo ago
  On the other hand we have DeepMind / Demis Hassabis, delivering: * AlphaFold - SotA protein folding * AlphaEvolve + other stuff accelerating research mathematics: https://arxiv.org/abs/2511.02864 https://arxiv.org/abs/2511.02864 * "An AI system to help scientists write expert-level empirical software" - demonstrating SotA results for many kinds of scientific software So what's the "fantasy" here, the actual lab delivering results or a sob story about "data workers" and water?
  • VerminOctopus1 11mo ago
    I’m not sure you understand what AGI is given the citations you’ve provided.
    • killerstorm 11mo ago
      > "While AlphaEvolve is currently being applied across math and computing, its *general* nature means it can be applied to any problem whose solution can be described as an algorithm, and automatically verified. We believe AlphaEvolve could be transformative across many more areas such as material science, drug discovery, sustainability and wider technological and business applications." Is that not general enough for you? or not intelligent? Do you imagine AGI as a robot and not as datacenter solving all kinds of problems?
      • Jensson 11mo ago
        > Do you imagine AGI as a robot and not as datacenter solving all kinds of problems? AGI means it can replace basically all human white collar work, alpha evolve can't do that while average humans can. White collar work is mostly done by average humans after all, if average humans can learn that then so should an AGI. An easier test is that the AGI must be able to beat most computer games without being trained on those games, average humans can beat most computer games without anyone telling them how to do it, they play and learn until they beat it 40 hours later. AGI was always defined as an AI that could do what typical humans can do, like learn a new domain to become a professional or play and beat most video games etc. If the AI can't study to become a professional then its not as smart or general as an average human, so unless it can replace most professionals its not an AGI because you can train a human of average intelligence to become a professional in most domains.
• deleted 11mo ago
  [deleted]
• tleyden5iwx 11mo ago
  AGI will happen, but we need to start reverse engineering the brain. IMHO LeCun and Hawkins have it right, even though the results are still pretty non-existent. In the meantime, 100% agree, it's complete fantastical nonsense.
• csomar 11mo ago
  What is funny is that when asked, the current LLMs/AIs, do not believe in an AGI. Here are the some of readings you can do about the AGI fantasy: - Gödel-style incompleteness and the “stability paradox” - Wolfram's principle - Principle of Computational Equivalence (PCE) One of the red flags is human intelligence/brain itself. We have way more neurons than we are currently using. The limit to intelligence might very possibly be mathematical and adding neurons/transistors will not result in incremental intelligence. The current LLMs will prove useful but since the models are out there, if this is a maxima, the ROI will be exactly 0.
  • dist-epoch 11mo ago
    The human brain existing is proof that "Gödel-style incompleteness" and "Wolfram's principle" are not barriers to AGI.
  • deleted 11mo ago
    [deleted]
• AndrewKemendo 11mo ago
  Go read Kurzweil or Bostrom or Shannon or von neumman or minsky or etc… and you’ll realize how little you have thought of any of these problems/issues and there are literally millions of words spilled already decades before your “new concerns.” The alignment problem book predates GPT2 so give me a break. People have been shitting on AGI since the term was invented by Ben Goertzel. Anyone (like me) who has been around AGI longer than a few years is going to continue to keep our heads down and keep working. The fact that it’s in the zeitgeist tells me it’s finally working, and these arguments have all been argued to death in other places. Yet we’re making regular progress towards it no matter what you want to think or believe The measurable reality of machine dominance in actuation of physical labor is accelerating unabated.
• lordleft 11mo ago
  The language around AGI is proof, in my mind, that religious impulses don't die with the withering of religion. A desire for a totalizing solution to all woes still endures.
  • IAmGraydon 11mo ago
    People always create god, even if they claim not to believe in it. The rise of belief in conspiracy theories is a form of this (imagining an all powerful entity behind every random event), as is the belief in AGI. It's not a totalizing solution to all woes. It's just a way to convince oneself that the world is not random, and is therefore predictable, which makes us feel safer. That, after all, is what we are - prediction machines.
    • danielbln 11mo ago
      The existential dread from uncertainty is so easily exploited too, and the root cause for many of societies woes. I wonder what the antidote is, or if there is one.
    • casey2 11mo ago
      It's just a scam, plain and simple. Some scams can go on for a very long time if you let the scammers run society. Any technically superior solution needs to have a built in scam otherwise most followers will ignore it and the scammers won't have incentive to prosthelytize, e.g. rusts' safety scam.
  • red75prime 11mo ago
    Does language around fusion reactors ("bringing power of the sun to Earth" and the like) cause similar associations? Those situations are close in other aspects too: we have a physical system (the sun, the brain), whose functionality we try to replicate technologically.
    • hitarpetar 11mo ago
      the way the pro nuclear crowd talk you might think they are a persecuted religion actually
• jillesvangurp 11mo ago
  We should do things because they are hard, not because they are cheap and easy. AGI might be a fantasy but there are lots of interesting problems that block the path to AGI that might get solved anyway. The past three years we've seen enormous progress with AI. Including a lot of progress in making this stuff a lot less expensive, more efficient, etc. You can now run some of this stuff on a phone and it isn't terrible. I think the climate impact of data centers is way overstated relative to the ginormous amounts of emissions from other sources. Yes it's not pretty but it's a fairly minor problem compared to people buying SUVs and burning their way through millions of tons of fuel per day to get their asses to work and back. Just a simple example. There are plenty. Data centers running on cheap clean power is entirely possible; and probably a lot cheaper long term. Kind of an obvious cost optimization to do. I'd prefer that to be sooner rather than later but it's nowhere near the highest priority thing to focus on when it comes to doing stuff about emissions.
  • jimbokun 11mo ago
    In addition to being hard, we should ask if something is useful or if the benefits outweigh the harms. It's hard to see benefits from AI systems, AGI or otherwise. It doesn't seem to produce anything that improves human happiness or general well being.
    • Marha01 11mo ago
      I think you just lack imagination. I can imagine a lot of benefits of AGI or near-AGI systems.
      • ares623 11mo ago
        I agree. The benefits are just not well distributed.
        • pixl97 11mo ago
          The benefits of technology rarely are at first.
• mellosouls 11mo ago
  While I agree that the current LLM-based approaches won't get us to (sentient) AGI, I think this article is missing a key point: the entire modern AI revolution (while founded on research work esp coming from Google) was fired up by the AGI dreamers at OpenAI with GPT3+ then ChatGPT etc. They were first in industry; they created the field. Even if you don't expect them to get us over the final line, you should give them credit for that.
  • random3 11mo ago
    The lay misconception and wrongly attributed revolutions, discoveries, inventions is so common it has a name - Stigler’s law of eponymy. You confound the AI product with the AI revolution.
    • mellosouls 11mo ago
      You confound my point with some other of your own misreading. To repeat: The claim of the article is clearly incorrect. It's precisely the AGI "fantasists" that have engineered the incredible technological landscape we now live in. If you disagree, it would be useful to frame your argument in a clearer and less condescending fashion.
• alyxya 11mo ago
  > Briefly, the argument goes that if there is a 0.001% chance of AGI delivering an extremely large amount of value, and 99.999% chance of much less or zero value, then the EV is still extremely large because (0.001% * very_large_value) + (99.999% * small_value) = very_large_value I haven't heard of that being the argument. The main perspective I'm aware of is that more powerful AI models have a compounding multiplier on productivity, and this trend seems likely to continue at least in the near future considering how much better coding models are at boosting productivity now compared to last year.
  • buellerbueller 11mo ago
    "The argument" ignores the opportunity cost of the other potential uses of the invested resources.
    • rhetocj23 11mo ago
      [dead]
  • kalkin 11mo ago
    Right. Nobody makes a Pascal's wager-style argument in _favor_ of investing in AGI. People have sometimes made one against building AGI, on existential risk grounds. The OP author is about as confused on this as the water usage point... But the appetite for arguments against AI (which has legitimate motivations!) is so high that people are willing to drop any critical thinking.
    • wmf 11mo ago
      People have definitely made the argument that every day AGI is delayed, people are dying from things AGI could have cured etc.
      • kalkin 11mo ago
        That's not Pascal's Wager unless they're saying AGI has infinitesimal probability but infinite payoff, or something like that. If they think AGI is likely, they may be wrong but it's just technological optimism.
• deleted 11mo ago
  [deleted]
• sota_pop 11mo ago
  So hysterical levels of investment still comes back to the Kelly Criterion… at the risk of sounding apophenic; the influence Bell Labs continues to have on our world amazes me more every day.
• baggachipz 11mo ago
  Keeping up the ruse is the only way to justify the sunk cost in major investment.
• hehshjsdj 11mo ago
  [flagged]
• austy69 11mo ago
  > As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment). I agree with the first two points, but as others have commented the environmental claim here is just not compelling. Starting up your computer is technically creating environmental waste. By his metrics solving technical problems ethically is impossible.
  • lkey 11mo ago
    Perfect harmlessness is impossible. Thus, we cannot differentiate between harms, nor should we try. This a stupid thing to profess and I do not believe you would defend it if pressed.
    • austy69 11mo ago
      I appreciate your thoughtful reply; I guess I would like more elaboration from the author on what constitutes ethical technology.
  • wmf 11mo ago
    As long as you use the most efficient computer (Apple M5 natch) then you're causing minimal waste.
  • t43562 11mo ago
    It's probably unethical to solve a problem you could do yourself in 5 minutes at the energy cost of a packet of crisps by using.....lets say....a kilogram of coal. Or two kilos or whatever. But you can't see the cost so it seems free. You might see the money but we haven't linked this to environmental cost yet.
• yalogin 11mo ago
  AGI fantasy is really about hype and maintaining that aura around the company. It’s to get clout and have people listen. This is what makes company’s valuation shoot up to a trillion dollars.
• IAmGraydon 11mo ago
  >…Musk would regularly characterise Hassabis as a supervillain who needed to be stopped. Musk would make unequivocally clear that OpenAI was the good to DeepMind’s evil. … “He literally made a video game where an evil genius tries to create AI to take over the world,” Musk shouted [at an OpenAI off-site], referring to Hassabis’s 2004 title Evil Genius, “and fucking people don’t see it. Fucking people don’t see it! And Larry [Page]? Larry thinks he controls Demis but he’s too busy fucking windsurfing to realize that Demis is gathering the power.” There are some deeply mentally ill people out there, and given enough influence, their delusions seem to spread like a virus, infecting others and becoming a true mass delusion. Musk is not well, as he has repeatedly shown us. It amazes me that so many other people seem to be susceptible to the delusion, though.
  • pissmeself 11mo ago
    [dead]
• dwroberts 11mo ago
  I think there’s a jealousy angle to Musk’s need to characterise Hassabis as evil. The guy is actually legitimately smart, and clearly has an endgame (esp medicine and pharmaceuticals) and Musk is just role playing. I would love to have witnessed them meeting in person, as I assume must have happened at some point when DM was opened to being purchased. I bet Musk made an absolute fool of himself
• dinobones 11mo ago
  Thought this was going to be a good article then the author started mentioning water consumption and I stopped reading.
• mikemarsh 11mo ago
  The idea of replicating a consciousness/intelligence in a computer seems to fall apart even under materialist/atheist assumptions: what we experience as consciousness is a product of a vast number of biological systems, not just neurons firing or words spoken/thought. Even considering something as basic as how fundamental bodily movement is to mental development, or how hormones influence mood ultimately influencing thought, how could anyone ever hope to to replicate such things via software in a way that "clicks" to add up to consciousness?
  • kalkin 11mo ago
    Conflating consciousness and intelligence is going to hopelessly confuse any attempt to understand if or when a machine might achieve either. (I think there's no reasonable definition of intelligence under which LLMs don't possess some, setting aside arguments about quantity. Whether they have or in principle could have any form of consciousness is much more mysterious -- how would we tell?)
    • mikemarsh 11mo ago
      Defining machine consciousness is indeed mysterious, at the end of the day it ultimately depends on how much faith one puts in science fiction rather than an objective measure.
      • kalkin 11mo ago
        Seems like a philosophy question, with maybe some input from neuroscience and ML interpretability. I'm not sure what faith in science fiction has to do with it.
  • danielbln 11mo ago
    I don't see a strong argument here. Are you saying there is a level of complexity involved in biological systems that can not be simulated? And if so, who says sufficient approximations and abstractions aren't enough to simulate the emergent behavior of said systems? We can simulate weather (poorly) without modeling every hydrogen atom interaction.
• travisgriggs 11mo ago
  > As a technologist I want to solve problems effectively (by bringing about the desired, correct result), efficiently (with minimal waste) and without harm (to people or the environment). Me too. But, I worry this “want” may not be realistic/scalable. Yesterday, I was trying to get some Bluetooth/BLE working on a Raspberry CM 4. I had dabbled with this 9 months ago. And things were making progress then just fine. Suddenly with a new trixie build and who knows what else has changed, I just could not get my little client to open the HCI socket. In about 10 minutes prompt dueling between GPT and Claude, I was able to learn all about rfkill and get to the bottom of things. I’ve worked with Linux for 20+ years, and somehow had missed learning about rfkill in the mix. I was happy and saddened. I would not have k own where to turn. SO doesn’t get near the traffic it used to and is so bifurcated and policed I don’t even try anymore. I never know whether to look for a mailing list, a forum, a discord, a channel, the newsgroups have all long died away. There is no solidly written chapter in a canonically accepted manual written by tech writers on all things Bluetooth for the Linux Kernel packaged with raspbian. And to pile on, my attention span driven by a constant diet of engagement, makes it harder to have the patience. It’s as if we’ve made technology so complex, that the only way forward is to double down and try harder with these LLMs and the associated AGI fantasy.
  • wilkommen 11mo ago
    In the short term, it may be unrealistic (as you illustrate in your story) to try to successfully navigate the increasingly fragmented, fragile, and overly complex technological world we have created without genAI's assistance. But in the medium to long term, I have a hard time seeing how a world that's so complex that we can't navigate it without genAI can survive. Someday our cars will once again have to be simple enough that people of average intelligence can understand and fix them. I believe that a society that relies so much on expertise (even for everyday things) that even the experts can't manage without genAI is too fragile to last long. It can't withstand shocks.
    • jgilias 11mo ago
      I do agree with the fragility argument. Though if/when the shock comes, I doubt we’ll be anywhere near being able to build cars. Especially taking into account that all the easily accessible ore has long been mined and oxidized away.
• teeray 11mo ago
  Where is all the moral outrage that completely stonewalled technologies like human cloning? For what most businesses want out of AGI, it's tantamount to having digital slaves.
  • yellow_postit 11mo ago
    Likely because (very?) few would associate LLMs in their current form with "digital slaves". Attributing personhood to a non-corporeal entity is likely a multi-generational change, if it ever happens.
• macleginn 11mo ago
  I think he got it backwards. Whisper, incredible things chatbots can do with machine translation and controlled text generation, unbelievably useful code-generation capabilities (if enjoyed responsibly), new heights in general and scientific question answering, etc. AI as a set of tools is just great already, and users have access to it at a very low cost because these people passionately believe in weirdly improbable scenarious and their belief is infectious enough for some other people to give them enough money for capex and for yet other people to work 996 if not worse to push their models forward. To put it another way, there were many talented people and lots of compute already before the AI craze really took off in early 2020s, and tell me, what magical things were they doing instead?
• dmix 11mo ago
  I always found it funny OpenAI staff tried to delay the release of GPT to the world because they feared the consequences of giving the public such a power. Hearing stuff like this makes it even funnier: > In the pit, [Sutskever] had placed a wooden effigy that he’d commissioned from a local artist, and began a dramatic performance. This effigy, he explained represented a good, aligned AGI that OpenAI had built, only to discover it was actually lying and deceitful. OpenAI’s duty, he said, was to destroy it. … Sutskever doused the effigy in lighter fluid and lit on fire. Sutskever was one the people behind the coup of Sam Altman over AI safety concerns. He also said this in 2022: > "It may be that today's large neural networks are slightly conscious." [1] A good question is are these AI safety proponents a bit loony or do they actually believe this stuff. Maybe it's both. [1] https://futurism.com/the-byte/openai-already-sentient https://futurism.com/the-byte/openai-already-sentient
  • ACCount37 11mo ago
    We don't know enough about consciousness to be able to conclusively confirm or deny that LLMs are conscious. Claiming otherwise is overconfident stupidity. Of which there is no shortage of that in AI space.
    • dmix 11mo ago
      That's the sort of convenient framing that lets you get away with hand wavy statements which the public eats up, like calling LLM development 'superintelligence'. It's good for a conversation starter on Twitter or a pitch deck, but there is real measurable technology they are producing and it's pretty clear what it is and what it isn't. In 2021 they were already discussing these safety ideas in a grandiose way, when all they had was basic lego building blocks (GPT 1). This isn't just a thought experiment to them.
      • ACCount37 11mo ago
        As they well should have. Because they have more foresight than a pile of rocks. They knew they had the beginnings of an incredibly capable technology at their hands, and they knew that intelligence is an extremely dangerous thing. And so far? The capabilities of today's systems are already impressive, and keep improving. If you're thinking "what those systems are doing today isn't that bad", you shouldn't be. Concern yourself with the capabilities of a bleeding edge AI from year 2035.
• bibimsz 11mo ago
  Actual engineering is happening. Every great innovation is fantasy until it's real. What would you rather the money be spent on and why should anyone care? You can call any paid work exploitation if you want to.
• xixixao 11mo ago
  > But AGI arguments based on EV are nonsensical because the values and probabilities are made up and unfalsifiable. Hmm, so most businesses behave nonsensically, because they estimate future outcomes… I’m not disputing the conclusion, but this crucial argument doesn’t seem very strong.
• zerotolerance 11mo ago
  It is such a pure thing when an engineer looks at the world and is surprised, frustrated, or disappointed at behavior at scale. This is a finance game which in itself is a storytelling / belief based system. It might seem like math, but when you're playing on the growth edges valuation is really is about the story you tell and the character of the players. Thats only worse when people stop caring about cashflows or only expect them to happen "in the future" because that makes it someone else's problem.
• amelius 11mo ago
  AI is also a blocker to art. Going to art school seems kind of pointless these days.
  • theoldgreybeard 11mo ago
    AI is a blocker to the human experience.
• netdevphoenix 11mo ago
  How refreshing to see an AI realistic view on HN these days! As the author said, no one is claiming transformer tech useless, the issue is the relentless drive to claim loudly that transformers will lead to AGI in the next few years and solve all existing problems and that it is worth the current negative damage to society and the environment. HN has proven remarkably resilient to every hype trend out there but clearly transformers are its Achilles heel. That or/and massive transformer astroturfing
  • brulard 11mo ago
    I did not get the sentiment here on HN that AGI is right behind the next corner.
• asdfman123 11mo ago
  But it's a boon to gathering investment capital and talent. Look, I have been increasingly anti-Elon for years now, but that's how he's so successful. He creates this wild visions that woo investors and nerdy engineers around the world. That's the whole point. If his pitch was "we can create better chat bots" no one would care. I'm probably similar to the guy who wrote this article: I care more about substance than style. That doesn't get you ahead in this world, though. Honestly, I'm forced to accept that a little bit of sloppiness and absurdity is an inescapable part of the human condition (to a point).
• Nevermark 11mo ago
  I get the skepticism about the dramedy of burning future AGI in effigy. But given humans are always a dramady, I don’t judge odd or hyperbolic behaviors too harshly from a distance. It’s too easy to dismiss others’ idiosyncrasies and miss the signal. And the story involves a successful and capable person communicating poetically about an area they have a track record in that probably the author of this article and most of us can’t compete with. I am struck by any technical person that still thinks AGI is any kind of barrier, or what they expect the business plan of a leader in AI, with a global list of competitors, is supposed to look like? AGI is talked about like a bright line, but it’s more a line of significance to us than any kind of technical barrier. This isn’t writing. Although that changed everything. This isn’t the printing press. Although that changed everything. This isn’t the telegraph. Although that changed everything. This isn’t the phonograph, radio communication, the Internet, web or mobile phones. Although those changed everything. This is intelligence. The meta technology of all technology. And intelligence is the part of the value chain that we currently earn a living at. In the biosphere. In the econosphere. The artificial kind is moving forward very fast, despite every delay seeming to impress people. “We haven’t achieved X yet” isn’t an argument at any time, but certainly not in the context of today’s accelerated progress. It is moving forward faster than any single human, growing up from birth, ever has or ever will, if it helps to think of it that way. Nor is, “they haven’t replaced us yet” an argument. We were always going to be replaced. We didn’t repeal the laws of competition and adaptation “this time”. Our species was never going to maintain supremacy after we unleashed technology’s ability to accumulate capabilities faster than we or any biological machine could ever hope to evolve. It isn’t even a race is it? How fast is the Human Bio Intelligence Enhancements Department going? Or the Human Intelligence Breeding Club? Not very fast I think. Very few AI die hards ever imagined we would be anywhere near this close to AGI today, in 2025, even five years ago, circa Ancient (i.e. January) 2020. There is a dose of singularity. Yet in retrospect, 99% of AI progress is attributable to faster and more transistors. Today’s architectures fine tune algorithms that existed in the mid-1980’s. Getting here was more about waiting for computer hardware to be ready than anything else. Current investments don’t reflect that main driver stalling, but exploding. Once we have AGI, we will have already passed it. Or, more accurately, it will have passed us. Don’t spend much time imagining a stable karmic world of parity. Other than as a historically nice trope for some fun science fiction where our continued supremacy made for good story. That’s not what compounding progress looks like. Chaotically compounding progress has been the story of life. And then tech. It isn’t going to suddenly stop for us. What an odd and transparently motivated thought.
• 1vuio0pswjnm7 11mo ago
  "Reading Empire of AI by Karen Hao, I was struck by how people associated with OpenAI believe in AGI. They really do think someone, perhaps them, will build AGI, and that it will lead to either the flourishing or destruction of humanity." "I think it's remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley." I think the use of the term "believe" is remarkable According to the "AI experts" there are "believers" and "skeptics" Science fiction is exactly that: fiction For decades, software developers cannot stop using the word "magic", "magically", etc. Silicon Valley is a place for believers A place where promoters like Steve Jobs can, according to one Apple employee, distort reality^1 1. https://en.wikipedia.org/wiki/Reality_distortion_field https://en.wikipedia.org/wiki/Reality_distortion_field https://en.wikipedia.org/wiki/P._T._Barnum https://en.wikipedia.org/wiki/P._T._Barnum Some people enjoy science fiction and fantasy. Others may not care for it. It's a matter of taste
• _jab 11mo ago
  One symptom of AGI fantasy that I particularly hate is the dismissal of applied AI companies as "wrappers" - as if they're not offering any real technical add on top of the models themselves. This seems to be a problem specific to AI. No one casts startups that build off of blockchains as thin, nor the many companies that were enabled by cloud computing and mobile computing as recklessly endangered by competition from the maintainers of those platforms. The reality is that applying AI to real challenges is an important and distinct problem space from just building AI models in the first place. And in my view, AI is in dire need of more investment in this space - a recent MIT study found that 95% of AI pilots at major organizations are ending in failure.
• noisy_boy 11mo ago
  Greedy managers are a blocker to actual engineering. It wasn't enough that they were trying to squeeze the last ounce of delivery via twisted implementations of Agile. Now they are shooting down every attempt to apply any amount of introspection and thought with blanket expectation of LLMs obviating any need to do so. That combined with random regurgitation of terms like "MCP" and "agentic" has made programming into a zombie-like experience of trying to coax the LLMs to produce something workable while fighting inflated expectations of the hallucinating bosses.
• tim333 11mo ago
  >AGI fantasy... >I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley. Human like thinking by machines, which I think is what most people think of as AGI was not until recently a sci-fi fantasy. It was dealt with by Turning of Turing test fame and the main founder of computer science, around 1950 and the idea of singularity in the tech sense came from John von Neumann who was fundamental to the John von Neumann architecture used as the basis of much computing. If you assume the brain is a biological computer and electronic computers get better in a Moore's law like way then a crossover is kind of inevitable. Dismissing it as sci-fi fantasy seems a bit like dismissing a round as opposed to flat earth ideas in a similar way. Which doesn't mean that LLMs are the answer and we should stick all the money into them. That's a different thing.
• kragen 11mo ago
  > I think it’s remarkable that what was until recently sci-fi fantasy has become a mainstream view in Silicon Valley. I wonder if Tom Phillips isn't very familiar with the history of Silicon Valley, because this is the most unremarkable thing. 60 years ago personal computers were sci-fi fantasy, and 50 years ago they were a mainstream view in Silicon Valley. 50 years ago usable GUIs were sci-fi fantasy, and 40 years ago they were a mainstream view in Silicon Valley. 40 years ago a global computer network used by almost everyone was sci-fi fantasy, and 30 years ago it was a mainstream view in Silicon Valley. 30 years ago always-internet-connected wearable computing devices were sci-fi fantasy, and 20 years ago they were a mainstream view in Silicon Valley. 20 years ago renewable energy cheaper than coal was a sci-fi fantasy, and 10 years ago it was a mainstream view in Silicon Valley. There were a lot of other sci-fi fantasies that didn't pan out, of course, or at least haven't panned out yet, so this isn't a sign that AGI will definitely happen. Xanadu, the Information Superhighway, graphical programming, virtual reality, freedom, world peace, energy too cheap to meter, the full automation of production, flying cars, and so on. But the entire point of Silicon Valley is that it's a machine for turning sci-fi fantasies into the realities of everyday life.
• deleted 11mo ago
  [deleted]
• vatsachak 11mo ago
  Data centers are good because they are resource hogs that are designed by the some of the most technically gifted people in the US. This will hasten gains in water and energy availability
• tonetegeatinst 11mo ago
  Many arguments I see from people skeptical of AI or other revolutionary theory's seem to overlap. I'm confident the same arguments could have been used in the industrial revolution....the industry rapidly outgrowing the supply of energy, the overinvesting by people hoping they found a golden goose by being first to market etc.
• t43562 11mo ago
  Why is AGI so hungry? Our brains learn on the fly and do amazing things and how much energy do they use? So IMO we certainly haven't hit the "right technology" yet and the attempt to achieve it by spending billions, building nuclear power plants etc, is vulnerable to a technical development. So why should we screw up our environmental situation for something that can obviously be done in a vastly better way on the energy equivalent of 3 square meals a day?
  • ACCount37 11mo ago
    Your brain has more capacity than an average LLM, and a self-supervised training regime etched into it by millions of years of evolution. Its inductive bias is very well tuned for the environment. It still takes decades to build itself up to peak performance. And at its peak? Human brain doesn't actually have an overwhelming advantage over an LLM. It's a mixed bag of advantages and disadvantages. LLMs think fast, and can input and output data much faster than a human. But they struggle to work on the same task for a long time, and have a problem with visual inputs and object manipulation. LLMs have more knowledge in total, but humans have better meta-knowledge, which is useful for hallucination avoidance. LLMs can only learn in context efficiently, but humans learn continuously and retain what they learned. LLMs and humans are currently trading blows when it comes to inference energy efficiency - especially when you account for things like sleep or rest. I don't think there's a "right technology" at all. There may not be a state-change upgrade that gets us x100000 on a dime and goes all the way to an AGI on every smartphone and an ASI in any datacenter worth the name. I expect there to be a lot of little +5% and +10% upgrades that add up over time.
    • t43562 11mo ago
      The history of AI has been to reject human inspired forms and then be forced to adopt them. OK that's a sweeping statement but I think it encapsulates the general attitude quite well for a single sentence. We fly in planes which are not like birds but they have wings and tails and flaps and slats and we're even getting into making the wings warp. Sleep is needed for memory I think so there is a price to pay for continuous learning. It's important to be able to forget too. - to rotate the logfiles so to speak. AGI only interests me because it will be able to understand us - therefor it has to work like us and experience things like us.
• jonahx 11mo ago
  > and exploits and traumatises data workers to make sure ChatGPT doesn’t generate outputs like child sexual abuse material and hate speech or encourage users to self-harm. It's hard to take this seriously, especially with the inflated language. They're essentially doing moderation. It's a job that's been done and needed on internet platforms of all stripes for at least 30 years. Sure, it can been unpleasant work. If it's "traumatizing" you, you shouldn't be doing it. Acting like this is some novel and horrific phenomenon springing from the quixotic pursuit of AGI is ridiculous. It would be needed even if no one believed AGI was possible.
• tmsh 11mo ago
  I believe in AGI because I believe you can estimate where a line is going by its slope. Is there not a way to evaluate the resilience of a rate of change? Like the variance of that rate of change? If so you could almost prove with greater and greater certainty that we will get there.
  • perfmode 11mo ago
    what looks exponential can be the left side of a sigmoid. true or false?
• jgord 11mo ago
  I feel the same frustration, seen from another angle : I think we can use current ML techniques to solve 3 or 4 hard problems in 3D reconstruction, and doing so would unlock a vast amount of value - we could turn lidar scans and photos of buildings and industrial plants into accurate 3D models automatically. BUT I think the bottleneck is _funding_ of small early risky startups to do the needed engineering work. My notes on this : https://quantblog.wordpress.com/2025/10/29/digital-twins-the-missing-pieces/ https://quantblog.wordpress.com/2025/10/29/digital-twins-the... LLMs, GPU datacenters attract all the big money, and the med and small VCs seem to be leaving their money in the bank earning high interest rates, unless there is a slam dunk opportunity with guaranteed traction and MRR growth. We seem to be betting that only the large companies will innovate, when historically this has not been the case - Deepseek is a recent counterexample.
• OBELISK_ASI 11mo ago
  [dead]
• arisAlexis 11mo ago
  Calling it a fantasy is a biased and limited thinking
• brittneyleighba 11mo ago
  AGI can't be trained to develop consciousness when we don't even scientifically understand it yet.
• falcor84 11mo ago
  > the exploitation of data workers and the environment unacceptable If that were true, the last 200 years of civilization would not have happened.