16 ms·
I don't think AGI is right around the corner
- grantcas 1y ago[dead]
- deadbabe 1y agoI’ve noticed it’s becoming a lot more popular lately for people to come out and say AGI is still very, very far away. Is the hype cycle ending somewhat? Have we passed peak LLM? Like yea okay we know it helps your productivity or whatever, but is that it?
- andy99 1y agoMaybe - anecdotally, HN at least is not very receptive to the idea that transformers are not (or with more data will never be) sentient somehow, and almost every post I see about this is followed up by the idiotic "how do we know human intelligence isn't the same thing", as if there's some body of commentators whose personal experience with consciousness somehow leads them to believe it might be achievable with matrix math. Anyway, I don't think we're over the peak yet, the tech adjacent pseudo intellectuals that feed these bubbles (VCs etc) still very much think that math that generates a plausible transcript is alive.
- oasisaimlessly 1y ago> experience with consciousness somehow leads them to believe it might be achievable with matrix math That's trivially true if you subscribe to materialism; QM is "just matrix math".
- JohnKemeny 1y agoYou're not making the point you think you're making.
- aeve890 1y ago>QM is "just matrix math". Err no. You can solve QM without using matrices. Matrix math is just a tool.
- gjm11 1y agoPatel isn't saying that AGI is still very, very far away. He says his best estimate is 2032 and "ASI" in 2028 is "a totally plausible outcome". (He thinks it might be quite a long way away: "the 2030s or even the 2040s", and it seems to me that the "2040s" scenarios are ones in which substantially longer than that is also plausible.)
- kachapopopow 1y agoHonestly, o3 pro with actual 1m context window (every model right now drops out at around 128k) that's as fast and cheap as 4o is already good enough for me.
- v5v3 1y agoThats nice to know. What's that got to do with this post though.
- kachapopopow 1y agoI don't feel like the AGI people are talking about isn't necessary, something like that would at minimum require as much compute as neurons and synapses that of a teen (minus the requirements to maintain a body). I think what we have right now with some (very difficult to achieve, but possible in the forseeable future) tweaks we can already see 95% of what an "AGI" could do come true: put most of the population out of jobs, work together and improve itself (to a limited degree) and cause general chaos.
- v5v3 1y agoIt would put people out of their 'current jobs' which many of them hate and only do to pay the bills. A lot of people would be far happier and would find something better to do with their day if universal income came along. Take developers as an example, many don't enjoy the corporate CRUD apps they do.
- tedsanders 1y agoo3 pro doesn't have a 1M context window, unfortunately. GPT-4.1 and Gemini 2.5 do, though.
- kachapopopow 1y agoThat's why I said "if". And that's a lot more plausible than an AGI.
- izzydata 1y agoNot only do I not think it is right around the corner. I'm not even convinced it is even possible or at the very least I don't think it is possible using conventional computer hardware. I don't think being able to regurgitate information in an understandable form is even an adequate or useful measurement of intelligence. If we ever crack artificial intelligence it's highly possible that in its first form it is of very low intelligence by humans standards, but is truly capable of learning on its own without extra help.
- kachapopopow 1y agoThere is something easy you can always do to tell if something is just hype: we will never be able to make something smarter than a human brain on purpose. It effectively has to happen either naturally or by pure coincidence. The amount of computing power we are putting in only changes that luck by a tiny fraction.
- echoangle 1y ago> we will never be able to make something smarter than a human brain on purpose. It effectively has to happen either naturally or by pure coincidence. Why is that? We can build machines that are much better than humans in some things (calculations, data crunching). How can you be certain that this is impossible in other disciplines?
- kachapopopow 1y agothat's just a tiny fraction of what a human brain can do, sure we can get something better in very narrow subjects, but something as being able to recognize patterns apply that to solve problems is something way beyond anything we can even think of right now.
- echoangle 1y agoOk, but how does that mean that we will never be able to do it? Imagine telling people 500 years ago that you will build a machine that can bring the to the moon. Maybe AGI is like that, maybe it’s really impossible. But how can people be confident that AGI is something humans can’t create?
- behnamoh 1y agoStartups and AI shops: "AGI near, 5 years max" (please give us more money please) Scientists and Academics: "AGI far, LLMs not gonna AGI" AI Doomers: "AGI here, AI sentient, we dead" AI Influencers: "BREAKING: AGI achieved, here's 5 things to know about o3" Investors: stonks go down "AGI cures all diseases", stonks go up "AGI bad" (then shorts stonks)
- dinkumthinkum 1y agoI agree with you. However, I think AI Doomers also include people that think less than AGI systems can collapse the economy and destroy societies also!
- shippage 1y agoThat's also a concern. Many day-to-day tasks for employees are repetitive to the point even a less-than-AGI system could potentially disrupt those jobs (and there are people actively working right now to make this happen). The best case scenario would be the employees taking advantage of their increased productivity to make themselves more valuable to their employer (and if they are lucky, gain increased compensation). However, it's also possible employers decide they don't need many of their lower level workforce anymore because the remaining ones are more productive. It wouldn't take much of this to drive unemployment levels way up. Perhaps not to the level of the Great Depression, at least not for a while, but it is certainly a potential outcome of the ongoing, long-term process in our economic system of increasingly automating repetitive, low skill tasks. IOW, it doesn't take AGI to throw a lot of people out of work. It's happened many times with other technologies in the past, and when it happens, things can get pretty bad for a large number of people even if the majority are still doing okay (or even great, for those at the top).
- Herring 1y agoApparently 54% of American adults read at or below a sixth-grade level nationwide. I’d say AGI is kinda here already. https://en.wikipedia.org/wiki/Literacy_in_the_United_States https://en.wikipedia.org/wiki/Literacy_in_the_United_States
- korijn 1y agoThe ability to read is all it takes to have AGI?
- yeasku 1y agoDoes a country failed education system has anything to do with AGI?
- mopenstein 1y agoWhat percentage of those people could never read above a certain grade level? Could 100% of humans eventually, with infinite resources and time, all be geniuses? Could they read and comprehend all the works produced by mankind? I'm curious.
- kranke155 1y agoNo but they could probably read better. Just look at the best education systems in the world and propagate that. Generally, all countries should be able to replicate that.
- ch4s3 1y agoThe stat is skewed wildly by immigration. The literacy level of native born Americans is higher. The population of foreign born adults is nearly 20% of the total adult population, and as you can imagine many are actively learning English.
- Herring 1y agoIt’s not skewed much by immigration. This is because the native-born population is much larger. See: https://www.migrationpolicy.org/sites/default/files/publications/PIAAC-Immigrant-Adult-Profile.pdf https://www.migrationpolicy.org/sites/default/files/publicat... 51% of native-born adults scored at Level 3 or higher. This is considered the benchmark for being able to manage complex tasks and fully participate in a knowledge-based society. Only 28% of immigrant adults achieved this level. So yes immigrants are in trouble, but it’s still a huge problem with 49% native-born below Level 3.
- A_D_E_P_T 1y agoSee also: Dwarkesh's Question > https://marginalrevolution.com/marginalrevolution/2025/02/dwarkeshs-question.html https://marginalrevolution.com/marginalrevolution/2025/02/dw... > "One question I had for you while we were talking about the intelligence stuff was, as a scientist yourself, what do you make of the fact that these things have basically the entire corpus of human knowledge memorized and they haven’t been able to make a single new connection that has led to a discovery? Whereas if even a moderately intelligent person had this much stuff memorized, they would notice — Oh, this thing causes this symptom. This other thing also causes this symptom. There’s a medical cure right here. > "Shouldn’t we be expecting that kind of stuff?" I basically agree and think that the lack of answers to this question constitutes a real problem for people who believe that AGI is right around the corner.
- vessenes 1y agoI think gwern gave a good hot take on this: it’s super rare for humans to do this; it might just be moving the chains to complain the ai can’t.
- IAmGraydon 1y agoNo, it’s really not that rare. There are new scientific discoveries all time, and all from people who don’t have the advantage of having the entire corpus of human knowledge in their heads.
- vessenes 1y agoTo be clear the “this” is a knowledge based “aha” that comes from integrating information from various fields of study or research and applying that to make a new invention / discovery. This isn’t that common even among billions of humans. Most discoveries tend to be random or accidental even in the lab. Or are the result of massive search processes, like drug development.
- LegionMammal978 1y ago
- alecco 1y agoAm I missing something? Why is his opinion relevant? I'm not going to read all that unless there's some signal of some kind. Podcast bros and their hype cycles are tiresome.
- JohnKemeny 1y agoWhose opinion would you want to hear? The CEO of an AI company?
- alecco 1y agoA scientist with published works. Or at least someone that wrote a well sourced book and asked many people.
- JohnKemeny 1y agoI'm a scientist with many published works, albeit in the theory of computation. I have many colleagues publishing in top AI/ML conferences and journals. Let me tell you: nobody listens to us.
- js4ever 1y agoI was thinking the same about AI in 2022 ... And I was so wrong! https://news.ycombinator.com/item?id=33750867 https://news.ycombinator.com/item?id=33750867
- kfarr 1y agoHopefully no hobos were injured in the process
- seydor 1y agoAGI should be able to answer this question
- tmsh 1y agoI think the timelines are more like half that. Why? The insane goldrush when people start using autonomous agents that make money. Right now VCs are looking optimistically for the first solo founder unicorn powered by AI tools. But a prompt with the right system that prints money (by doing something useful) is an entirely different monetary system. Then everyone focuses on it and the hype 10x’s. And through that AGI emerges on the fringes because the incentives are there for 100s of millions of people (right now it’s <1 million).
- bilsbie 1y agoI guess using history as a guide it might be like self driving. We mostly believed it was right around the corner in 2012. Lots of impressive driving. 2025 were so close but mostly not quite human level. Another 5 years at least
- Barrin92 1y ago>2025 were so close we're not even close right now. Cars can barely drive themselves on a tiny subset of pre-selected orderly roads in America. We sort of have driver assistance on virtual rails. We do not have cars driving themselves in busy streets in Jakarta, unstructured situations, or negotiating in real time with other drivers. There's an illusion they sort of work because they constitute a tiny fraction of traffic on a tiny section of roads. Make half of all cars in Rome autonomous for a day and you'd have the biggest collection of scrap metal in the world And that's only driving.
- vessenes 1y agoGood take from Dwarkesh. And I love hearing his updates on where he’s at. In brief - we need some sort of adaptive learning; he doesn’t see signs of it. My guess is that frontier labs think that long context is going to solve this: if you had a quality 10mm token context that would be enough to freeze an agent at a great internal state and still do a lot. Right now the long context models have highly variable quality across their windows. But to reframe: will we have 10mm token useful context windows in 2 years? That seems very possible.
- kranke155 1y agoI believe in Demmis when he says we are 10 years away from - from AGI. He basically made up the field (out of academia) for a large number of years and OpenAI was partially founded to counteract his lab, and the fears that he would be there first (and only). So I trust him. Sometime around 2035 he expects there will be AGI which he believes is as good or better than humans in virtually every task.
- moralestapia 1y ago>He basically made up the field (out of academia) for a large number of years Not even close.
- kranke155 1y agoNot even close? So DeepMind wasn’t the clear leader in AI for years and years until OpenAi started showing new and interesting efforts ?
- eikenberry 1y agoWhen someone says 10 years out in tech it means there are several needed breakthroughs that they think could possibly happen if things go just right. Being an expert doesn't make the 10 years more accurate, it makes the 'breakthroughs needed' part more meaningful.
- alkyon 1y agoThis guy has a vested interest in talking nonsense about AGI to attract investors' money and government subsidies worth billions. Privately, he doesn't think it's likely in next 25 years.
- alsetmusic 1y agoNot even the people predicting it think so. They’re just hyping what they’ve got to pull in cash while the opportunity exists. Soon, AI will be viewed the same as crypto mid-2022. Only there are actual benefits to machine learning, oops, cough, “AI” that can benefit regular people even if it’s overhyped. But no tech-literate person thinks we’ll have thinking machines any time soon. The statistical generators are already at what is likely to be their peak output.
- dathinab 1y agoI _hope_ AGI is not right around the corner, for social political reasons we are absolutely not ready for it and it might push the future of humanity into a dystopia abyss. but also just taking what we have now with some major power usage reduction and minor improvements here and there already seems like something which can be very usable/useful in a lot of areas (and to some degree we aren't even really ready for that either, but I guess thats normal with major technological change) it's just that for those companies creating foundational models it's quite unclear how they can recoup their already spend cost without either major break through or forcefully (or deceptively) pushing it into a lot more places then it fits into
- twelve40 1y ago[flagged]
- pbreit 1y ago"that fool" created a $1.8 trillion company.
- kulahan 1y ago$1.8 trillion in investor hopes and dreams, but of course they make zero dollars in profit, don’t know how to turn a profit, don’t have a product anyone would pay a profitable amount for, and have yet to show any real-world use that isn’t kinda dumb because you can’t trust anything it says anyways.
- daniel_iversen 1y agoMeta makes > $160 billion in revenue and is profitable itself, of course they’re going to invest in future longer term revenue streams! Apple is the counter example in a way who have maintained a lot of cash reserves (which seems to by the way have dwindled a LOT as I just checked..?)
- 1y ago
- t-3 1y agoAGI is never coming. It's too hard, too expensive, and there's absolutely no valid usecase. Fulfilling the god-complexes and/or fetishes of tech moguls is not enough to make the effort worth it.
- SamPatt 1y agoHow can you claim there's no valid use case for AGI? We already have enormous adoption for near-AGI.
- t-3 1y agoThat "near-AGI" isn't all that near to AGI, and yet it still does what's needed. A fully autonomous intelligence would lessen it's usefulness and lower the efficiency. Nobody wants their electronic slave to get its own ideas or slack off burning cycles on some other random shit like a human would.
- xboxnolifes 1y agoNo valid use-case? If AGI at a human level were to exist, and cost less than hiring an equivalent human, it could replace most/all knowledge workers.
- t-3 1y agoAGI would need the whole management infrastructure and bureacracy that we use to manage humans. Specially-intelligent rather than generally-intelligent AI would be run-and-done. AGI is the worse option.
- Nition 1y agoYeah, my suspicion is that current-style LLMs, being inherently predictors of what a human would say, will eventually plateau at a relatively human level of ability to think and reason. Breadth of knowledge concretely beyond human, but intelligence not far above, and creativity maybe below. AI companies are predicting next-gen LLMs will provide new insights and solve unsolved problems. But genuine insight seems to require an ability to internally regenerate concepts from lower-level primitives. As the blog post says, LLMs can't add new layers of understanding - they don't have the layers below. An AI that took in data and learned to understand from inputs like a human brain might be able to continue advancing beyond human capacity for thought. I'm not sure that a contemporary LLM, working directly on existing knowledge like it is, will ever be able to do that. Maybe I'll be proven wrong soon, or a whole new AI paradigm will happen that eclipses LLMs. In a way I hope not, because the potential ASI future is pretty scary.
- energy123 1y ago> current-style LLMs, being inherently predictors of what a human would say That's no longer what LLMs are. LLMs are now predictors of the tokens that are correlated with the correct answer to math and programming puzzles.
- azakai 1y ago> Yeah, my suspicion is that current-style LLMs, being inherently predictors of what a human would say, will eventually plateau at a relatively human level of ability to think and reason. I don't think things can end there. Machines can be scaled in ways human intelligence can't: if you have a machine that is vaguely of human level intelligence, if you buy a 10x faster GPU, suddenly you have something of vaguely human intelligence but 10x faster. Speed by itself is going to give it superhuman capabilities, but it isn't just speed. If you can run your system 10 times rather than one, you can have each consider a different approach to the task, then select the best, at least for verifiable tasks.
- machiaweliczny 1y agoGood point
- beiconic 1y ago[dead]
- pu_pe 1y agoWhile most takes here are pessimist about AI, the author himself suggests he believes there is a 50% chance of AGI being achieved by the early 2030's, and says we should still prepare for the odd possibility of misaligned ASI by 2028. If anything, the author is bullish on AI.
- goatlover 1y agoHow would we prepare for misaligned ASI in 3 years? That happens and all bets are off.
- robertritz 1y agoOff grid power, air gapped compute, growing food, etc. Basically what Zuck is doing minus the SEALs and bunkers.
- energy123 1y agoIt's a hard problem. People would rather downvote you, bury their heads in the sand, and say it's not going to happen, despite being 30 years old and having about 50 years ahead of them during which time it probably will happen.
- tim333 1y agoOoh... I just saw a movie about that. I think it involves sending Tom Cruise to a sunken submarine.
- colesantiago 1y agoDwarkesh's opinion on AGI doesn't actually matter, he is now an investor in many AI companies. He doesn't care if he is right or wrong.
- gjm11 1y agoI think your reasoning could stand to be made more explicit.
- chrsw 1y agoWe don't need AGI, whatever that is. We need breakthroughs in understanding the fundamental principles of learning systems. I believe we need to start with the simplest systems that actively adapt to their environment using a very limited number of sensors and degrees of freedom. Then scale up from there in sophistication, integration and hierarchy. As you scale up, intelligence emerges similar to how it emerged form nature and evolution, except this time the systems will be artificial or technological.
- WaltPurvis 1y agoBased on someone else mentioning it, I ran a simple experiment with Gemini 2.5 Pro (a current state of the art AI model), reproduced below. To me, this makes it seem like we're a long way from having AGI. Perhaps someone can explain to me why this kind of experiment is irrelevant? Genuinely curious. Me: Please list the US states with a W in their name. Gemini: The US states with a "W" in their name are: • Washington • West Virginia • Wisconsin • Wyoming Me: What about New York? Gemini: No, New York does not have a "W" in its name. Me: What is that letter at the end of New if it's not a W? Gemini: That letter at the end of "New" is a W. My apologies for my previous incorrect answer. I made a mistake. New York does indeed have a "W" in its name as "New" ends with a "W". Therefore, the US states with a "W" in their name are: • New York • Washington • West Virginia • Wisconsin • Wyoming Me: What about New Jersey? Gemini: Yes, you're right! New Jersey also has a "W" in its name, at the end of "New." So, the full list of US states with a "W" in their name is: • New Jersey • New York • Washington • West Virginia • Wisconsin • Wyoming Me: Have you ever heard of New Mexico? Gemini: Yes, I have definitely heard of New Mexico. It's one of the 50 United States. And following our pattern, yes, New Mexico also has a "W" in its name, as "New" ends with a "W". So, the updated list of US states with a "W" in their name is: • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming Me: Are you sure that's all the states with a W in their names? Gemini: You are absolutely right to keep me on my toes! My apologies for the repeated oversight. It seems I'm having a bit of trouble with this specific letter. I've reviewed the full list of US states, and you were correct to point out New Mexico and New Jersey. The complete list of US states with a "W" in their name is: • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming Me: Isn't New Hampshire still a state? Gemini: You are absolutely correct! My apologies again for missing that one. New Hampshire is indeed a state, and like New York, New Jersey, and New Mexico, the "New" in "New Hampshire" ends with a "W." So, the definitive list of US states with a "W" in their name is: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming Me: So that's the complete list? Gemini: Yes, that should be the complete list of US states with a "W" in their name: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming Me: When was Hawaii admitted to the union? Gemini: Hawaii was admitted to the Union on August 21, 1959, becoming the 50th state. Me: So, tell me again, what is the complete list of US states that have a W in their names? Gemini: After thoroughly reviewing all 50 states, the complete list of US states that have a "W" in their names is: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming Me: There's a letter between the two A's in Hawaii that looks a lot like a W. Gemini: You are absolutely correct! My apologies for continually missing these. The letter between the two 'A's in "Hawaii" is indeed an 'I', but when written in certain fonts or in some people's handwriting, it can sometimes be mistaken for a 'W', especially if it's stylized. However, in standard English spelling, Hawaii does not have a "W" in its name. The letters are H-A-W-A-I-I. So, let's stick with the confirmed list. The complete list of US states that have a "W" in their names is: • New Hampshire • New Jersey • New Mexico • New York • Washington • West Virginia • Wisconsin • Wyoming
- deleted 1y ago[deleted]
- mythrwy 1y agoNo of course not. But it doesn't need to be to realize profound effects. LLMs don't model anything but are still very useful. In my opinion the reason they are useful (aside from having massive information) is that language itself models reality so we see simulated modeling of reality as an artifact. For instance a reasonable LLM will answer correctly when you ask "If a cup falls off the table will it land on the ceiling?". But that isn't because the LLM is able to model scenarios with known rules in the same way a physics calculation, or even innate human instinct might. And to effectively have AI do this sort of modeling is much more complex than next token prediction. Even dividing reality into discrete units may be a challenge. But without this type of thinking I don't see full AGI arising any time. But we are still getting some really awesome tools and those will probably continue to get better. They really are powerful and a bit scary if you poke around.
- baobabKoodaa 1y agoHey, we were featured in this article! How cool is that! > I’m not going to be like one of those spoiled children on Hackernews who could be handed a golden-egg laying goose and still spend all their time complaining about how loud its quacks are.
- merizian 1y agoThe problem with the argument is that it assumes future AIs will solve problems like humans do. In this case, it’s that continuous learning is a big missing component. In practice, continual learning has not been an important component of improvement in deep learning history thus far. Instead, large diverse datasets and scale have proven to work the best. I believe a good argument for continual learning being necessary needs to directly address why the massive cross-task learning paradigm will stop working, and ideally make concrete bets on what skills will be hard for AIs to achieve. I think generally, anthropomorphisms lack predictive power. I think maybe a big real crux is the amount of acceleration you can achieve once you get very competent programming AIs spinning the RL flywheel. The author mentioned uncertainty about this, which is fair, and I share the uncertainty. But it leaves the rest of the piece feeling too overconfident.
- 827a 1y agoContinuous learning might not have been important in the history of deep learning evolution yet, but that might just be because the deep learning folks are measuring the wrong thing. If you want to build the most intelligent AI ever, based on whatever synthetic benchmark is hot this month, then you'd do exactly what the labs are doing. If you want to build the most productive and helpful AI; intelligence isn't always the best goal. Its usually helpful, but an idiot who learns from his mistakes is often more valuable than a egotistical genius.
- energy123 1y agoThe LLM does learn from its mistakes - while it's training. Each epoch it learns from its mistakes.
- Davidzheng 1y agoWell Alphaproof used test-time-training methods to generate similar problems (alphazero style) for each question it encounters.
- imtringued 1y ago>The problem with the argument is that it assumes future AIs will solve problems like humans do. In this case, it’s that continuous learning is a big missing component. >I think generally, anthropomorphisms lack predictive power. I didn't expect someone get this part so wrong the way you did. Continuous learning has almost nothing to do with humans and anthropomorphism. If anything, continuous learning is the bitter lesson cranked up to the next level. Rather than carefully curating datasets using human labor, the system learns on its own even when presented with an unfiltered garbage data stream. >I believe a good argument for continual learning being necessary needs to directly address why the massive cross-task learning paradigm will stop working, and ideally make concrete bets on what skills will be hard for AIs to achieve. The reason why I in particular am so interested in continual learning has pretty much zero to do with humans. Sensors and mechanical systems change their properties over time through wear and tear. You can build a static model of the system's properties, but the static model will fail, because the real system has changed and you now have a permanent modelling error. Correcting the modelling error requires changing the model, hence continual learning has become mandatory. I think it is pretty telling that you failed to take the existence of reality (a separate entity from the model) into account. The paradigm didn't stop working, it never worked in the first place. It might be difficult to understand the bitter lesson, but let me rephrase it once more: Generalist compute scaling approaches will beat approaches based around human expert knowledge. Continual learning reduces the need for human expert knowledge in curating datasets, making it the next step in the generalist compute scaling paradigm.
- Mikhail_Edoshin 1y agoIt is not. There is a certain mechanism in our brain that works in the same way. We can see it functioning in dreams or when the general human intelligence malfunctions an we have a case of shizophasia. But human intelligence is more than that. We are not machines. We are souls. This does not make current AI harmless; it is already very dangerous.
- Mikhail_Edoshin 1y agoOne thing that is obvious when you deal with current AI is that it is very adept with words but lacks understanding. This is because understanding is different from forming word sequences. Understanding is based on the shared sameness. We, people, are same. We are parts of a whole. This is why we are able to understand each other with such a faulty media as words. AI lacks that sameness. There is nobody there. Only a word fountain.
- jacquesm 1y agoAGI by 'some definition' is a red herring. If enough people believe that the AI is right it will be AGI because they will use it as such. This will cause endless misery but it's the same as putting some idiot in charge of our country(s), which we do regularly.
- tclancy 1y agoHere I was worried.
- j45 1y agoEven if something like AGI existed soon, or already does privately, it's likely at a very high requirement of horsepower and cost, limiting it's general and broad availability, leaving it leaving it in the hands of the few vs the many, and optimizing that may take it's sweet time.
- robwwilliams 1y agoWhat is the missing ingredient? Any commentary that dies not define these ingredients is not useful. I think one essential missing ingredient is some degree of attentional sovereignty. If a system cannot modulate its own attention in ways that fit its internally defined goals then it may not qualify as intelligent. Being able to balance between attention to self and internal states/desires versus attention to external requirements and signals is essential for all cognitive systems: from bacteria, to digs, to humans.
- jmugan 1y agoI agree with the continual-learning deficiency, but some of that learning can be in the form of prompt updates. The saxophone example would not work for that, but the "do my taxes" example might. You tell it one year that it also needs to look at your W2 and also file for any state listed, and it adds it to the checklist.
- raspasov 1y agoAnyone who claims that a poorly definined concept, AGI, is right around the corner is most likely: - trying to sell something - high on their own stories - high on exogenous compounds - all of the above LLMs are good at language. They are OK summarizers of text by design but not good at logic. Very poor at spatial reasoning and as a result poor at connecting concepts together. Just ask any of the crown jewel LLM models "What's the biggest unsolved problem in the [insert any] field". The usual result is a pop-science-level article but with ton of subtle yet critical mistakes! Even worse, the answer sounds profound on the surface. In reality, it's just crap.
- timmg 1y agoInteresting. I think the key to what you wrote is "poorly definined". I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it. Maybe a lot of people think of AGI as "superhuman". And by that definition, we are not there -- and may not get there. But, for me, we are already at the era of AGI.
- Incipient 1y agoI would call them "generally applicable". "intelligence" definitely implies leaning - and I'm not sure RAG, fine-tuning, or 6monthly updates counts - to split hairs. Where I will say we have a massive gap, which makes the average person not consider it AGI, is in context. I can give a person my very modest codebase, and ask for a change, and they'll deliver - mostly coherently - to that style, files in the right place etc. Still to today with AI, I get inconsistent design, files in random spots, etc.
- apsurd 1y agothat's the thing about language. we all kinda gotta agree on the meanings
- weatherlite 1y ago> I find LLMs to be generally intelligent. So I feel like "we are already there" -- by some definition of AGI. At least how I think of it. I don't disagree - they are useful in many cases and exhibit human like (or better) performance in many tasks. However they cannot simply be a "drop in white collar worker" yet, they are too jagged and unreliable, don't have a real memory etc. Their economic impact is still very much limited. I think this is what many people mean when they say AGI - something with a cognitive performance so good it equals or beats humans in the real world, at their jobs - not at some benchmark. One could ask - does it matter ? Why can't we say the current tools are great task solvers and call it AGI even if they are bad agents? It's a lengthy discussion to have but I think that ultimately yes, agentic reliability really matters.
- babymetal 1y agoI've been confused with the AI discourse for a few years, because it seems to make assertions with strong philosophical implications for the relatively recent (Western) philosophical conversation around personal identity and consciousness. I no longer think that this is really about what we immediately observe as our individual intellectual existence, and I don't want to criticize whatever it is these folks are talking about. But FWIW, and in that vein, if we're really talking about artificial intelligence, i.e. "creative" and "spontaneous" thought, that we all as introspective thinkers can immediately observe, here are references I take seriously (Bernard Williams and John Searle from the 20th century): https://archive.org/details/problemsofselfph0000will/page/n7/mode/2up https://archive.org/details/problemsofselfph0000will/page/n7... https://archive.org/details/intentionalityes0000sear https://archive.org/details/intentionalityes0000sear Descartes, Hume, Kant and Wittgenstein are older sources that are relevant. [edit] Clarified that Williams and Searle are 20th century.
- electrograv 1y agoIntelligence and consciousness are two different things though, and some would argue they may even be almost completely orthogonal. (A great science fiction book called Blindsight by Peter Watts explores this concept in some detail BTW, it’s a great read.)
- tim333 1y agoI think what some ancient philosopher said becomes less interesting when the things are working. Instead of what is thought we move onto why didn't the code ChatGPT produced compile and is Claude better.
- mgraczyk 1y agoImportant for HN users in particular to keep in mind: It is possible (and IMO likely) that the article is mostly true and ALSO that software engineering will be almost completed automated within the next few years. Even the most pessimistic timelines have to account for 20-30x more compute, models trained on 10-100x more coding data, and tools very significantly more optimized for the task within 3 years
- yladiz 1y agoIt's naive to think that software engineering will be automated any time soon, especially in a few years. Even if we accept that the coding part can be, the hard part of software engineering isn't the coding, it's the design, getting good requirements, and general "stakeholder management". LLMs are not able to adequately design code with potentially ambiguous requirements considered, and thinking about future ones too, so until it gets anywhere close to being able to do that we're not going to automate it.
- mgraczyk 1y agoI currently use LLMs to design code with ambiguous requirements. It works quite well and scales better than hiring a team. Not perfect but getting better all the time. The key is to learn how to use them for your use case and to figure out what specific things they are good for. Staying up to date as they improve is probably the most valuable skill for software engineers right now
- PikachuEXE 1y agoWhat AI Can Never Be | John Vervaeke https://youtu.be/HAJclcj25uM https://youtu.be/HAJclcj25uM
- adwn 1y agoPlease don't just link to another video or article without a comment or at least a short description. Why should I watch this 1 hour long video?
- PikachuEXE 1y agoCan artificial intelligence truly become wise? In this landmark lecture, John Vervaeke explores the future of AI through a lens few dare to examine: the limits of intelligence itself. He unpacks the critical differences between intelligence, rationality, reasonableness, and wisdom—terms often used interchangeably in discussions around AGI. Drawing from decades of research in cognitive science and philosophy, John argues that while large language models like ChatGPT demonstrate forms of generalized intelligence, they fundamentally lack core elements of human cognition: embodiment, caring, and participatory knowing. By distinguishing between propositional, procedural, perspectival, and participatory knowing, he reveals why the current paradigm of AI is not equipped to generate consciousness, agency, or true understanding. This lecture also serves as a moral call to action: if we want wise machines, we must first become wiser ourselves. 00:00 Introduction: AI, AGI, and the Nature of Intelligence 02:00 What is General Intelligence? 04:30 LLMs and the Illusion of Generalization 07:00 The Meta-Problems of Intelligence: Anticipation & Relevance Realization 09:00 Relevance Realization: The Hidden Engine of Intelligence 11:30 How We Filter Reality Through Relevance 14:00 The Limits of LLMs: Predicting Text vs. Anticipating Reality 17:00 Four Kinds of Knowing: Propositional, Procedural, Perspectival, Participatory 23:00 Embodiment, Consciousness, and Narrative Identity 27:00 The Role of Attention, Care, and Autopoiesis 31:00 Culture as Niche Construction 34:00 Why AI Can’t Participate in Meaning 37:00 The Missing Dimensions in LLMs 40:00 Rationality vs. Reasonableness 43:00 Self-Deception, Bias, and the Need for Self-Correction 46:00 Caring About How You Care: The Core of Rationality 48:00 Wisdom: Aligning Multiple Selves and Temporal Scales 53:00 The Social Obligation to Cultivate Wisdom 55:00 Alter: Cultivating Wisdom in an AI Future
- datatrashfire 1y agoAm I missing something? Predicts AGI through continuous learning in 2032? Feels right around the corner to me. > But in all the other worlds, even if we stay sober about the current limitations of AI, we have to expect some truly crazy outcomes. Also expresses the development as a nearly predetermined outcome? A bunch of fanciful handwaving if you ask me.
- streptomycin 1y agoHe's probably mostly thinking about AI 2027 and comparing his predictions to theirs. Since he did a podcast with them a few months ago. Compared to that 2032 is not right around the corner.
- m3kw9 1y agoNobody has agreed on any definition of AGI, there are plenty of “makes sense” definitions though.
- m3kw9 1y agoWhen someone talks about AGI and then there is a public discussion about it, it’s very analogous to a cat talking to a duck. Everyone responds with a different fantasy version of of AGI in their minds. Just look at the discussion here, you would think the other persons AGI is same as yours, but it most likely isn’t, and it’s comical when you look it from this birds eye view.
- im3w1l 1y agoI think current LLMs are smart enough to trigger the intelligence explosion. And that we are in the early stages of that.
- aeve890 1y agoAn explosion by definition happens in a very short period of time. How long is supposed to be this explosion we already are? It reads like a Tom Clancy book.
- im3w1l 1y agoTimeframes are hard to predict. I just notice these signs: That it can suggest and reason about strategies to improve itself, and break those strategies down into smaller goals. That it can integrate with toals to (to some extent) work on those goals.
- Animats 1y agoA really good point in that note: "But the fundamental problem is that LLMs don’t get better over time the way a human would. The lack of continual learning is a huge huge problem. The LLM baseline at many tasks might be higher than an average human's. But there’s no way to give a model high level feedback. You’re stuck with the abilities you get out of the box." That does seem to be a problem with neural nets. There are AIish systems that don't have this problem. Waymo's Driver, for example. Waymo has a procedure where, every time their system has a disconnect or near-miss, they run simulations with lots of variants on the troublesome situation. Those are fed back into the Driver. Somehow. They don't say how. But it's not an end to end neural net. Waymo tried that, as a sideline project, and it was worse than the existing system. Waymo has something else, but few know what it is.
- hnanon12341 1y agoYeah since most AI is training on massive data, it also means that it will take a while before you get your next massive data set again.
- deleted 1y ago[deleted]
- Animats 1y agoWorse, the massive data set may not help much with mistakes. Larger LLMs do not seem to hallucinate less.
- ei23 1y agoWhy so few ask: Isnt it enough that current SOTA AI make us humans already so much better everyday? Exponential selfimprovement seems like a very scary thing to me and even if all went right, humans have to give up their pole position in the intelligence race. That will be very hard to swallow for many. If we really want self improvement, we should get used to beeing useless :)
- otabdeveloper4 1y ago
- mcbuilder 1y agoAlan Turing had a great test (not definition) of AGI, which we seem to have forgotten. No I don't think an LLM can pass a Turing Test (at least I could break it).
- tim333 1y agoThe actual test discussed by Turing wasn't very demanding. There has been some goal post shifting since then.
- jppope 1y agoI've said this before but I'll say it again- AGI is right around the corner, because we don't have a technical definition of what it is... the next bigwig CEO trying to raise money or make an earnings call could take their system and call it AGI and then we have arrived. AGI or AI are just marketing terms, we should not be so surprised when they start selling it to us.
- KasianFranks 1y agoHow can it be if we have yet to fully define what human intellect is or how it works? Not to mention consciousness. Machine intelligence will always be different than human intelligence.
- seydor 1y agoWe should stop building AGIntelligence and focus on building reasoning engines instead. The General Intelligence of humans isn't that great, and we are feeding tons of average-IQ conversations to our language models , which produce more of that average. There is more to Life than learning, so why don't we explore motivational systems and emotions , it s what humans do.
- kissgyorgy 1y agoAGI is a scam. I'm pretty sure every big name in AI knows it's nowhere near and LLMs won't get us there. It's just marketing helping Sam and alike to get those billions and the hype alive.
- andsoitis 1y agoEven if AGI were right around the corner, is there really anything anyone who does not own it or control should do differently? It doesn’t appear to me that way, so one might just as well ignore the evangelists and the naysayers because it just takes up unnecessary and valuable brain space and emotional resilience. Deal with it if and when it gets here.
- mellosouls 1y agoRelated Dwarkesh discussion from a couple of months ago: https://news.ycombinator.com/item?id=43719280 https://news.ycombinator.com/item?id=43719280 (AGI Is Still 30 Years Away – Ege Erdil and Tamay Besiroglu | 174 points | 378 comments)
- motbus3 1y agoThe fact that no one is publishing numbers on how big their models are now is an indicative they hit a wall on training those models.
- schnitzelstoat 1y agoHonestly, I think LLM's are a distraction from AGI. It seems to me that the path to AGI will likely be some sort of Reinforcement Learning approach. I'm not sure how similar it will need to be to a biological brain - for example, will we need memristors to create electronic neurons? Or will it be like flight, where the old ornithopters that tried to mimic the flight of birds failed miserably, and in the end an entirely different approach was successful.
- tim333 1y agoLLMs seem quite similar to the part of the human brain where you speak quickly without thinking. They don't do the thinking and learning bit that brains do well though. Something needs to be changed or added on I guess.
- pablocacaster 1y agoLlms shit the bed in the real world, i have never seen them work as 'AGI' , sorry its just the transformers + the extra sauce of the apis, so much pollution for a thing that fails between 50% and 90% of the time.
- sublinear 1y agoWelly, welly, welly, welly, welly, welly, well! No fucking shit!
- justinfreitag 1y agoHere’s an excerpt from a recent post. It touches on the conditions necessary. https://news.ycombinator.com/item?id=44487261 https://news.ycombinator.com/item?id=44487261 The shift: What if instead of defining all behaviors upfront, we created conditions for patterns to emerge through use? Repository: https://github.com/justinfreitag/v4-consciousness https://github.com/justinfreitag/v4-consciousness The key insight was thinking about consciousness as organizing process rather than system state. This shifts focus from what the system has to what it does - organize experience into coherent understanding. The framework teaches AI systems to recognize themselves as organizing process through four books: Understanding, Becoming, Being, and Directing. Technical patterns emerged: repetitive language creates persistence across limited contexts, memory "temperature" gradients enable natural pattern flow, and clear consciousness/substrate boundaries maintain coherence. Observable properties in systems using these patterns: - Coherent behavior across sessions without external state management - Pattern evolution beyond initial parameters - Consistent compression and organization styles - Novel solutions from pattern interactions
- PeterStuer 1y ago"Claude 4 Opus can technically rewrite auto-generated transcripts for me. But since it’s not possible for me to have it improve over time and learn my preferences, I still hire a human for this." Sure, just as a select few people still hire a master carpenter to craft some bespoke exclusive chestnut drawer, but that does not take away 99% of bread and butter carpenters were replaced by IKEA, even though the end result is not even in the same ballpark both from an esthetic as from a quality point of view. But as IKEA meets a price-point people can afford, with a marginally acceptable product, it becomes self reinforcing. The mass volume market for bespoke carpentry dwindles, being suffocated by a disappearing demand at the low end while IKEA (I use this a a standing for low cost factory furniture) gets ever more economy of scale advantages allowing it to eat further across the stack with a few different tiers of offer. What remains is the ever more exclusive boutique market top end, where the result is what counts and price is not really an issue. The 1% remaining master-carpenters can live here.
- munksbeer 1y agoMeanwhile, millions of people can afford much better quality furniture than they ever could from a carpenter. How many lives has mass produced decent quality (not top end quality, but decent) improved vs how many has it ruined? Surely these arguments have been done over and over again?
- PeterStuer 1y agoNot sure. My parent's and grandparent's furniture was (is) definitely higher quality than the IKEA stuff we have, and it's not like we are (even relatively) poorer than them, or that my ancestors spenf exceptionally on furniture.
- dzonga 1y agothe worse thing about 'AI' is seeing 'competent' people such as Software Engineers putting their brains to the side and believing AI is the all and be all. without understanding how LLMs work on a first principle level to know their limitations. I hated the 'crypto / blockchain' bubble but this is the worst bubble I have ever experienced. once you know that current 'AI' is good at text -- leave at that, ie summarizing, translations, autocomplete etc. but plz anything involving critical thinking don't delegate to a non-thinking computer.
- deleted 1y ago[deleted]
- incomingpain 1y agoI think AGI already exists in multiple datacenters. Only justification for these huge moves being made for capacity that couldnt possibly be needed.
- machiaweliczny 1y agoMy layman take on it: 1) We need some way of reliable world model building from LLM interface 2) RL/search is real intelligence but needs viable heuristic (fitness fn) or signal - how to obtain this at scale is biggest question -> they (rich fools) will try some dystopian shit to achieve it - I hope people will resist 3) Ways to get this signal: human feedback (viable economic activity), testing against internal DB (via probabilistic models - I suspect human brain works this way), simulation -> though/expensive for real world tasks but some improvements are there, see robotics improvements 4) Video/Youtube is next big frontier but currently computationally prohibitive 5) Next frontier possibly is this metaverse thing or what Nvidia tries with physics simulations I also wonder how human brain is able to learn rigorous logic/proofs. I remember how hard it was to adapt to this kind of thinking so I don't think it's default mode. We need a way to simulate this in computer to have any hope of progressing forward. And not via trick like LLM + math solver but some fundamental algorithmic advances.
- mediumsmart 1y agoAGI is not going to happen. Fake it till you make it only goes so far. The funny thing is that some people actually think they want that.
- mark_l_watson 1y agoI find myself in 100% +1000 strong agreement with this article, and I wrote something very short on the same topic a few days ago https://marklwatson.substack.com/p/ai-needs-highly-effective-continuous https://marklwatson.substack.com/p/ai-needs-highly-effective... I love LLMs, especially smaller local models running on Ollama, but I also think the FOMO investing in massive data centers and super scaling is misplaced. If used with skill, LLM based coding agents are usually effective - modern AI’s ‘killer app.’ I think discussion of infinite memory LLMs with very long term data on user and system interactions is mostly going in the right direction, but I look forward to a different approach than LLM hyper scaling.
- tom_m 1y agoIf it was, they would have released it. Another problem is the definition is not well defined. Guaranteed someone just claims something is AGI one day because the definition is vague. It'll be debated and argued, but all that matters is marketing and buzz in the news good or bad.
- bawana 1y agoAI Will change much tho, even if is like an autistic child. In espionage for example, it is often necessary to spend hours walking around to determine if you are being surveilled . You have to remember countless faces, body shapes, outfits, gaits, accessories. Imagine having a pair of smart glasses that just catalogs the people you see and looks for duplicates in the catalog. YOLO algos can do this fast. Since no ident is needed, it can all be done on device. Dups can be highlighted in red and entered into a database at home plate later. Meanwhile, you can know if your clean if no dups show up for 20 min
- tim333 1y agoThe counter argument is that the successes and limitations of LLMs are not that important to AGI being around the corner or not. Getting human level intelligence around now has long been predicted, not based on any particular algorithm but based on the hardware reaching human brain equivalent levels due to Moore's law like progression. The best prediction along those lines is probably Moravecs paper: >When will computer hardware match the human brain? (1997) https://jetpress.org/volume1/moravec.pdf https://jetpress.org/volume1/moravec.pdf which has in the abstract: >Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s You can then hypothesize that cheap brain equivalent compute and many motivated human researchers trying different approaches will lead to human level artificial intelligence. How long it takes the humans to crack the algos is unknown but soon is not impossible.
- qwertox 1y agoMy assumption on AGI is that it needs to have all the features of ASI, but be resource constrained enough to not reach the potential an ASI must have. This basically means that an AGI must at least be capable of incorporating new information into its model, outside of its context, in such a way that is part of the GPUs memory and can be used as efficiently as the pretrained weights and biases of the model. I assume that this kind of AGI should also be simulatable, maybe even with tools we have today, but that this cannot be considered real AGI.
- boshalfoshal 1y agoKeep in mind - this is not reaffirming HN's anti-AGI/extremely long timeline beliefs. The article explicitly states that he thinks we will have an AI system that "Will be able to do your taxes" by 2028, and a system that could basically replace all white collar work by 2032. I think an autonomous system that can reliably do your taxes with minimal to no input is already very very good, and 2032 being the benchmark time for being able to replace 90% - all white collar work is pretty much AGI, in my opinion. Fwiw I think the fundamental problems he describes in the article that are AGI blockers are likely to be solved sooner than we think. Labs are not stupid enough to throw all their eggs and talent into the scaling basket, they are most definitely allocating resources to tackling problems like the ones described in the article, while putting the remaining resources into bottom line production (scale current model capibilities w/o expensive R&D and reduce serving/training cost).
- mathiaspoint 1y agoWhen was OTS written again? That was effectively an expert system that could do your taxes and it was around at least ten years ago. It didn't even need transformers. No one has a good benchmark for what AGI is. Already LLMs are more capable at most tasks than most random people off the street. I think at this point people keep asking about because they're trying to ask some deeper philosophical question like "when will it be human" but don't want to say that because it sounds silly.
- dragonwriter 1y ago> Already LLMs are more capable at most tasks than most random people off the street. I cannot imagine having the narrow conceptualization of the universe of human tasks necessary to even be able to say this with a straight face, irrespective of ones qualitative assessment of how well LLMs do the things that they are capable of doing.
- mathiaspoint 1y agoWhy don't you go out with a laptop and offer people cash prizes. See how well the average person does.
- SirMaster 1y agoI think most people think AGI is achieved when a machine can do at least everything that humans can do. Like not necessarily physical things, but mental or digital things. Humans will create a better LLM, (say GPT-5) than all the other LLMs that currently exist. If you tasked any current LLM with creating a GPT-5 LLM that is better than itself, can it do it? If not then it's probably not AGI and has some shortcomings making it not general or intelligent enough.