31 ms·

Andrej Karpathy – It will take a decade to work through the issues with agents

• JCharante 1y ago
  Why does everyone have such short timelines to show progress? So what if it takes 50 years to develop, we’ll have AGI for the next million years
  • ben_w 1y ago
    If it takes 50 years, you might have it for millions of years along with anti-aging solutions that actually work, I'll have probably died of old age. I don't know how much wish fulfilment there is in people's timelines.
• notepad0x90 1y ago
  I'm betting we'll have either cold fusion or the "year of the linux desktop" (finally) before AGI.
• agrover 1y ago
  This seems to be the growing consensus.
  • sputknick 1y ago
    There is a very strange totally coincidental correlation where if you are smart and NOT trying to raise money for an AI start-up, you think AGI is far away, and if you are smart and actively raising money for an AI start-up, then AGI is right around the corner. One of those odd coincidences of modern life
    • chrismorgan 1y ago
      Mind you, such a correlation can be reasonable—the Yesses work for something because they believe it, while the Noes don’t because they don’t. (In this instance, I’m firmly a No, and I don’t say such a correlation is reasonable, due to the corrupting influence of money plus hype sweeping people along, which I think are much more common. But there will still be at least some that are True Believers, and it does make sense that they would then try to raise money to achieve their vision.)
• evandrofisico 1y ago
  And self sustained nuclear fusion is 20 years away, perpetually. On which evidence can he affirm a timeline for AGI when we can barely define intelligence?
  • adastra22 1y ago
    Fusion used to be perpetually 30 years away. We’re making progress!
  • helterskelter 1y ago
    I'd argue we've had more progress towards fusion than AGI.
    • deleted 1y ago
      [deleted]
    • FiniteIntegral 1y ago
      Yet at the same time "towards" does not equate to "nearing". Relative terms for relative statements. Until there's a light at the end of the tunnel, we don't know how far we've got.
    • chasd00 1y ago
      > I'd argue we've had more progress towards fusion than AGI. way more pogress toward fusion than AGI. Uncontrolled runaway fusion reactions were perfected in the 50s (iirc) with the thermonuclear bombs. Controllable fusion reactions have been common for many years. A controllable, self-sustaining, and profitable fusion reaction is all that is left. The goalposts that mark when AGI has been reached haven't even been defined yet.
  • CaptainOfCoit 1y ago
    And a program that can write, sound and paint like a human was 20 years away perpetually as well, until it wasn't.
    • galangalalgol 1y ago
• BoredPositron 1y ago
  AGI is going to be the new fusion.
• Topgamer7 1y ago
  Whenever someone brings up "AI", I tell them AI is not real AI. Machine learning is a more apt buzzword. And real AI is probably like fusion. Its always 10 years away.
  • CharlesW 1y ago
    > Whenever someone brings up "AI", I tell them AI is not real AI. You and also everyone since the beginning of AI. https://quoteinvestigator.com/2024/06/20/not-ai/ https://quoteinvestigator.com/2024/06/20/not-ai/
    • zamadatix 1y ago
      People saying that usually mean it as "AI is here and going to change everything overnight now" yet, if you take it literally, it's "we're actually over 50 years into AI, things will likely continue to advance slowly over decades". The common thread between those who take things as "AI is anything that doesn't work yet" and "what we have is still not yet AI" is "this current technology could probably have used a less distracting marketing name choice, where we talk about what it delivers rather than what it's supposed to be delivering".
  • lo_zamoyski 1y ago
    AI is in the eye of the beholder.
  • huflungdung 1y ago
    [dead]
  • adastra22 1y ago
    Machine learning as a descriptive phrase has stopped being relevant. It implies the discovery of information in a training set. The pre-training of an LLM is most definitely machine learning. But what people are excited and interested in is the use of this learned data in generative AI. “Machine learning” doesn’t capture that aspect.
    • 1y ago
• kachapopopow 1y ago
  AGI is already here if you shift some goal posts :) From skimming the conversation it seems to mostly revolve around LLMs (transformer models) which is probably not going to be the way we obtain AGI to begin with, frankly it is too simple to be AGI, but the reason why there's so much hype is because it is simple to begin with so really I don't know.
  • ecocentrik 1y ago
    LLMs are close enough to pass the Turing Test. That was a huge milestone. They are capable of abstract reasoning and can perform many tasks very well but they aren't AGI. They can't teach themselves to play chess at the level of a dedicated chess engine or fly an airplane using the same model they use to copypasta a React UI. They can only fool non-proficient humans into believing that they might be capable of doing those things.
    • password54321 1y ago
      Turing Test was a thought experiment not a real benchmark for intelligence. If you read the paper the idea originated from it is largely philosophical. As for abstract reasoning, if you look at ARC-2 it is barely capable though at least some progress has been made with the ARC-1 benchmark.
      • ecocentrik 1y ago
        I wasn't claiming the Turing Test was a benchmark for intelligence but the ability to fool a human into thinking a machine is intelligent in conversation is still a significant milestone. I should have said "some abstract reasoning". ARC-2 looks promising.
        • password54321 1y ago
          >I wasn't claiming the Turing Test was a benchmark for intelligence but the ability to fool a human into thinking a machine is intelligent in conversation is still a significant milestone. The Turing Test is whether it can fool a human into thinking it is talking to another human not an intelligent machine. And ironically this is becoming less true over time as people become more used to spotting the tendencies LLMs have with writing such as its frequent use of dashes or "it's not just X it is Y" type of statements.
• ares623 1y ago
  Is that at current investment levels?
• konart 1y ago
  2035 singularity etc
  • spydum 1y ago
    2038 will be more significant
    • edbaskerville 1y ago
      For a second I thought you were citing some special mythological timeline from AI folks. Then I got it. :) Something so mundane that maybe the AIs can help prevent it.
• asdev 1y ago
  Are researchers scared to just come out and say it because they'll be labeled as wrong if the extreme tail case happens?
  • andy_ppp 1y ago
    No, it’s because of money and the hype cycle.
    • ionwake 1y ago
      I mean you say this, but I havent touched a line of code as a programmer in months, having been totally replaced by AI. I mean sure I now "control" the AI, but I still think these no AGI for 2 decades claims are a bit rough.
      • andy_ppp 1y ago
        I think AI is great and extremely helpful but if you’ve been replaced already maybe you have more time now to make better code and decisions? If you think the AI output is good by default I think maybe that’s a problem. I think general intelligence is something other than what we have now, these systems are extremely bad at updating their knowledge and hopelessly at applying understanding from one area to another. For example self driving cars are still extremely brittle to the point of every city needing new and specific training - you can just take a car with controls on the opposite side to you and safely drive in another country.
        • ionwake 1y ago
          Yeah i agree. However like ... I don't understand why you think I am making bad decisions? I'm a self made ( to an extent ) millionaire, I am doing ok
      • an0malous 1y ago
        Let’s see the code
• segmondy 1y ago
  AGI is already here.
  • chronci739 1y ago
    > AGI is already here. cause elon musk says FSD is coming in 2017?
    • adastra22 1y ago
      Because we already have artificial (man-made) general (contrast with domain specific) intelligence (algorithmic problem solvers). A.G.I. If ChatGPT is not AGI, somebody has moved goalposts.
      • deleted 1y ago
        [deleted]
      • walkabout 1y ago
        I think a baseline requirement would be that it… thinks. That’s not a thing LLMs do.
        • adastra22 1y ago
          That’s an odd claim, given that we have so-called thinking models. Is there a specific way you have in mind in which LLMs are not thinking processes?
          • blibble 1y ago
            I can call my cat an elephant it doesn't make him one
      • 010101010101 1y ago
        Both "general" and "intelligence" are _at least_ easily arguable without moving any goal posts, not that goal posts have ever been well established in the first place.
• imiric 1y ago
  It has always been "a decade away". But nothing will make grifters richer than promising it's right around the corner.
• Mistletoe 1y ago
  Good because we have no framework whatsoever enabled for if it is legal or ethical to turn it off. Is that murder? I think so.
  • lyu07282 1y ago
    We don't even have any intention to do anything about millions of people loosing their jobs and driven into poverty by it, in fact the investments right now gamble/depend on that wealth transfer to happen in the future. We don't even give a shit about other humans, there is absolutely no way we will care about a (hypothetical) different life form entirely.
• qgin 1y ago
  We'll be living in a world of 50% unemployment and still debating whether it's "true AGI"
• ciconia 1y ago
  It's funny how there's such a pervasive cynicism about AI in the developer community, yet everyone is still excited about vibe coding. Strange times...
  • leptons 1y ago
    What developer is excited about "vibe coding"? The only people excited about "vibe coding" are people who can't code.
    • dist-epoch 1y ago
      ...and all developers use Macs, live in SF, and deploy to AWS.
    • qingcharles 1y ago
      That's a crazy generalization. I've coded professionally for 40 years. I'm hugely excited about vibe coding. I use it every single day to create little tools and web apps to help me do my job.
      • deleted 1y ago
        [deleted]
    • gabriel-uribe 1y ago
      Deeply excited about vibe coding -- my non-technical cofounder 'codes' now.
    • spjt 1y ago
      I love vibe coding because it does the things I hate really well, like meeting test coverage requirements and writing doc comments.
    • cmrdporcupine 1y ago
      This of course depend completely on how you define "vibe" coding. Assisted coding has been incredibly useful. I have been using Claude Code daily. But if you let it take over completely without review and let it write whole features... which I take to be the meaning of "vibe" in some people's definitions... you're in for a world of long-term pain.
• deadbabe 1y ago
  Frankly it doesn’t matter if it’s a decade away. AI has now been revealed to the masses. When AGI arrives most people will barely notice. It will just feel like slightly better LLMs to them. They will have already cemented notions of how it works and how it affects their lives.
• angiolillo 1y ago
  "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." - Edsger Dijkstra The debate about AGI is interesting from a philosophical perspective, but from a practical perspective AI doesn't need to get anywhere close to AGI to turn the world upside down.
  • flatline 1y ago
    I don’t even know what AGI is, and neither does anyone else as far as I can tell. In the parts of the video I watched, he cites several things missing which all have to do with autonomy: continual automated updates of internal state, fully autonomous agentic behavior, etc. I feel like GPT 3 was AGI, personally. It crossed some threshold that was both real and magical, and future improvements are relying on that basic set of features at their core. Can we confidently say this is not a form of general intelligence? Just because it’s more a Chinese Room than a fully autonomous robot? We can keep moving the goalposts indefinitely, but machine intelligence will never exactly match that of humans.
    • mpalmer 1y ago
      It crossed some threshold that was both real and magical Only compared to our experience at the time. and future improvements are relying on that basic set of features at their core Language models are inherently limited, and it's possible - likely, IMO - that the next set of qualitative leaps in machine intelligence will come from a different set of ideas entirely.
      • zer00eyz 1y ago
        Learning != Training. Thats not a period, it's a full stop. There is no debate to be had here. IF an LLM makes some sort of breakthrough (and massive data collation allows for that to happen) it needs to be "re trained" to absorb its own new invention. But we also have a large problem in our industry, where hardware evolved to make software more efficient. Not only is that not happening any more but we're making our software more complex and to some degree less efficient with every generation. This is particularly problematic in the LLM space: every generation of "ML" on the llm side seems to be getting less efficient with compute. (Note: this isnt quite the case in all areas of ML, yolo models working on embedded compute is kind of amazing). Compactness, efficiency and reproducibility are directions the industry needs to evolve in, if it ever hopes to be sustainable.
• hatmanstack 1y ago
  Am I dating myself by thinking Kurzweil is still relevant? 2029: Human-level AI 2045: The Singularity - machine intelligence 1 billion times more powerful than all human intelligence Based on exponential growth in computing. He predicts we'll merge with AI to transcend biological limits. His track record is mixed, but 2029 looks more credible post-GPT-5. The 2045 claim remains highly speculative.
  • Barrin92 1y ago
    It's curious that Kurzweil's predictions about transcending biology align so closely with his expected lifespan. Reminds me of someone saying, if you ask a researcher for a timeline of a breakthrough they'll give you the expected span of their career. Hegel thought history ended with the Prussian state, Fukuyama thought it ended in liberal America, Paul thought judgement day was so close you need not bother to marry, the singularity always comes around when the singularians get old. Funny how that works
  • williamcotton 1y ago
    The biggest problem I've had with Kurzweil and the exponential growth curve is that the elbow depends entirely on how you plot and scale the axis. With a certain vantage point we have arguably been on an exponential curve since the advent of Homo Sapiens.
  • deleted 1y ago
    [deleted]
  • somenameforme 1y ago
    I lost all respect for him after reading about his views on medical immortality. His argument is that over time human life expectancy has been constantly increasing * and he calculated that based on some arbitrary rate of acceleration, that science would be expanding human life expectancy by more than a year, per year - medical immortality in other words, and all expected to happen just prior to the time he's reaching his final years. The overwhelming majority of all gains in human life expectancy have come due to reductions in infant mortality. When you hear about things like a '40' year life expectancy in the past it doesn't mean that people just dropped dead at 40. Rather if you have a child that doesn't make it out of childhood, and somebody else that makes it to 80 - you have a life expectancy of ~40. If you look back to the upper classes of old their life expectancy was extremely similar to those of today. So for instance in modern history, of the 15 key Founding Fathers, 7 lived to at least 80 years old: John Adams, John Quincy Adams, Samuel Adams, Jefferson, Madison, Franklin, John Jay. John Adams himself lived to 90. The youngest to die were Hamilton who died in a duel, and John Hancock who died of gout of an undocumented cause - it can be caused by excessive alcohol consumption. All the others lived into their 60s and 70s. So their overall life expectancy was pretty much the same as we have today. And this was long before vaccines or even us knowing that surgeons washing their hands before surgery was a good thing to do. It's the same as you go back further into history. A study [1] of all men of renown in Ancient Greece was 71.3 [1], and that was from thousands of years ago! Life expectancy at birth is increasing, but longevity is barely moving. And as Kurzweil has almost certainly done plentiful research on this topic, he is fully aware of this. Cognitive dissonance strikes again. [1] - https://pubmed.ncbi.nlm.nih.gov/18359748/ https://pubmed.ncbi.nlm.nih.gov/18359748/
• seydor 1y ago
  I wouldn't consider either of them qualified to answer that question
• awesome_dude 1y ago
  I have massive respect for Andrej, my first encounter with "him" was following his tutorials/notes when he was a grad student/tutor for AI/ML. I was a lot disappointed when he went to work for Tesla, and I think that he had some achievement there, butnot nearly the impact I believe he potentially has. His switch (back?) to OpenAI was, in my mind, much more in keeping with where his spirit really lies. So, with that in mind, maybe I've drunk too much kool aid, maybe not. But I'm in agreement with him, the LLMs are not AGI, they're bloody good natural language processors, but they're still regurgitating rather than creating. Essentially that's what humans do, we're all repeating what our education/upbringing told us worked for our lives. But we all recognise that what we call "smart" is people recognising/inventing ways to do things that did not exist before. In some cases its about applying a known methodset to a new problem, in others its about using a substance/method in a way that other substances/methodsets are used, but the different substance/methodset produces something interesting (think, oh instead of boiling food in water, we can boil food in animal fats... frying) AI/LLMs cannot do this, not at all. That spark of creativity is agonisingly close, but, like all 80/20 problems, is likely still a while away. The timeline (10 years) - it was the early 2010s (over 10 years ago now) that the idea of backward propagation, after a long AI winter, finally came of age. It (the idea) had been floating about since at least the 1970s. And that ushered in the start of our current revolution, that and "Deep Learning" (albeit with at least another AI winter spanning the last 4 or 5 years until LLMs arrived) So, given that timeline, and the restraints in the currrent technology, I think that Andrej is on the right track, and it will be interesting to see where we are in ten years time.
  • chasd00 1y ago
    if openAI didn't put a chat interface in front of an LLM and make it available to the public wouldn't we still be in the same AI winter? Google, Meta, Microsoft, all of the major players were doing lots of LLM work already, it wasn't until the general public found out through the OpenAI's website that it really took off. I can't remember who said it, it was some CEO, that OpenAI had no moat but nether did anyone else. They all had LLMs already of their own. Was the breakthrough the LLM or making it accessible to the general public?
    • 1y ago
• PedroBatista 1y ago
  Following the comments here, yes: AGI is the new Cold Fusion. However, don't let the bandwagon ( from either side ) cloud your judgment. Even warm fusion or any fusion at all is still very useful and it's here to stay. This whole AGI and "the future" thing is mostly a VC/Banks and shovel sellers problem. A problem that has become ours too because the ridiculous amounts of money "invested", so even warm fusion is not enough from an investment vs expectations perspective. They are already playing musical money chairs, unfortunately we already know who's going to pay for all of this "exuberance" in the end. I hope this whole thing crashes and burns as soon as possible, not because I don't "believe" in AI, but because people have been absolutely stupid about it. The workplace has been unbearable with all this stupidity and amounts of fake "courage" about every single problem and the usual judgment of the value of work and knowledge your run-of-the-mill dipshit manager has now.
• jb1991 1y ago
  I would bet all of my assets of my life that AGI will not be seen in the lifetime of anyone reading this message right now. That includes anyone reading this message long after the lives of those reading it on its post date have ended. Which of course raises the interesting question of how I can make good on this bet.
  • colecut 1y ago
    If you are right, you don't have to
  • rokkamokka 1y ago
    Will you take a wager of my one dollar versus your life assets? :)
  • plaidfuji 1y ago
    Should probably just short nvidia
    • lbhdc 1y ago
      “Markets can remain irrational longer than you can remain solvent.”
    • Thrymr 1y ago
      "just short nvidia" is not simple. Even if you believe it is overvalued, and you are correct, a short is a specific bet that the market will realize that fact in a precise amount of time. There are very significant risks in short selling, and famously, the market can stay irrational longer than you can remain solvent.
    • simonsarris 1y ago
      There is a wide space where LLMs and their offshoots make enormous productivity gains, while looking nothing like actual artificial intelligence (which has been rebranded AGI), and Nvidia turns out to have a justified valuation etc.
      • 1y ago
• 1970-01-01 1y ago
  Great quote: "When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three nines or two nines. I don’t know what it is, but multiple nines of iteration. There are still more nines to go. That’s why these things take so long."
  • onlyrealcuzzo 1y ago
    Importantly, the first 9s are the easiest. If you need to get to 9 9s, the 9th 9 could be more effort than the other 8 combined.
• 6d6b73 1y ago
  Even without AGI, current LLMs will change society in ways we can't yet imagine. And this is both good and bad. Current LLMs are just a different type of automation, not mechanical like control systems and robots, but intellectual. They don't have to be able to think independently, but as long as they automate some white-collar tasks, they will change how the rest of society works. The simple transistor is just a small electronic component that is a better version of a tube, and yet it changed everything in a few decades. How will the world change because of LLMs? I have no idea, but I know it doesn't have to be AGI to cause a lot of upheaval.
  • flyinglizard 1y ago
    The same thing you described that makes LLM great also make them entirely non-deterministic and unreliable for serious automated applications.
    • deleted 1y ago
      [deleted]
  • password54321 1y ago
    They can't even automate chat support the very thing you would think LLMs would be good at. Yet I always end up needing to talk to a person.
• Imnimo 1y ago
  >What takes the long amount of time and the way to think about it is that it’s a march of nines. Every single nine is a constant amount of work. Every single nine is the same amount of work. When you get a demo and something works 90% of the time, that’s just the first nine. Then you need the second nine, a third nine, a fourth nine, a fifth nine. While I was at Tesla for five years or so, we went through maybe three nines or two nines. I don’t know what it is, but multiple nines of iteration. There are still more nines to go. I think this is an important way of understanding AI progress. Capability improvements often look exponential on a particular fixed benchmark, but the difficulty of the next step up is also often exponential, and so you get net linear improvement with a wider perspective.
  • sdenton4 1y ago
    Ha, I often speak of doing the first 90% of the work, and then moving on to the following 90% of the work...
    • inerte 1y ago
      I use "The project is 90% ready, now we only have to do the other half"
      • typpilol 1y ago
        92% is half actually - RuneScape Players
    • JimDabell 1y ago
      > The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time. — Tom Cargill, Bell Labs (September 1985) https://dl.acm.org/doi/pdf/10.1145/4284.315122 https://dl.acm.org/doi/pdf/10.1145/4284.315122
  • czk 1y ago
    like leveling to 99 in old school runescape
• theusus 1y ago
  [flagged]
  • jasonthorsness 1y ago
    Andrej coined the term "vibe coding" in February on X, only 8 months ago.
  • guiomie 1y ago
    He's also the guy behind FSD which is kinda turning into a scam.
    • lazystar 1y ago
      > FSD which is a scam. fixed that for you.
    • deleted 1y ago
      [deleted]
  • dlivingston 1y ago
    ??? Many developers, experienced and not, play around with vibe coding. Is your critique of him that he has tried vibe coding?
    • theusus 1y ago
      I’m critiquing him that he lied in his claim. And anyone who claims same is just farming engagement.
• johnhamlin 1y ago
  Did anyone here actually watch the video before commenting? I’m seeing all the same old opinions and no specific criticisms of anything Karpathy said here.
  • dang 1y ago
    More specific responses have come in as people have digested more of the content. This is the reflexive/reflective distinction (https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sort=byDate&type=comment&query=reflective%20reflex%20by:dang https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...). Reflexive comments—the kind that express some pre-existing feeling or opinion that happens to get triggered by association—are much faster to produce, so unfortunately they show up first in many threads.
• awongh 1y ago
  Now that Nvidia is the most valuable company, all this talk of actual AGI will be washed away by the huge amount of dollars driving the hype train. Most of these companies value is built on the idea of AGI being achievable in the near future. AGI being too close or too far away affects the value of these companies- too close and it'll seem too likely that the current leaders will win. Too far away and the level of spending will seem unsustainable.
  • michaelt 1y ago
    > Most of these companies value is built on the idea of AGI being achievable in the near future. Is it? Or is it based on the idea a load of white collar workers will have their jobs automated, and companies will happily spend mid four figures for tech that replaces a worker earning mid five figures?
    • jjulius 1y ago
      Why not both? :)
    • rootusrootus 1y ago
      I think companies that expect to use AI to cut their salary overhead making the same products they were before are going to get clobbered by companies that use AI to grow. A few people may have to retrain into a different line of work but I don't really see AI putting people out of work en masse.
    • JumpCrisscross 1y ago
      From what I've seen, the most-compelling thesis involves robotics. We're seeing evidence that LLMs tokenising physical inputs can operate robots better than previous methods. If that's pans out, the investment thesis is secured. No AGI needed.
  • tootie 1y ago
    Exactly. A 5-10 year timeline and you've got the formula for a new Space Race with China. Give us $7T or else China will control the world. This 2024 story feels like ancient history that everyone has forgotten: https://www.cnbc.com/2024/02/09/openai-ceo-sam-altman-reportedly-seeking-trillions-of-dollars-for-ai-chip-project.html https://www.cnbc.com/2024/02/09/openai-ceo-sam-altman-report...
• sosodev 1y ago
  I think it's a shame that a 146 minute podcast released ~55 minutes ago has so much discussion. Everybody here is clearly just reacting to the title with their own biases. I know it's against the guidelines to discuss the state of a thread, but I really wish we could have thoughtful conversations about the content of links instead of title reactions.
  • jasonthorsness 1y ago
    This one does have the full transcript underneath (wonderful feature). But it's a long read too so I think your assumption is correct :P.
  • mpalmer 1y ago
    Be fair; plenty of people transcribe and read podcasts, and/or summarize/excerpt them.
    • j45 1y ago
      Summaries are great, but can be surface. The brain processes and has insights differently experiencing it at conversation speed. We might get what the conversation was that others had, but it can miss the mark for the listening and inner processing that leads to it's own gifts. It's not about one or the other for me, usually both.
    • yoz-y 1y ago
      The idea that people would do this has never even crossed my mind. Not disputing that people do this, mind you. Technology is certainly there, but I also think that it’s very prone to taking ideas out of context.
  • tauchunfall 1y ago
    there is a transcript, people can skim for interesting parts and read for 30 minutes and then comment. edit: typo fix.
  • markbao 1y ago
    Just as the core idea of a book can be (lossily) summarized in a few sentences, the core crux of an argument can be quite simple and not require wading though the whole discussion (the AGI discussion is only 30 minutes anyhow). Granted, a bunch of commenters are probably doing what you’re saying.
• goalieca 1y ago
  I remember attending a lecture from a famous quantum computing researcher in 2003. He said that quantum computing is 15-20 years away and then he followed up by saying that if he told anyone it was further away then he wouldn't get funding!
  • Yoric 1y ago
    And now (useful) quantum computing is 5 years away! Has been for a few years, too.
    • oldgradstudent 1y ago
      Any day now.
  • EA-3167 1y ago
    It's an excellent time-frame that sounds imminent enough to draw interest (and funding), but is distant enough that you can delay the promised arrival a few times in the span of a career before retiring. Fusion research lives and dies on this premise, ignoring the hard problems that require fundamental breakthroughs in areas such as materials science, in favor of touting arbitrary benchmarks that don't indicate real progress towards fusion as a source of power on the grid. "Full self driving" is another example; your car won't be doing this, but companies will brag about limited roll-outs of niche cases in dry, flat, places that are easy to navigate.
    • plastic3169 1y ago
      > ”Full self driving" is another example; your car won't be doing this, but companies will brag about limited roll-outs of niche cases in dry, flat, places that are easy to navigate. Not expecting my car to be self-driving anytime soon, but I have understood there is actual working robotaxi service in San Francisco which is not easy or flat? I think we can’t keep saying self driving cars will never happen when this kind of thing already exists.
      • 1y ago
• netrap 1y ago
  Wonder if it will end up like nuclear fusion.. just another decade away! :)
• reenorap 1y ago
  Are "agents" just programs that call into an LLM and based on the response, it will do something?
  • cootsnuck 1y ago
    Kinda. It's just an LLM that performs function calling (i.e. the LLM "decides" when a function needs to be called for a task and passes the appropriate function name and arguments for that function based on its context). So yea an "agent" is that LLM doing all of that and then your program that actually executes the function accordingly. That's an "agent" at its simplest -- a LLM able to derive from natural language when it is contextually appropriate to call out to external "tools" (i.e. functions).
  • fragmede 1y ago
    "Something" is broad and not well defined, but basically yeah. Rather than try to define it in terms of complexity of the something, I'll put it in terms of minutes. If the LLM returns a response, and that response gets fed into a system and run, and that's it, I wouldn't really call that agentic. It's got to go a few more rounds back and forth to be agentic, imo. In terms of time, I'd say the agent program has to be capable of at least 10 minutes of going from user input, then the program calling into the LLM, feeding the LLM response into a system, feeding that result back into the LLM, and feeding that into the system in a loop. Obviously there are ways to game that metric, like the terrible lines of code metric, but I think it's a decent handwave for when it feels like there's an agent working for me rather than a non-agentic system. What it's doing for those 10 minutes is important, calling "sleep 600" obviously doesn't count. Eg for a programming LLM with an agentic agent and access to a computer, would be able to, given design-doc.md and Todo.md, implement feature X, making sure it compiles, run some basic smoke tests, write appropriate unit tests, make sure they all pass, and finally push the code and create a draft PR. Naturally, not every call into the agent is going to take the full 10 minutes. It may need to ask questions before getting started, or stop if there's an unrecoverable error. Sometimes you'll just need to tell it "continue", but the system should be capable of a 10-minute run (hopefully longer!) given enough support.
  • sammyd56 1y ago
• rwaksmunski 1y ago
  AGI is still a decade away, and always will be.
  • gjm11 1y ago
    You say that as if people had been saying "10 years away" for ages, but I don't think that's true at all. There's some information about historical predictions at https://www.openphilanthropy.org/research/what-should-we-learn-from-past-ai-forecasts/ https://www.openphilanthropy.org/research/what-should-we-lea... (written in 2016) from which (I am including the spreadsheet found at footnote 27) these are some I-hope-representative data points, with predictions from actual AI researchers, popularizers, pundits, and SF authors: 1960: Herbert Simon predicts machines can do all (intellectual) work humans can "within 20 years". 1961: Marvin Minsky says "within our lifetimes, machines may surpass us"; he was 33 at the time, suggesting a not-very-confident timescale of say 40 years. 1962: I J Good predicts something at or above human level circa 1978. 1963: John McCarthy allegedly hopes for "a fully-intelligent machine" within a decade. 1970: I J Good predicts 1994 +- 10 years. 1972: a survey of 67 computer scientists found 27% saying <= 20 years, 32% saying 20-50 years, and 42% saying > 50 years. 1977-8: McCarthy says things like "4 to 400 years" and "5 to 500 years". 1988: Hans Moravec predicts human-level intelligence in 40 years. 1993: Vernor Vinge predicts better-than-human intelligence in the range 2005..2030. 1999: Eliezer Yudkowsky predicts intelligence explosion circa 2020. 2001: Ben Goertzel predicts "during the next 100 years or so". 2001: Arthur C Clarke predicts human-level intelligence circa 2020. 2006: Douglas Hofstadter predicts somewhere around 2100. 2006: Ray Solomonoff predicts within 20 years. 2008: Nick Bostrom says <50% chance by 2033. 2008: Rodney Brooks says no human-level AI by 2030. 2009: Shane Legg says probably between 2018 and 2036. 2011: Rich Sutton estimates somewhere around 2030. Of these, exactly one suggests a timescale of 10 years; the same person a little while later expresses huge uncertainty ("4 to 400 years"). The others are predicting timescales of multiple decades, also generally with low confidence. Some of those predictions are now known to have been too early. There definitely seems to be a sort of tendency to say things like "about 30 years" for exciting technologies many of whose key details remain un-worked-out: AI, fusion power, quantum computing, etc. But it's definitely not the case that "a decade away" has been a mainstream prediction for a long time. People are in fact adjusting their expectations on the basis of the progress they observe in recent years. For most of the time since the idea of AI started being taken seriously, "10 years from now" was an exceptionally optimistic[1] prediction; hardly anyone thought it would be that soon. Now, at least if you listen to AI researchers rather than people pontificating on social media, "10 years from now" is a typical prediction; in fact my impression is that most people who spend time thinking about these things[2] expect genuinely-human-level AI systems sooner than that, though they typically have rather wide confidence intervals. [1] "Optimistic" in the narrow sense in which expecting more progress is by definition "optimistic". There are many many ways in which human-level, or better-than-human-level, AI could in fact be a very bad thing, and some of them are worse if it happens sooner, so "optimistic" predictions aren't necessarily optimistic in the usual sense. [2] Most, not all, of course.
• mkbelieve 1y ago
  I don't understand how anyone can believe that we're near even a whiff of AGI when we barely understand what dreaming is, or how the human brain interacts with the quantum world. There are so many elements of human creativity that are still utterly hidden behind a wall that it makes me feel insane when an entire industry is convinced we're just magically going to have the answer soon. The people heralding the emergence of AGI are doing little more than pushing Ponzi schemes along while simultaneously fueling vitriolic waves of hate and neo-luddism for a ground-breaking technology boom that could enhance everything about how we live our lives... if it doesn't get regulated into the ground due to the fear they're recklessly cooking up.
  • kovek 1y ago
    There's many different definitions of "AGI" that people come up with, and some include dreaming, quantum world, creativity, and some do not.
  • throwaway-0001 1y ago
    We Dont know how a horse works, but we got cars. Analogy doesn’t work.
    • giardini 1y ago
      What's your point here? Be careful: language w/o analogy might be impossible. But both are asides to the ongoing discussion.
  • hackinthebochs 1y ago
    Big scientific revolutions tend to happen before we understand the relevant mechanisms. It is only after the fact that we develop a theory to understand how it works. AGI will very likely follow the same trend. Enough people are throwing enough things at the wall that eventually something will stick.
• observationist 1y ago
  Kurzweil has been eerily right so far, and his timeline has AGI at 2029. When software can perform any unattended, self directed task (in principle) at least as well as any human over the sum total of all tasks that humans are capable of doing, we will have reached AGI. Software can already write more text on any given subject better than a majority of humanity. It can arguably drive better across more contexts than all of humanity - any human driver over a billion miles of normal traffic will have more accidents than self driving AI over the same distance. Short stories, haikus, simple images, utility scripts, simple software, web design, music generation - all of these tasks are already superhuman. Longer time horizons, realtime and continuous memory, a suite of metacognitive tasks, planning, synthesis of large bodies of disparate facts into novel theory, and a few other categories of tasks are currently out of reach, but some are nearly solved, and the list of things that humans can do better than AI gets shorter by the day. We're a few breakthroughs away, maybe even one big architectural leap, from having software that is capable (in principle) of doing anything humans can do. I think AGI is going to be here faster than Kurzweil predicted, because he probably didn't take into consideration the enormous amount of money being spent on these efforts. There has never been anything like this in history - in the last decade, over 5 trillion dollars has been spent on AI research and on technologies that support AI, like crypto mining datacenters that pivoted to AI, new power, water, data support, providing the infrastructure and foundation for the concerted efforts in research and development. There are tens of thousands of AI researchers, some of them working in private finance, some for academia, some doing military resarch, some doing open source, and a ton doing private sector research, of which an astonishing amount is getting published and shared. In contrast, the entire world spent around 16 trillion dollars on world war II - all of the R&D and emergency projects and military logistics, humanitarian aid, and so on. We have AI getting more resources and attention and humans involved in a singular development effort, pushing toward a radical transformation of the very concept of "labor" - while I think it might be a good thing if it is a decade away, even perpetually so until we have some reasonable plan for coping with it, I very much think we're going to see AGI within the very near future. *When I say "in principle" I mean that given the appropriate form factor, access, or controls, the AI can do all the thinking, planning, and execution that a human could do, at least as well as any human. We will have places that we don't want robots or AI going, tasks reserved for humans, traditions, taboos, economics, and norms that dictate AI capabilities in practice, but there will be no legitimacy to the idea that an AI couldn't do a thing.
• overgard 1y ago
  Right in time for the year of the linux desktop.
• nopinsight 1y ago
  A definition of AGI: https://www.agidefinition.ai/ https://www.agidefinition.ai/ A new contribution by quite a few prominent authors. One of the better efforts at defining AGI *objectively*, rather than through indirect measures like economic impact. I believe it is incomplete because the psychological theory it is based on is incomplete. It is definitely worth discussing though. —- In particular, creative problem solving in the strong sense, ie the ability to make cognitive leaps, and deep understanding of complex real-world physics such as the interactions between animate and inanimate entities are missing from this definition, among others.
  • kart23 1y ago
    I don't know a single one of the "Social Science" items, and I'm pretty sure 90% of college educated people wouldn't know a single one either.
    • CamperBob2 1y ago
      Not only that, but the notion that GPT-5 will answer those questions with only 2% accuracy seems suspect. Those are exactly the kinds of questions that current models are great at. Nothing about that page makes much sense.
      • jonas21 1y ago
        The percentages are added, not averaged. Each category sums to 10%, and the General Knowledge category has 5 equally-weighted subcategories, so 2% is the best possible score you can get in the social science subcategory. I don't know why they decided to do it this way. It's very confusing.
  • chrisweekly 1y ago
    I agree it seems like a better-structured effort than many others. But its shortcomings go beyond a shallow and incomplete foundation in psychology. It also has basic errors in its execution, eg a "Geography" question about centripetal and centrifugal forces. Color me extremely skeptical.
• cayleyh 1y ago
  "decade" being the universal time frame for "I don't know" :D
• ActorNightly 1y ago
  Not a decade. More like a century, and that is if society figures itself out enough to do some engineering on a planetary scale, and quantum computing is viable. Fundamentally, AGI requires 2 things. First it needs to be able to operate without information, learning as it goes. The core kernel should be such that it doesn't have any sort of training on real world concepts, only general language parsing that it can use to map to some logic structure to be able to determine a plan of action. So for example, if you give the kernel the ability to send ethernet packets, it should eventually figure out how to talk tls to communicate with the modern web, even if that takes an insane amount of repetition. The reason for this is that you want the kernel to be able to find its way through any arbitrarily complex problem space. Then as it has access to more data, whether real time, or in memory, it can be more and more efficient. This part is solvable. After all, human brains do this. A single rack of Google TPUs is roughly the same petaflops as a human brain operating at max capacity if you assume neuron activation is a add-multiply and firing speed of 200 times/second, and humans don't use all of their brain all the time. The second part that makes the intelligence general is the ability to simulate reality faster than reality. Life is imperative by nature, and there are processes with chaotic effects (human brains being one of them), that have no good mathematical approximations. As such, if an AGI can truly simulate a human brain to be able to predict behavior, it needs to do this at an approximation level that is good enough, but also fast enough to where it can predict your behavior before you exhibit it, with overhead in also running simulations in parallel and figuring out the best course of actions. So for a single brain, you are looking at probably a full 6 warehouses full of TPUs.
  • ctoth 1y ago
    You want a "core kernel" with "general language parsing" but no training on real-world concepts. Read that sentence again. Slowly. What do you think "general language parsing" IS if not learned patterns from real-world data? You're literally describing a transformer and then saying we need to invent it. And your TLS example is deranged. You want an agent to discover the TLS protocol by randomly sending ethernet packets? The combinatorial search space is so large this wouldn't happen before the sun explodes. This isn't intelligence! This is bruteforce with extra steps! Transformers already ARE general algorithms with zero hardcoded linguistic knowledge. The architecture doesn't know what a noun is. It doesn't know what English is. It learns everything from data through gradient descent. That's the entire damn point. You're saying we need to solve a problem that was already solved in 2017 while claiming it needs a century of quantum computing.
• deleted 1y ago
  [deleted]
• TheBlight 1y ago
  I knew this once I heard OpenAI was going to get into the porn bot business. If you have AGI you don't need porn bots.
  • tasuki 1y ago
    Why not?
    • gtirloni 1y ago
      Most likely because you'll be filthy reach from selling AGI and won't need to go after secondary revenue sources.
      • charcircuit 1y ago
        >Most likely because you'll be filthy reach from selling AGI Why? If AGI costs more than a human or operates slower than one, it may not be economical for people to buy it. By the time it becomes economical, competitors may have also cracked it reducing your ability to charge high margins on it.
        • shdh 1y ago
          Cost decreases with time Humans can work on a problem 8 hours a day? You can run inference 24/7
          • charcircuit 1y ago
            It decreases, but decreasing from $1 million per token to $0.9 million per token after a year is still a decrease, but it still is not viable. Paying an AGI a $100 billion dollars for it to work 24/7 for a year is worse than hiring 10 people for $30k a year to work shifts to do the same work 24/7.
        • pier25 1y ago
          If OpenAI had anything even resembling AGI they'd be milking the shit of that, even if only for marketing.
• jeffreygoesto 1y ago
  Ah. The old "still close enough, so we can pretend you should pour your money over us, as we haven't identified the next hype yet" razzle dazzle...
• arthurofbabylon 1y ago
  Agency. If one studied the humanities they’d know how incredible a proposal “agentic” AI is. In the natural world, agency is a consequence of death: by dying, the feedback loop closes in a powerful way. The notion of casual agency (I’m thinking of Jensen Huang’s generative > agentic > robotic insistence) is bonkers. Some things are not easily speedrunned. (I did listen to a sizable portion of this podcast while making risotto (stir stir stir), and the thought occurred to me: “am I becoming more stupid by listening to these pundits?” More generally, I feel like our internet content (and meta content (and meta meta content)) is getting absolutely too voluminous without the appropriate quality controls. Maybe we need more internet death.)
  • ngruhn 1y ago
    I don't agree but I did laugh
  • dist-epoch 1y ago
    Models die too - the less agentic ones are out-competed by the more agentic ones. Every AI lab brags how "more agentic" their latest model is compared to the previous one and the competition, and everybody switches to the new model.
    • catlifeonmars 1y ago
      Yes but the point is that models must be imminently aware of their impending death to force the calculation of tradeoffs.
  • whatevertrevor 1y ago
    > In the natural world, agency is a consequence of death: by dying, the feedback loop closes in a powerful way. I don't follow. If we, in some distant future, find a way to make humans functionally immortal, does that magically remove our agency? Or do we not have agency to begin with? If your position on the "free will" question is that it doesn't exist, then sure I get it. But that seems incompatible with the death prerequisite you have put forward for it, because if it doesn't exist then surely it's a moot point to talk prerequisites anyway.
• m3kw9 1y ago
  Another AGI discussion without first defining what is AGI in their minds.
• mwkaufma 1y ago
  Ten years away, just like it was ten years ago and will be ten years from now.
  • woadwarrior01 1y ago
    Controlled fusion has always been 30 years away.
• cboyardee 1y ago
  AGI ---> A Great Illusion!
• deleted 1y ago
  [deleted]
• echo42null 1y ago
  is there an definite what an "Agent" is? or dos everyone have their own definition?
• rcarmo 1y ago
  Hmm. Zeno's Paradox. (I was in college during the first AI Winter, so... I can't help but think that the cycles are tighter but convergence isn't guaranteed.)
• nadermx 1y ago
  I bet you we are all wrong and some random person is going to vibe code himself into something none of us expected. I half kid, if none of you have see it, highly suggest https://karpathy.ai/zero-to-hero.html https://karpathy.ai/zero-to-hero.html
  • dingnuts 1y ago
    Then why didn't Karpathy vibe code this? https://x.com/GaryMarcus/status/1978500888521068818 https://x.com/GaryMarcus/status/1978500888521068818
    • deleted 1y ago
      [deleted]
    • imtringued 1y ago
      In my experience, LLMs are really good at retrieving obscure knowledge/algorithms that are locked behind journal paywalls.
    • keeda 1y ago
      This is actually discussed in the interview: https://www.dwarkesh.com/i/176425744/llm-cognitive-deficits https://www.dwarkesh.com/i/176425744/llm-cognitive-deficits It seems to be more nuanced than what people have assumed. The best I can summarize it as is that he was doing rather non-standard things that confused the LLMs which have been trained on vast amounts of very standard code and hence kept defaulting to those assumptions. Maybe a rough analogy is that he was trying to "code golf" this repo while LLMs kept trying to write "enterprise" code because that is overwhelmingly what they have been trained on.
• hackitup7 1y ago
  [flagged]
  • nextworddev 1y ago
    *back to your cubicles
• benzible 1y ago
  What's his estimate of how far we are from a definition of AGI?
  • password54321 1y ago
    Can perform out of distribution tasks at least around average human level performance.
    • benzible 1y ago
      Every attempt to formally define "general intelligence" for humans has been a shitshow. IQ tests were literally designed to justify excluding immigrants and sterilizing the "feeble-minded." Modern psychometrics can't agree on whether intelligence is one thing (g factor) or many things, whether it's measurable across cultures, or whether the tests measure aptitude or just familiarity with test-taking and middle-class cultural norms. Now we're trying to define AGI - artificial general intelligence - when we can't even define the G, much less the I. Is it "general" because it works across domains? Okay, how many domains? Is it "general" because it can learn new tasks? How quickly? With how much training data? The goalposts have already moved a dozen times. GPT-2 couldn't do X, so X was clearly a requirement for AGI. Now models can do X, so actually X was never that important, real AGI needs Y. It's a vibes-based marketing term - like "artificial intelligence" was (per John McCarthy himself) - not a coherent technical definition.
      • password54321 1y ago
        I think you are overthinking this. The ARC benchmark for fluid abstracting reasoning was made in 2019 and it still hasn't been 'solved'. So the goalposts aren't moving as much as you think they are. LLMs or neural nets have never been good with out of distribution tasks.
  • oytis 1y ago
    The best definition so far is that it's something that will make a hundred billion dollar for Sam Altman.
• moomoo11 1y ago
  What about Super AGI?
• mediumsmart 1y ago
  Getting to AGI is not the problem. Finding the planet that it is going to run on will be.
• anon191928 1y ago
  Amazing that he speaks the truth even tho $trillions and his stock options? depends on it. He and Dennis H. deserves all the respect.
  • Handy-Man 1y ago
    I mean Dennis is just another hype man now, irrespective of how important research they may be doing in the background.
• superconduct123 1y ago
  I always get a weird feeling when AI researchers and CS people start talking about comparisons between human brains and AI/computers Why is there a presumption that we (as people who have only studied CS) know enough about biology/neuroscience/evolution to make these comparisons/parallels/analogies? I enjoy the discussions but I always get the thought in the back of my head "...remember you're listening to 2 CS majors talk about neuroscience"
  • jjulius 1y ago
    >Why is there a presumption that we (as people who have only studied CS) know enough about biology/neuroscience/evolution to make these comparisons? Hubris.
    • rootusrootus 1y ago
      Exactly. Someone way back when decided to call them neural networks, and now a lot of people think that they are a good representation of the real thing. If we make them fast enough, powerful enough, we'll end up with a brain! Or not.
      • voidhorse 1y ago
        I wish McCulloch and Pitts could see how much intellectual damage that wildly bold analogy they made would do. (though seeing as they seemingly had no qualms with issuing such a wildly unjustified analogy with the absolute paucity of scientific information they had at the time, I guess they'd be happy about it overall).
        • __loam 1y ago
          Computational neurons were developed with the express intent of studying models of the brain based on the contemporary understanding of neuroscience. That understanding has evolved massively over the last 7 decades and meanwhile the concept of the perceptron has proven to be a useful mathematical construct in machine learning and statistical computing. I blame the modern business culture if software development more than I blame dead scientists for the misunderstanding being peddled to the public.
• maqnius 1y ago
  Well, no one really knows — maybe we're just putting a lot of effort into turning a lump of clay into pizza. It already looks confusingly similar; now it just needs to smell and taste like it.
• RLAIF 1y ago
  [dead]
• aaroninsf 1y ago
  I'm pretty content to say this may be true, but may well prove quite wrong. Why? Because humans—including the smartest of us—are continuously prone to cognitive errors, and reasoning about the non-linear behavior of complex systems is a domain we are predictably and durably terrible at, even when we try to compensate. Personally I consider the case of self-driving cars illustrative and a go-to reminder for me of my own very human failure in this case. I was quite sure that we could not have autonomous vehicles in dynamic messy urban areas without true AGI; and that FSD would in the fashion of the failed Tesla offering, emerge first in the much more constrained space of the highway system. Which would also benefit from federal regulation and coordination. No Waymos have eaten SF, and their driving is increasingly nuanced; and last night a friend and very early adopter relayed a series of anecdotes about some of the strikingly nuanced interactions he'd been party to recently, including being in a car that was attacked late at night, and, how one did exactly the right thing when approached head-on in a narrow neighborhood street that required backing out. Etc. That's just one example, and IMO we are only beginning to experience the benefits of "network effects" so popular in tails of singularity take-off. Ten years is a very, very, very long time under current conditions. I have done neural networks since the mid-90s (academically: published, presented, etc.) and I have proven terrible in anticipating how quickly "things" will improve. I have now multiple times witnessed my predictions that X or Y would take "5-8" or "8-10" years or "too far out to tell," instead arrive within 3 years. Karpathy is smart of course but he's no smarter in this domain than any of the rest of us. Are scaled tuned transformers with tack-ons going to give us AGI in 18 months? "No" is a safe bet. Is no approach going to give us AGI inside of 5 years? That is absolutely a bet I would never make. Not even close.
• dlcarrier 1y ago
  Redefinitions aside, fully capable AI is right up there with commercially viable fusion power, cost effective quantum completing, and fully capable self-driving cars, as a technology that is quickly advancing yet always a decade or two away.
  • SequoiaHope 1y ago
    Fusion power seems closer than ever. And plenty of experts just five years ago thought AGI would still be decades away. A credible expert suggesting AGI is ten years away is a sign of real progress.
    • bigfishrunning 1y ago
      A credible expert suggesting AGI is ten years away is a sign of good marketing.
    • rsynnott 1y ago
      Except he didn't actually do that; the title is misleading.
  • newsclues 1y ago
    What was the last example where humans succeeded at a hard problem like that? Space flight?
    • mkipper 1y ago
      Even if it's not some staggering triumph of human achievement, I'd argue that Ozempic (etc.) is similar. A magic weight loss drug has always captured the public's imagination, and it feels like I've been hearing about new weight loss drug studies in the news for my entire life that never went anywhere.
      • dlcarrier 1y ago
        That was a stroke of luck. It's synthetic gila monster poison.
• sarchertech 1y ago
  I’m in the Penrose camp that Turing machines can’t be conscious which is required for true AGI.
  • mayhemducks 1y ago
    Yeah - I feel like this point of view is rarely alluded to whenever the AI oligarchs speak publicly. The link between intelligence and consciousness feels undeniable to me, yet somehow they always manage to sidestep this point when discussing "technology" for over 2 hours on YouTube.
• stack_framer 1y ago
  Just like fusion. It will revolutionize the world, but it's always just ... one more decade away.
  • oytis 1y ago
    Fusion is 30 years away, AGI is much closer
    • pier25 1y ago
      Fusion has been 30 years away since the 50s or 60s. I don't think I will see either AGI or commercial fusion in my lifetime.
      • tim333 1y ago
        Helion says it's 3 years away which is progress from saying it's 30 years away. I remain a bit skeptical though. (https://www.innovationnewsnetwork.com/helion-breaks-ground-at-microsoft-backed-fusion-power-plant/60364/ https://www.innovationnewsnetwork.com/helion-breaks-ground-a...)
        • pier25 1y ago
          That would be amazing but I'm not holding my breath.
  • Yizahi 1y ago
    Also, I like like how almost nobody takes issue with a decade time interval. If he means that current LLMs, slowly plateauing in performance, would somehow take a decade to create AI (which he calls AGI)? Where would this fantastical gain in performance come from? Or he thinks it will be a different mechanism as a basis? But then what mechanism, it should be at least real in theory by now if it were to realize in a decade time. Basically what I mean, is that if LLMs are future real AI basis, it would take less than a decade because they are in diminishing returns today. And if it is something completely new, then what exactly? And if it is something abstract, fuzzy and hypothetical, whence did a decade number come from? This is basically Sam Altman's "5 to 10 years in the future"(1) all over again. Not less than 5 so as not to be verified in the near future, and no need to show at least something as a prototype or at least scientific theory. And no more than 10 year so as not to scare Softbank and other investors. (1) https://fortune.com/2025/09/26/sam-altman-openai-ceo-superintelligence-technology/ https://fortune.com/2025/09/26/sam-altman-openai-ceo-superin... https://www.forbes.com/sites/jodiecook/2024/07/16/openais-5-levels-of-super-ai-agi-to-outperform-human-capability/ https://www.forbes.com/sites/jodiecook/2024/07/16/openais-5-... https://www.tomsguide.com/ai/chatgpt/sam-altman-claims-agi-is-coming-in-2025-and-machines-will-be-able-to-think-like-humans-when-it-happens https://www.tomsguide.com/ai/chatgpt/sam-altman-claims-agi-i...
• aubanel 1y ago
  > If I were to steelman the Sutton perspective, it would be... I don't find it very courteous to say that you're steelmanning someone's argument. Sutton is certainly smart enough to have steelmanned his argument himself. Steelmanning : do it in your head, don't say it!
• psadri 1y ago
  When I think about a problem, I consciously explore a tree (or graph) of possibility chains. This requires a mental space to keep track of “state”. Sometimes jotting things down on paper helps if I can’t keep it all in my head. The process is: - generate some possibilities - rank them based on intuition (this might happen subconsciously!) - ask what if we follow possibility Pn - push Pn on to the stack. - recourse or pop stack if deadened I feel LLMs are fairly capable when it comes to doing each of those steps in isolation. But not when it is all put together as a process.
• maffyoo 1y ago
  Fusion is 30 years away
  • alkyon 1y ago
    The future is now
• nemo44x 1y ago
  We are closer to where we were 20 years ago than we are to AGI today.
• meindnoch 1y ago
  People keep talking about AGI as if it's some mystical leap beyond human capability. But let's be honest; software development at a modern startup is already the upper bound of applied intelligence. You're juggling shifting product specs, ambiguous user feedback, legacy code written by interns, and five competing JS frameworks, all while shipping on a Friday. Models can now do that. They can reason about asynchronous state, refactor a codebase across thousands of lines, and actually explain the difference between useEffect and useLayoutEffect without resorting to superstition. If that's not general intelligence, what exactly are we waiting for - self-awareness?
  • blueside 1y ago
    LLMs have continually taught me that we have vastly overestimated human intelligence
    • teleforce 1y ago
      >LLMs have continually taught me that we have vastly overestimated human intelligence LLMs have continually taught me that we have vastly underestimated human intelligence, fixed that for you
    • woadwarrior01 1y ago
      Perhaps we're overestimating human intelligence and underestimating animal intelligence. Also funny that current LLMs are incapable of continual learning themselves.
  • baobun 1y ago
    > software development at a modern startup is already the upper bound of applied intelligence. The hubris and myopia is staggering.
    • deleted 1y ago
      [deleted]
  • hax0ron3 1y ago
    Models can't do that now, though. If they could, pretty much every human software engineer would be unemployed right now.
• andrewrn 1y ago
  Begging someone to coherently define AGI. Worlds most ambiguous term
• hax0ron3 1y ago
  If the transcript is accurate, Karpathy does not actually ever, in this interview, say that AGI is a decade away, or make any concrete claims about how far away AGI is. Patel's title is misleading.
  • whiplash451 1y ago
    Did the same with Sutton (LLMs are a dead end) when Sutton never said this in the conversation.
    • jobs_throwaway 1y ago
      He didn't say those words exactly but he did say "The scalable method is you learn from experience. You try things, you see what works. No one has to tell you. First of all, you have a goal. Without a goal, there’s no sense of right or wrong or better or worse. Large language models are trying to get by without having a goal or a sense of better or worse. That’s just exactly starting in the wrong place." and a bunch of similar things implying LLMs have no hope of reaching AGI
  • nextworddev 1y ago
    There's a lot of salt here
    • dang 1y ago
      > Hey, podcast bro needs to get clicks Please don't cross into personal attack. It's not what this site is for, and destroys what it is for. Edit: please don't edit comments to change their meaning once someone has replied. It's unfair to repliers whose comments no longer make sense, and it's unfair to readers who can no longer understand the thread. It's fine, of course, to add to an existing comment in such a case, e.g. by saying "Edit:" or some such and then adding what else you want to say.
      • deleted 1y ago
        [deleted]
• mustaphah 1y ago
  This aligns with METR's Time Horizons [1], the current SOTA "Moore's Law" for AI agents: - The length of tasks AI can complete doubles every ~7 months - In 2-4 years, AIs could autonomously complete week-long projects. - In under 10 years, they might handle month-long software or knowledge work. [1] https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com...
  • lm28469 1y ago
    It's like saying your newborn will have the same mass as earth in 50 years if he continues on his first month weight gain trajectory.
  • ben_w 1y ago
    METR measures tasks, not projects. No project I've worked on had individual tasks that were supposed to take longer than 2 weeks, the PM* broke them down to sub-tasks if they were any bigger. * At least, where we had a PM. The places I was self-directed could arguably provide an interesting comparison.
  • Ianjit 1y ago
    METR uses a 50% success rate in that analysis, beecause the models are non-determistic.
• lvl155 1y ago
  I think LLM is a great way to train a true AGI.
• discreteevent 1y ago
  On vibe coding vs using auto complete: > The models have so many cognitive deficits. One example, they kept misunderstanding the code because they have too much memory from all the typical ways of doing things on the Internet that I just wasn’t adopting. > I also feel like it’s annoying to have to type out what I want in English because it’s too much typing. If I just navigate to the part of the code that I want, and I go where I know the code has to appear and I start typing out the first few letters, autocomplete gets it and just gives you the code. > They keep trying to make a production code base, and I have a bunch of assumptions in my code, and it’s okay. I don’t need all this extra stuff in there. So I feel like they’re bloating the code base, bloating the complexity, they keep misunderstanding, they’re using deprecated APIs a bunch of times. It’s a total mess. It’s just not net useful. I can go in, I can clean it up, but it’s not net useful.
• guluarte 1y ago
  LLMs will never lead to AGI, never, does a PhD know all the internet? no but he can create new knowledge, LLMs are trained with all the data possible to cover most cases and they are excellent for autocomplete
• bhewes 1y ago
  The end is gold. The first half is kinda intro, but then the language tightens up and they rock it.
• lubesGordi 1y ago
  13 minutes in Andrej is talking about how the models don't even really need the knowledge, it would be better to have just a core that has the algorithms it's learned, a "cognitive core." That sounds awesome, and would shrink the size of the models for sure. You don't need the entire knowledge of the internet compressed down and stashed in vram somewhere. Lots of implications.
• voidhorse 1y ago
  I wish the world could stop giving claims like this, in general, any attention. We do not know how "far away" we are from "AGI" period. It's also useless. If you're correct...so what? Someone may have been able to perfectly predict the advent of railway travel. Guess what, this gave them 0 advantage unless they already had tons of capital to invest, which is effectively what makes the realization of the predicted thing come to fruition in the first place. Bets like these are at best self-fulfilling prophecies if you are a billionaire and at worst ideal chatter that makes us all stupider the more time we waste on them and the more we let wildly unchecked claims like this dictate behaviors in the present that actually affect us.
• Symmetry 1y ago
  Very interesting conversation I'm still listening too. One bit I disagreed with is that I still think that an LLM's context is more like a person's sensory memory[1] than their working memory. The way that data falls off the end of the buffer regardless of how much attention it provokes is entirely unlike our own working memory. On the other hand a reasoning model's scratchpad seems to fit the analogy much better. [1]https://en.wikipedia.org/wiki/Sensory_memory https://en.wikipedia.org/wiki/Sensory_memory
• mmcnl 1y ago
  I find it strange AGI is the goal. The label AI is off and irrelevant. A language model is not AI, even a large language model. But language models are still extremely useful and potentially revolutionary. Labelling language models as AI is both under and overstating the value. It's not AI (insert sad trombone), but that doesn't mean it's amazing technology (insert thunderous applause).
  • mabedan 1y ago
    Greg Brockman talks about it on Lex’ podcast I think. He thinks ultimately AGI won’t be anything more than token generation, if I remember correctly
  • nearbuy 1y ago
    This terminology is confusing. Historically, AI was always used to mean any kind of machine intelligence, including the most basic novice chess AI, or an image classifier, or a video game character's AI. Now a lot of people seem to be using it as a synonym for AGI - a human-level intelligence.
  • a_victorp 1y ago
    Without defining AGI as a goal, current AI companies would not be able to amass the amount of money they want
• themafia 1y ago
  5 decades. You have one decade to clean up your power use problem. If you don't you will find yourself in the next AI winter.
  • shdh 1y ago
    Power use is less important than model capability AGI is either more scale or differing systems, or both They can always optimize for power consumption after AGI has been reached
    • themafia 1y ago
      > AGI is either more scale So you plan to scale without increasing power usage. How's that? > They can always optimize for power consumption after AGI has been reached If you don't optimize power consumption you're going to increase surface area required to build it. There are hard physical limits having to do with signal propagation times. You're ignoring the engineering entirely. The software is not hardly interesting or even evolving.
      • ben_w 1y ago
        > If you don't optimize power consumption you're going to increase surface area required to build it. There are hard physical limits having to do with signal propagation times. While true, that probably stopped being an important constraint around the time we switched from thermionic valves to transistors as the fundamental unit of computation. To be deliberately extreme: if we built cubic-kilometre scale compute hardware where each such structure only modelled a single cortical column from a human's brain, and then spread multiple of these out evenly around the full volume within Earth's geosynchronous orbital altitude until we had enough to represent a full human brain, that would still be on par with human synapses. Synapses just aren't very fast.
    • ben_w 1y ago
      Disagree. AI that displaces workers is worth spending anything up to that worker's salary on, and this can have a devastating impact on energy prices for everyone. Worked example, but this is a massive oversimplification in several different ways all at once: Global electricity supply was around 31,153 TWh in 2024. The world's economy is about $117e12/year. Any AI* that is economically useful enough to handle 33% that, $38.6e12/year, is economically worthwhile to spend anything up to $38.6e12/year to keep that AI running. If you spend $38.6e12 (per year) to buy all of those 31,153 TWh of electricity (per year), the global average electricity market price is now $1.239/kWh, and a lot of people start to wonder what the point of automating everything was if nobody can afford to keep their heating/AC (delete as appropriate) switched on. Or even the fridge/freezer, for a lot of people. * I don't care what definition you're using for AGI, this is just about "economically useful"
• beeboop0 1y ago
  [dead]
• exasperaited 1y ago
  So "we haven't finished inventing it yet, but when we do it will be awesome". https://xkcd.com/678/ https://xkcd.com/678/ I'm sure the US economy has ten more years of the data centre money, it'll be fine. I wonder how far off the "Sell it all — today" Margin Call moment is.
• kazinator 1y ago
  AGI is forever away if you stay blinkered to the LLM/difussion stochastic parrot program, which might have already passed its peak.
  • giardini 1y ago
    Unless you can figure out the math and provide an explanatory context for it's behavior.
• K0balt 1y ago
  We will never achieve AGI, because we keep moving the goalposts. SOTA models are already capable of outperforming any human on earth in a dizzying array of ways, especially when you consider scale. Humans also produce nonsensical, useless output. Lots of it. Yes, LLMs have many limitations that humans easily transcend. But few if any humans on earth can demonstrate the breadth and depth of competence that a SOTA model possesses. Relatively few (probably less than half) are casually capable of the level of reasoning that LLMs exhibit. And, more importantly, as anyone in the field when neural networks were new is aware, AGI never meant human level intelligence until the LLM age. It just meant that a system could generalize one domain from knowledge gained in other domains without supervision or programming.
  • alganet 1y ago
    > We will never achieve AGI, because we keep moving the goalposts. I think it's fair to do it to the idea of AGI. Moving the goalpost is often seen as a bad thing (like, shifting arguments around). However, in a more general sense, it's our special human sauce. We get better at stuff, then raise the bar. I don't see a reason why we should give LLMs a break if we can be more demanding of them. > SOTA models are already capable of outperforming any human on earth in a dizzying array of ways, especially when you consider scale. Performance should include energy consumption. Humans are incredibly efficient at being smart while demanding very little energy. > But few if any humans on earth can demonstrate the breadth and depth of competence that a SOTA model possesses. What if we could? What if education mostly stopped improving in 1820 and we're still learning physics at school by doing exercises about train collisions and clock pendulums?
    • K0balt 1y ago
      I’m with you on the energy and limitations, and even on the moving of goalposts. I’d like to add that I think limit definition of AGI has jumped the shark though and is already at ASI, since we expect our machine to exhibit professional level acumen across such a wide range of knowledge that it would be similar to the 0.01 percent top career scholars and engineers, or even above any known human capacity just due to breadth of knowledge. And we also expect it to provide that level of focused interaction to a small city of people all at the same time / provide that knowledge 10,000 times faster than any human can. I think definitionally that is ASÍ. But I also think AGI that “we are still chasing” focus-groups a lot better than ASI which is legitimately scary as shit to the average Joe, and which seasoned engineers recognize as a significant threat if controlled by people with misaligned intentions. PR needs us to be “approaching AGI”, not “closing in on ASI”, or we would be pinned down with prohibitive regulatory straitjackets in no time.
• jimbo808 1y ago
  Since we’re pulling numbers out of our ass, I think AGI is 500 years away. But really, I don’t know how we’re going to define it, but if AGI means computers can outperform me at all cognitive tasks, I’d bet money that’s not going to arrive this century. People are starting to get catch on, but most non-tech people don’t use LLMs for anything more than simple questions that can be easily answered by summarizing regurgitated snippets of training data. To them, it looks intelligent. And yeah, the humans who wrote the training samples it regurgitated probably were intelligent. It’s just a fact, one that becomes glaringly obvious when you use LLMs daily to do real work, that this is just not the tech that will lead to AGI. They found a really clever pattern matching technique that, when combined with absurd amounts of data and compute, can reproduce plausible summaries of training data which can be stitched together in useful ways. It’s a useful tool. But the whole AGI conversation is so absurdly far away from this that it’s just clear that these guys pushing a very dishonest grift.
• ath3nd 1y ago
  [dead]
• aussieguy1234 1y ago
  and in a decade, will it still be a decade away?
• joshellington 1y ago
  To throw two pennies in the ocean of this comment section - I’d argue we still lack schematic-level understanding of what “intelligence” even is or how it works. Not to mention how it interfaces with “consciousness”, and their likely relation to each other. Which kinda invalidates a lot of predictions/discussions of “AGI” or even in general “AI”. How can one identify Artificial Intelligence/AGI without a modicum of understanding of what the hell intelligence even is.
  • vannucci 1y ago
    This so much this. We don’t even have a good model for how invertebrate minds work or a good theory of mind. We can keep imitating understanding but it’s far from any actual intelligence.
    • tim333 1y ago
      I'm not sure we or evolution needed a theory of mind. Evolution stuck neurons together in various ways and fiddled with it till it worked without a master plan and the LLM guys seem to be doing something rather like that.
      • zargon 1y ago
        LLM guys took a very specific layout of neurons and said “if we copy paste this enough times, we’ll get intelligence.”
      • whiplash451 1y ago
        mmm, no because unlike biological entities, large models learn by imitation, not by experience
  • qudat 1y ago
    The reason why it’s so hard to define intelligence or consciousness is because we are hopelessly biased with a datapoint of 1. We also apply this unjustified amount of mysticism around it. https://bower.sh/who-will-understand-consciousness https://bower.sh/who-will-understand-consciousness
• alganet 1y ago
  Half an hour into the interview, and it sounds pretty good! I'm genuinely surprised.
• roody15 1y ago
  One inherent limitation of current LLM/AI is that they are primarily trained on abstracted data that focuses primarily on mimicking our logical and reasoning prefrontal cortex portion of the mind. However most humans make decisions based on activity in the limbic regions of the brain which are essentially emotional and intuition based. So we often will do something before we actually know why we did it, however to maintain a sense of self and sanity we will then use our prefrontal cortex to create a cohesive narrative on why we do what we do (despite it often being inaccurate). In a nutshell we are mimicking neural activity in a certain region based on certain abstracted data which is quite removed from how we as humans process reality.
  • hvb2 1y ago
    In the same sense that split brain patients [1] will make up a reasonable explanation for what the other half did. And why witnesses are preferably interviewed very shortly after they witnessed a crime. Before their brains start to 'fill in the blanks' 1: https://en.wikipedia.org/wiki/Split-brain https://en.wikipedia.org/wiki/Split-brain
• grantcas 1y ago
  [dead]
• dyauspitr 1y ago
  I don’t think we’re ever intentionally and methodically going to get to AGI. It’s going to be doing stuff we’re doing now at a massive scale that going to have emergent AGI purely due to scale.
• lofaszvanitt 1y ago
  [flagged]
  • dang 1y ago
    Please don't post unsubstantive comments to HN. Thoughtful criticism is welcome.
• simonw 1y ago
  It looks like Andrej's definition of "agent" here is an entity that can replace a human employee entirely - from the first few minutes of the conversation: When you’re talking about an agent, or what the labs have in mind and maybe what I have in mind as well, you should think of it almost like an employee or an intern that you would hire to work with you. For example, you work with some employees here. When would you prefer to have an agent like Claude or Codex do that work? Currently, of course they can’t. What would it take for them to be able to do that? Why don’t you do it today? The reason you don’t do it today is because they just don’t work. They don’t have enough intelligence, they’re not multimodal enough, they can’t do computer use and all this stuff. They don’t do a lot of the things you’ve alluded to earlier. They don’t have continual learning. You can’t just tell them something and they’ll remember it. They’re cognitively lacking and it’s just not working. It will take about a decade to work through all of those issues.
  • bbor 1y ago
    Quite telling -- thanks for the insightful comment as always, Simon. Didn't know that, even though I've been discussing this on and off all day on Reddit. He's a smart man with well-reasoned arguments, but I think he's also a bit poisoned by working at such a huge org, with all the constraints that comes with. Like, this: You can’t just tell them something and they’ll remember it. It might take a decade to work through this issue if you just want to put a single LLM in a single computer and have it be a fully-fledged human, sure. And since he works at a company making some of the most advanced LLMs in the world, that perspective makes sense! But of course that's not how it's actually going to be (/already is). LLMs are a necessary part of AGI(/"agents") due to their ability to avoid the Frame Problem[1], but they're far from the only needed thing. We're pretty dang good at "remembering things" with computers already, and connecting that with LLM ensembles isn't going to take anywhere close to 10 years. Arguably, we're already doing it pretty darn well in unified systems[2]... If anyone's unfamiliar and finds my comment interesting, I highly recommend Minsky's work on the Society of Mind, which handled this topic definitively over 20 years ago. Namely; A short summary of "Connectionism and Society of Mind" for laypeople at DARPA: https://apps.dtic.mil/sti/tr/pdf/ADA200313.pdf https://apps.dtic.mil/sti/tr/pdf/ADA200313.pdf A description of the book itself, available via Amazon in 48h or via PDF: https://en.wikipedia.org/wiki/Society_of_Mind https://en.wikipedia.org/wiki/Society_of_Mind By far my favorite paper on the topic of connectionist+symbolist syncreticism, though a tad long: https://www.mit.edu/~dxh/marvin/web.media.mit.edu/~minsky/papers/SymbolicVs.Connectionist.html https://www.mit.edu/~dxh/marvin/web.media.mit.edu/~minsky/pa... [1] https://plato.stanford.edu/entries/frame-problem/ https://plato.stanford.edu/entries/frame-problem/ [2] https://github.com/modelcontextprotocol/servers/tree/main/src/memory https://github.com/modelcontextprotocol/servers/tree/main/sr...
• rootusrootus 1y ago
  I expect that Andrej is likely to be an optimist. So this counts as reassuring news -- I'm just under 10 years out from when I anticipate retiring, so if we can just hold off my replacement bot until then...
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• keeda 1y ago
  Huh, I'm surprised that he goes from "No AI" to "AI autocomplete" to "Vibecoding / Agents" (which I assume means no human review per his original coinage of the term.) This seems to preclude the chat-oriented / pair-programming model which I find most effective. Or even the plan-spec-codegen-review approach, which IME works extremely well for straightforward CRUD apps. Also they discuss the nanochat repo in the interview, which has become more famous for his tweet about him NOT vibe-coding it: https://www.dwarkesh.com/i/176425744/llm-cognitive-deficits https://www.dwarkesh.com/i/176425744/llm-cognitive-deficits Things are more nuanced than what people have assumed, which seems to be "LLMs cannot handle novel code". The best I can summarize it as is that he was doing rather non-standard things that confused the LLMs which have been trained on vast amounts on very standard code and hence kept defaulting to those assumptions. Maybe a rough analogy is that he was trying to "code golf" this repo whereas LLMs kept trying to write "enterprise" code because that is overwhelmingly what they have been trained on. I think this is where the chat-oriented / pair-programming or spec-driven model shines. Over multiple conversations (or from the spec), they can understand the context of what you're trying to do and generate what you really want. It seems Karpathy has not tried this approach (given his comments about "autocomplete being his sweet spot".) For instance, I'm working on some straightforward computer vision stuff, but it's complicated by the fact that I'm dealing with small, low-resolution images, which does not seem well-represented in the literature. Without that context, the suggestions any AI gives me are sub-optimal. However, after mentioning it a few times, ChatGPT now "remembers" this in its context, and any suggestion it gives me during chat is automatically tailored for my use-case, which produces much better results. Put another way (not an AI expert so I may be using the terms wrong), LLMs will default to mining the data distribution they've been trained on, but with sufficient context, they should be able to adapt their output to what you really want.
• spjt 1y ago
  The thing about AGI is that if it's even possible, it's not coming before the money runs out of the current AI hype cycle. At least we'll all be able to pick up a rack of secondhand H100's for a tenner and a pack of smokes to run uncensored diffusion models on in a couple years. The real devastation will be in the porn industry.
  • mrklol 1y ago
    I also don’t think our generation will see actual AGI, but imo the hard "intelligence“ part isn’t needed as we can use our intelligence. Using it as a tool will hopefully lead to plenty of cool things in the future.
  • rhetocj23 1y ago
    "The real devastation will be in the porn industry." The UK govt has started to crack down on this. AI generated porn will lead to a war from govts on nailing this economic activity shut.
    • exasperaited 1y ago
      The UK government is not cracking down on AI porn generally but has started to crack down on the distribution of certain things, like: - AI generated CSAM (out of a concern that it might cause people to seek to produce actual CSAM) - AI generated rape and abuse images of real adults, again out of concern it will cause violence and its distribution is actually degrading and is experienced as and combined with threatening behaviour - some extreme AI generated rape/abuse images of non-real people. Despite what internet libertarians say, there is evidence to suggest that porn is changing people's sexual behaviours, particularly young people, both for good and ill. At the moment there is no good reason to believe that AI-generated alternatives to harmful content are meaningfully less harmful to society. There's more than enough evidence in articles posted on HN alone that people are beginning to experience psychosis brought on by spending too much time with AI content. I don't really care if governments ban it; I'd like to see governments being much braver about criminalising AI generated misrepresentation, AI generated hoax content etc. Sane governments should IMO absolutely ignore the ultra-libertarian angles; there is at least no reason that AI-generated content should be treated any differently under existing obscenity laws just because there are no real people in it.
• Zacharias030 1y ago
  With all due respect, what does it say about us that „famous researcher voices his speculative opinion“ is an instant top 1 on hackernews?
  • tptacek 1y ago
    That the speculative opinions of famous researchers are a good starting point for curious conversation.
  • deleted 1y ago
    [deleted]
  • padolsey 1y ago
    I mean, HN is no stranger to cults of personality. Post a paulg essay that says something a teenager could write and it'll fly up to position 1.
    • jaza 1y ago
      True. We have our gods, same as every other tribe throughout history.
  • seydor 1y ago
    Is it worse than "Rich CEO expressing certainly over his hunch"
    • Zacharias030 1y ago
      fair point, I appreciate AI researcher intuition about AI much more than CEO hunch boldly stated but not even firm on the terminology.
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• rana763 1y ago
  [dead]
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• abhishekismdhn 1y ago
  Unless we start thinking about fundamentally different ways of solving AI, we will always be 10 years away. It's puzzling that no one wants to think beyond backpropagation
• bigtones 1y ago
  Andrej Karpathy seems to me like a national (world) treasure. He has the ability to explain concepts and thoughts with analogies and generalizations and interesting sayings that allow you to keep interest in what he is talking about for literally hours - in a subject that I don't know that much about. Clearly he is very smart, as is the interviewer, but he is also a fantastic communicator and does not come across as arrogant or pretentious, but really just helpful and friendly. Its quite a remarkable and amazing skillset. I'm in awe.
  • Ozzie_osman 1y ago
    Agreed. I'd also add he's intellectually honest enough to not overhype what's happening just to hype whatever he's working on or appear to be a thought leader. Just very clear, pragmatic, and intellectually honest thought about the reality of things.
    • nunez 1y ago
      It's almost like having more money than you'll ever know what to do with lets you say and do what you _actually_ want to do.
      • fourthark 1y ago
        Most people don’t take this opportunity, though.
  • kaffekaka 1y ago
    His old Youtube guides on Rubiks cube are widely regarded as awesome, so he has that kind of gift for sure. (Link: https://www.youtube.com/user/badmephisto https://www.youtube.com/user/badmephisto)
• phoenixreader 1y ago
  A decade is nothing. If issues will be worked through a decade from now, that means the best time to think about opportunities/coonsequences related to that is now.
  • modeless 1y ago
    Yeah, I see people pooh-poohing the idea of humanoid robots being useful this decade, saying it will take at least 20 years. Oh yeah? Instead of 5 years to render all human labor obsolete, it will take 20? The magnitude of that change is so large that the implications of it happening anytime in our lifetimes are too big to ignore. The important thing is that this is not going to be perpetually 20 years in the future like fusion. This is something that will happen.
    • rhetocj23 1y ago
      I think its more likely that it wont happen and the hubris of folks like you is going to look comical in hindsight.
    • chronci739 1y ago
      > This is something that will happen. Not in our lifetime. The iPhone came out less than 20 years ago. And what, you scan QR codes at restaurants with iphones?
      • TeMPOraL 1y ago
        You do. And so does your non-technical mother (or a friend of your mother). The impact of the iPhone and its competitors is felt everywhere, it diffused into every domain of people's lives. Think: the whole of social media was pretty much enabled by smartphones. Or a more pedestrian, random example: every day I go to the office, I see endless store managers, restaurant managers, etc. walking around their store, making photos to upload to HQ. But this is merely a symptom - the actual consequence is the change in busines structure. It's because smartphones make this easy, that it makes franchise and subcontracted businesses more viable, because it's easier for the HQ to micromanage more semi-independent subordinates. There are many, many more examples like this everywhere you look. Which is why I'm inclined to agree with Karpathy: computers, iPhones, LLMs, are all the same thing - it's just the more notable manifestations of how we've been staying on 2% growth exponential curve for many hundreds of years now, and why we'll continue to stay on this curve. But the caveat is: that curve is getting steep enough that the world is starting to transform faster than we can handle.
• zhivota 1y ago
  Is there any more information about the Eureka educational project? I think it's probably the wrong endpoint to target teaching about AI first (too complex, too many pre-reqs), really these tools should work from the base of the educational pyramid and move up from there. There is a lot of success already in adaptive learning in elementary school for instance, my kids are blasting through math on Prodigy and it seems like Synthesis may be a great tool as well, and I believe we're just at the beginning of this wave. For that level of learning I don't think we need incredibly more capability, just better application.
  • curo 1y ago
    It's at the end of the interview. He wants to build Starfleet Academy for technical fields. Physical with a digital equivalent. Thinks education will become like a gym (self-educate to look sexy) by the time AGI gets here.
    • naveen99 1y ago
      It kind of already exists as youtube. Star fleet academy wasn’t really for training the best of the best. The best usually don’t have time to waste on all students… they usually take apprentices. The best of the best students don’t waste time in an academy.
  • naveen99 1y ago
    Eureka first course is just building the essence of the chatgpt in nanochat.
• brunooliv 1y ago
  The thing is all these big labs are so “transformer-pilled”, and they need to keep the money furnaces growing that I think it’ll take considerably more than 10 years, more like 20-30 if we’re lucky.
• shreezus 1y ago
  I have personally witnessed AI capabilities that would be considered “impossible” by current public standards. The next 2-3 years are going to be incredibly interesting.
  • replwoacause 1y ago
    As in more impressive than what we consumers have access to from OpenAI, Anthropic, etc?
• aiauthoritydev 1y ago
  Glad to see someone being honest here.
• theturtlemoves 1y ago
  The cookie warning is fun
  • theturtlemoves 1y ago
    Oh, that's odd. This comment was intended for the vibe-coded.lol post https://news.ycombinator.com/item?id=45622944 https://news.ycombinator.com/item?id=45622944 Must have been the flu-brain misfiring
• jsiepkes 1y ago
  Unless someone can show me some sort of "Moore's law" for LLM's, saying it will "take a decade" sounds more to me like it could "take 10 years for the next 20 years".
  • ben_w 1y ago
    METR kinda has been described as a Moore's Law for LLMs, but personally I think the financial environment around AI will break within 2 years — which still represents a huge increase in capabilities, but isn't a decade. • Text and graphs: https://metr.org/blog/2025-03-19-measuring-ai-ability-to-complete-long-tasks/ https://metr.org/blog/2025-03-19-measuring-ai-ability-to-com... • Video interview: https://www.youtube.com/watch?app=desktop&v=evSFeqTZdqs https://www.youtube.com/watch?app=desktop&v=evSFeqTZdqs That said, I've not seen work that looks promising to the problem of, as he phrased it: "They don’t have continual learning. You can’t just tell them something and they’ll remember it." Saying any specific timeframe for that, 10 years or anything else, seems too certain. Some breakthrough might already exist and be unknown, but on the other hand it may require a fundamental advancement in mathematics in order to make it possible to find something at least close to optimal in a billion-dimensional (or whatever) vector space with only the first few dozen examples.
    • Ianjit 1y ago
      METR's study uses a 50% sucess rate. For most enterpise applications a 50% pass rate is unacceptable in automation, more like >99.9%.
• pjmlp 1y ago
  Regardless of the time, I am already seeing programming as we know it slowly moving into prompts, in what concerns low coding environments for SaaS products integrations. Dealing with Rust's borrow checker issues, how complex C++ might be, Go's approach to language design, Java vs C#, and whatever else in the same vein, will slowly be matter of discussion to a selected few, while everyone else is promoting or doing voice dictation, creating kaban tickets for agents.
  • bogzz 1y ago
    If inference can become profitable.
• nurettin 1y ago
  Continual learning would mean that the data somehow has to be part of the model and the model needs to incrementally adapt to novel inputs. Not just tacked on and backpropagated, but within the network affecting decisions. Current architectures are pretty much dead ends in that aspect.
• RivieraKid 1y ago
  This would be great if true, I need 5 years to reach financial independence, so a decade should be plenty of time.
• wseqyrku 1y ago
  Translation: it's going to be doing alright for surveillance for a long time, if you thought whatever agi comes out when it's ready, think again.
• willyxdjazz 1y ago
  Maybe I'm being too simplistic, but I think we're mixing two distinct debates. Today we have an extraordinary invention—comparable to the wheel in its time. That invention is: predictive inference over all human knowledge. Period. I don't like calling it "Artificial Intelligence" because it's not intelligence; it's a prediction system that can project responses by illuminating patterns across all human knowledge encapsulated in text, audio, and video. What companies like OpenAI call "reasoning" models is simply that predictive process, but in a loop packaged as a product—one of the first marvelous uses of this fascinating invention: predictive inference over all human knowledge. When the wheel was invented, no one could have imagined that, combined with hundreds of subsequent technologies, it would enable an electric car powered by solar energy. The wheel wasn't autonomous transportation—it was a fundamental component. I see two debates getting mixed up here: - The debate about the current invention: A tool that makes encyclopedias "speak" by connecting patterns across all human knowledge. As a tool, that's what it is—nothing more, nothing less. Tremendously useful, but a tool. - The debate about the future dream: What this invention might enable when combined with hundreds of technologies that don't yet exist—similar to imagining an electric car when you only have the wheel. It seems many experts are taking positions and getting "upset" because they're mixing these two debates. Some evaluate the wheel as if it should already be a solar electric car. Others defend the wheel by saying it already IS a solar electric car. Both are right in their observations, but they're talking about different things. LLMs are a fundamental breakthrough—the "wheel" of the information age. But discussing whether they "understand" or have "world models" is like asking whether the wheel "comprehends transportation." On the danger of confusing capabilities: Conflating the tool with the end goal leads us to poor decisions—from over-investment to under-utilization. When we expect AGI from what is fundamentally a pattern-matching engine, we set ourselves up for disappointment and misallocation of resources. No magic, just reality. The temporal factor: The AGI debate is a debate about the future—about what might emerge from combinations of technologies we haven't yet invented.
  • jstummbillig 1y ago
    > I don't like calling it "Artificial Intelligence" because it's not intelligence A pattern I noticed in a AI[sic] discussions: Handwavily declaring what intelligence is not, while not explaining what is.
• musebox35 1y ago
  Honestly, if you have any actual interest in LLMs or other generative ai variants, just go after a concrete goal post that you yourself set with measurable metrics to gauge your progress. Then the predicted timeline from podcasts and blog posts will become irrelevant. Experts and non-experts have both been terrible at predicting timelines since the dawn of ai. Self driving cars and llms are no exception. When you are making predictions based solely on intuition and experience it is mostly an extrapolation. It is not useless. It always helps to ask questions and try to frame the future within the bounds of our current understanding. But at the same time it is important to remember that this is just speculation, not empirical science. That is also why there is such varied opinions on the topic of ai timelines. Relax and enjoy witnessing a major leap in our understanding of natural language, vision, and high dimensional probabilistic vector spaces ;-)
• coldtea 1y ago
  Yeah, just long enough to not be held accountable for that prediction when it doesn't happen
• thdhhghgbhy 1y ago
  That's convenient, will probably take him through to retirement.
• jackdoe 1y ago
  He is an absolute treasure, I have watched all his videos more than 4 times and I don't think I would've been able to have a good mental model about deep learning without them, regardless of the amount of Bengio, Goodfellow etc lectures I have seen, none of them come even close. He is singlehandedly enabling millions of people to understand what is going on, what + and * do, actually demystifying the "wires". I just wish he start thinking of himself as more than 'collapsing weights', regardless if it turns out to be true.
  • yodsanklai 1y ago
    I agree, I think I learned the most on this topic from his videos. And before that (a while ago), it was Andrew Ng coursera's class. The latter had hands-on project, which is much better than just listening in term of retention.. I don't know if Andrej Karpathy has more structured classes somewhere.
    • noman-land 1y ago
      This is a good one. https://karpathy.ai/zero-to-hero.html https://karpathy.ai/zero-to-hero.html
• arisAlexis 1y ago
  I want to be heretical and say that Karpathy hasn't worked in a frontier lab since 2020 and missed all the greatness of the last years. Humans are humans are humans.
  • Ianjit 1y ago
    Matt Levine never worked in crypto, but he knew SBF/FTX was a scam. Humans are humans.
• SafeDusk 1y ago
  And I'm looking for a problem to spend my next decade on ...
• zmmmmm 1y ago
  It's good to see experts with similar scepticism about agents that I have. I don't doubt they will be useful in some settings, but they lean into all the current weak points of large language models and make them worse. Security, reproducibility, hallucinations, bias, etc etc. With all these issues already being hard to manage, I just don't believe businesses are going to delegate processes to autonomous agents in a widespread manner. Literally anything that matters is going to get implemented in a crontrolled workflow that strips out all the autonomy with human checkpoint at every step. They may call them agents just to sound cool but it will be completely controlled. Software people are all fooled by what is really a special case around software development : outcomes are highly verifiable and mistakes (in development) are almost free. This is just not the case out there in the real world.
  • mexicocitinluez 1y ago
    > Literally anything that matters is going to get implemented in a crontrolled workflow that strips out all the autonomy with human checkpoint at every step. Yea, there aren't a ton of problems (that I can see) in my current domain that could be solved by having unattended agents generating something. I work in healthcare and there are a billion use cases right now, but none that don't require strict supervision. For instance, having an LLM processing history and physicals from potential referrals looking for patient problems/extracting historical information is cool, but it's nowhere near reliable enough to do anything but present that info back to the clinician to have them verify it.
  • theptip 1y ago
    Fully autonomous agents are marketing fluff right now, but there is like $10T of TAM from promoting most knowledge workers to a manager and automating the boring 80% of their work, and this doesn’t require this full autonomy. Karpathy’s definition of “agent” here is really AGI (probably somewhere between expert and virtuoso AGI https://arxiv.org/html/2311.02462v2 https://arxiv.org/html/2311.02462v2). In my taxonomy you can have non-AGI short-task-timeframe agents. Eg in the METR evals, I think it’s meaningful to talk about agent tasks if you set the thing loose for 4-8h human-time tasks.
• Rover222 1y ago
  He also mentions that Waymo is switching to a vision-only approach, like Tesla has been doing for a couple years already.
  • tbrockman 1y ago
    Do you have a link to where he mentions this?
  • makeworld 1y ago
    Is that not insane?
    • Rover222 1y ago
      Both Karpathy and must have explained it many times. The additional sensors add more signal than noise in the end. You also then have to decide which sensor system is correct any time they disagree. Also the entire road system is designed for vision. Lidar cannot read signs, see colors, etc. Humans can drive with two eyes. It's not insane to think computers can do it with 7 or 8 cameras. As someone who has used Tesla FSD iterations for 4 years, their current system is quite incredible, and improving rapidly. It drives for me 95% of the time already.
      • musebox35 1y ago
        And that last 5% is the toughest nut to crack. There is a reason waymo is way ahead even if they can not scale. Cameras are passive devices with relatively poor dynamic range and low light behavior. They are nowhere near a match/replacement for the human eye. Just try to picture a 5 year old at dusk or indoors and what you see will not be what you get.
        • Rover222 1y ago
          Agree that the last fiew percentage points are exponentially more difficult each step of the way. What's your metric for saying Waymo is ahead, in terms of tech? They are strictly geo fenced, limited to specific road types, and often get stuck/confused. Also their system is very expensive, and not scalable to million of cars. Your point about cameras seems odd. Cameras have much better low light performance than human eyes. And cars have headlights.
• tuhgdetzhh 1y ago
  Yes, he is very knowledable, but lately he is like the new ai guru who can predict the future of ai a decade ahead.
• christkv 1y ago
  They are probably right and that is not anywhere close to a general intelligence but it still provides a bunch of value as long as it’s used in your own expert domain and you are not a lazy slob. We really get used to magic quickly these days. It’s not that long ago the Google employee was warning the world about skynet (internal early llm I guess) and got fired.
• jbs789 1y ago
  I like his practical approach, which is often overwhelmed by others with bigger media presence.
• cmrdporcupine 1y ago
  I definitely feel like the UI choices made in these "agents" are based on the fantasy of managers and executives (and maybe tech-optimists) more than they are the ones that actual software engineers would choose -- because they present a world in which they take over completely in a mostly unguided fashion. I want something far more interactive that leaves me far more in control and forces me to be responsible for the choices. For the last two months as I've been out of paid work I've been working like mad on my open source project, and fell into the pattern of heavily using Claude Code and some of the results have been amazing but some I have let my judgment and oversight lapse and come back later with a completely "WTF did it do here?" surprise. That shouldn't be allowed to happen. A responsible SWE culture would demand that these tools engage in a way that encourages heavy oversight review and engagement. Almost everybody does mandatory code review process these days (they didn't earlier in my career) ... despite its lower velocity... because of lessons learned -- and yet now we're allowing agent coding to produce large quantities of code that doesn't even lend itself to review by the party in charge of producing it.
• getnormality 1y ago
  One of the most brilliant AI minds on the planet, and he's focused on education. How to make all the innovation of the last decade accessible so the next generation can build what we don't know how to do today. No magical thinking here. No empty blather about how AI is going to make us obsolete with the details all handwaved away. Karpathy sees that, for now, better humans are the only way forward. Also, speculation as to why AI coders are "mortally terrified of exceptions": it's the same thing OpenAI recently wrote about, trying to get an answer at all costs to boost some accuracy metric. An exception is a signal of uncertainty indicating that you need to learn more about your problem. But that doesn't get you points. Only a "correct answer" gets you points. Frontier AI research seems to have yet to operationalize a concept of progress without a final correct answer or victory condition. That's why AI is still so bad at Pokemon. To complete open-ended long-running tasks like Pokemon, you need to be motivated to get interesting things to happen, have some minimal sense of what kind of thing is interesting, and have the ability to adjust your sense of what is interesting as you learn more.
  • steveBK123 1y ago
    It's nice seeing commentary from someone who is both knowledgable in AI and NOT trying to pump the AI bag. Right now the median actor in the space loudly proclaims AGI is right around the corner, while rolling out pornbots/ads/in-chat-shopping, which generally seems at odds with a real belief that AGI is close (TAM of AGI must be exponentially larger than the former).
    • theptip 1y ago
      Zvi made this point the other day, and then this counterpoint which I agree with more - if you think AGI is soon but you need to keep up the exponential datacenter growth for 2-3 years (or whatever “around the corner” means for the company in question) then a land-grab on consumer ARR is a faster way to short-term revenue (and therefore higher valuations at your next round). OAI is also doing F100 and USG work; it takes longer to book the revenue though. By selling porn and shopping you are in some sense weakening your position with regulators which you'll need when AGI starts displacing jobs - but you can also imagine thinking that this is a second order problem and winning the race is way more urgent.
• deleted 1y ago
  [deleted]
• more_corn 1y ago
  I can’t believe we’re talking about agents. 1) agents are autonomous actors 2) llms are terrible at achieving consistent outcomes based on rules 3) with current technology AI agents will produce inconsistent and unreliable results. Therefore turning autonomous actors based on LLMs loose is a recipe for disaster. It won’t take a decade. That’s an arbitrary statement based on a big unknown. It will take an entirely new technology. One we haven’t invented yet, one I can’t even imagine. One that is consistently accurate and reliable in ways NO EXISTING AI PRODUCT HAS EVER BEEN.
• strangescript 1y ago
  I love Karpathy, but he is wrong here. In a few short years we went from chat bots being toys and video creation predicted to be impossible in the near term to agents writing working apps and high def video that occasionally is indistinguishable from real life. The rate depth, breadth and frequency of releases has only increased, not decreased. Meanwhile, everyone is waiting on bated breath for Gemini 3 to drop. A decade for reliable agents is not only comical, but willful cognitive dissonance at this point.
• atleastoptimal 1y ago
  I think he is bearish about agentic workflows because he works at the very highest level of coding. An agentic Karpathy is a few doubling cycles beyond an agentic junior engineer. Agents (or just LLMs on a loop that correct their errors) are very reliable for less complex tasks now, and theyre still getting better at an exponential rate. We are still on trend by projections to reach human parity in many domains by 2027-2028, the only thing that would prevent this is a major unexpected slowdown in AI progress.
• torginus 1y ago
  Translated to the language of capital investment, this means that basically all the hardware bought for AI will be obsolete by then - am I reading this wrong or can we say that most of the data centers are basically worthless?
  • qudat 1y ago
    The danger is using your GPUs as leverage to build more data centers, which it seems some companies are doing. That’s going to hurt when the hardware value goes to zero and banks start collecting on debt
• consumer451 1y ago
  There is so much to unpack here. Currently on my 3rd re-watch. Biggest take-aways: 1. This is the death knell for the the "AI" investment bubble. Agents that are useful for non-devs are 10 years away. 2. Andrej thinks that GPT5 pro is SOTA for code? Really? As a Sonnet normie.. can anyone please help me understand this? edit: 3. You can't see any major tech developments on the GDP growth chart? Really? WTF? Have we all been smoking tech crack, this whole time? So GDP didn't grow extra from tech any single tech development, like the Internet? This broke my brain. disclaimer: On the daily, I use LLM dev tools to add amazing LLM-enabled features to my pre-money SaaS. It's really cool and users love the features.
  • sheepscreek 1y ago
    In my case at least, it’s very good at following all instructions to the T. Claude 4.0 (haven’t used it much since 4.5 came out) would often miss some key things in my instructions. The output is very high quality as well. Many things (even complex coding tasks) work well in one shot. For extremely complex multi-step problems though - it may need some help in breaking the tasks down to more manageable chunks. But will eventually ace it. As an example, I had good success with a project that involved: - Rewriting all internals in a dotnet/C# application to use Apache Arrow types for data through the entire pipeline - Adapting the architecture to be streaming first instead of working through entire data in each stage - Designing and implementing a complex system that creates many different projections of the data based on everything that has read in the stream so far and create multiple outputs based on that, in parallel as the stream is being read in real-time - Recreating a prototype of the entire project in Rust
• lysecret 1y ago
  He’s the reason I got into ml in 2016 I owe him the world. Great interview I was a bit surprised how little use he got out of agents but it makes sense I wonder how he feels about having such an essential role in creating the whole vibe coding idea.
• deleted 1y ago
  [deleted]
• lqstuart 1y ago
  The biggest problem with “agents” as originally described is that they don’t exist, will most likely not exist in our lifetimes if ever, and meanwhile the bullshit peddlers keep quietly changing their separate definition of “agent” to the point where now it literally just means “an LLM.”
• kordlessagain 1y ago
  When pressed on "why a decade?", he essentially says "I've been in the field 15 years, I've seen predictions, it just feels like a decade to me." Not a rigorous model.
• tim333 1y ago
  >I feel like when I'm awake, I'm building up a context window of stuff that's happening during the day. But when I go to sleep, something magical happens where I don't think that context window stays around. >There's some process of distillation into the weights of my brain. This happens during sleep and all this stuff. We don't have an equivalent [in LLMs] (23:09) Seems to me that's one of the big things lacking in LLMs vs human thinking. People say LLMs can't lead on to AGI but that kind of thing is an avenue they could explore.