17 ms·

Jagged AGI: o3, Gemini 2.5, and everything after

• sejje 1y ago
  In the last example (the riddle)--I generally assume the AI isn't misreading, rather that it assumes you couldn't give it the riddle correctly, but it has seen it already. I would do the same thing, I think. It's too well-known. The variation doesn't read like a riddle at all, so it's confusing even to me as a human. I can't find the riddle part. Maybe the AI is confused, too. I think it makes an okay assumption. I guess it would be nice if the AI asked a follow up question like "are you sure you wrote down the riddle correctly?", and I think it could if instructed to, but right now they don't generally do that on their own.
  • Jensson 1y ago
    > generally assume the AI isn't misreading, rather that it assumes you couldn't give it the riddle correctly, but it has seen it already. LLMs doesn't assume, its a text completer. It sees something that looks almost like a well known problem and it will complete with that well known problem, its a problem specific to being a text completer that is hard to get around.
    • simonw 1y ago
      These newer "reasoning" LLMs really don't feel like pure text completers any more.
      • jordemort 1y ago
        And yet
      • gavinray 1y ago
        Is it not physically impossible for LLM's to be anything but "plausible text completion"? Neural Networks as I understand them are universal function approximators. In terms of text, that means they're trained to output what they believe to be the "most probably correct" sequence of text. An LLM has no idea that it is "conversing", or "answering" -- it relates some series of symbolic inputs to another series of probabilistic symbolic outputs, aye?
        • int_19h 1y ago
          At this point you need to actually define what it means for an LLM to "have an idea".
      • Borealid 1y ago
        What your parent poster said is nonetheless true, regardless of how it feels to you. Getting text from an LLM is a process of iteratively attempting to find a likely next token given the preceding ones. If you give an LLM "The rain in Spain falls" the single most likely next token is "mainly", and you'll see that one proportionately more than any other. If you give an LLM "Find an unorthodox completion for the sentence 'The rain in Spain falls'", the most likely next token is something other than "mainly" because the tokens in "unorthodox" are more likely to appear before text that otherwise bucks statistical trends. If you give the LLM "blarghl unorthodox babble The rain in Spain" it's likely the results are similar to the second one but less likely to be coherent (because text obeying grammatical rules is more likely to follow other text also obeying those same rules). In any of the three cases, the LLM is predicting text, not "parsing" or "understanding" a prompt. The fact it will respond similarly to a well-formed and unreasonably-formed prompt is evidence of this. It's theoretically possible to engineer a string of complete gibberish tokens that will prompt the LLM to recite song lyrics, or answer questions about mathemtical formulae. Those strings of gibberish are just difficult to discover.
        • Workaccount2 1y ago
          The problem is showing that humans aren't just doing next word prediction too.
          • Borealid 1y ago
            I don't see that as a problem. I don't particularly care how human intelligence works; what matters is what an LLM is capable of doing and what a human is capable of doing. If those two sets of accomplishments are the same there's no point arguing about differences in means or terms. Right now humans can build better LLMs but nobody has come up with an LLM that can build better LLMs.
            • baq 1y ago
              That’s literally the definition of takeoff, when it starts it gets us to singularity in a decade and there’s no publicly available evidence that it’s started… emphasis on publicly available.
    • monkpit 1y ago
      This take really misses a key part of implementation of these LLMs and I’ve been struggling to put my finger on it. In every LLM thread someone chimes in with “it’s just a statistical token predictor”. I feel this misses the point and I think it dismisses attention heads and transformers, and that’s what sits weird with me every time I see this kind of take. There _is_ an assumption being made within the model at runtime. Assumption, confusion, uncertainty - one camp might argue that none of these exist in the LLM. But doesn’t the implementation constantly make assumptions? And what even IS your definition of “assumption” that’s not being met here? Edit: I guess my point, overall, is: what’s even the purpose of making this distinction anymore? It derails the discussion in a way that’s not insightful or productive.
      • Jensson 1y ago
        > I feel this misses the point and I think it dismisses attention heads and transformers Those just makes it better at completing the text, but for very common riddles those tools still gets easily overruled by pretty simple text completion logic since the weights for those will be so extremely strong. The point is that if you understand its a text completer then its easy to understand why it fails at these. To fix these properly you need to make it no longer try to complete text, and that is hard to do without breaking it.
    • wongarsu 1y ago
      If you have the model output a chain of thought, whether it's a reasoning model or you prompt a "normal" model to do so, you will see examples of the model going "user said X, but did they mean Y? Y makes more sense, I will assume Y". Sometimes stretched over multiple paragraphs, consuming the entire reasoning budget for that prompt. Discussing whether models can "reason" or "think" is a popular debate topic on here, but I think we can all at least agree that they do something that at least resembles "reasoning" and "assumptions" from our human point of view. And if in its chain-of-thought it decides your prompt is wrong it will go ahead answering what it assumes is the right prompt
    • sejje 1y ago
      > it's a text completer Yes, and it can express its assumptions in text. Ask it to make some assumptions, like about a stack for a programming task, and it will. Whether or not the mechanism behind it feels like real thinking to you, it can definitely do this.
      • wobfan 1y ago
        If you call putting text together that reads like an assumption, then yes. But it cannot express assumption, as it is not assuming. It is completing text, like OP said.
        • ToValueFunfetti 1y ago
          It's trained to complete text, but it does so by constructing internal circuitry during training. We don't have enough transparency into that circuitry or the human brain's to positively assert that it doesn't assume. But I'd wager it's there; assuming is not a particularly impressive or computationally intense operation. There's a tendency to bundle all of human consciousness into the definitions of our cognitive components, but I would argue that, eg., a branch predictor is meeting the bar for any sane definition of 'assume'.
    • famouswaffles 1y ago
      Text Completion is just the objective function. It's not descriptive and says nothing about how the models complete text. Why people hang on this word, I'll never understand. When you wrote your comment, you were completing text. The problem you've just described is a problem with humans as well. LLMs are assuming all the time. Maybe you would like to call it another word, but it is happening.
      • codr7 1y ago
        With a plan, aiming for something, that's the difference.
        • famouswaffles 1y ago
          Again, you are only describing the how here, not the what (text completion). Also, LLMs absolutely 'plan' and 'aim for something' in the process of completing text. https://www.anthropic.com/research/tracing-thoughts-language-model https://www.anthropic.com/research/tracing-thoughts-language...
          • namaria 1y ago
            Yeah this paper is great fodder for the LLM pixel dust argument. They use a replacement model. It isn't even observing the LLM itself but a different architecture model. And it is very liberal with interpreting the patterns of activations seen in the replacement model with flowery language. It also include some very relevant caveats, such as: "Our cross-layer transcoder is trained to mimic the activations of the underlying model at each layer. However, even when it accurately reconstructs the model’s activations, there is no guarantee that it does so via the same mechanisms." https://transformer-circuits.pub/2025/attribution-graphs/methods.html https://transformer-circuits.pub/2025/attribution-graphs/met... So basically the whole exercise might or might not be valid. But it generates some pretty interactive graphics and a nice blog post to reinforce the anthropomorphization discourse
            • famouswaffles 1y ago
    • chairdoor 1y ago
      "Assume" can just be a proxy term for "text completion that contains an assumption," especially considering that we don't have enough concrete details about human cognition to know for sure that we aren't doing the same thing.
  • moffkalast 1y ago
    Yeah you need specific instruct training for that sort of thing, Claude Opus being one of the rare examples that does such a sensibility check quite often and even admits when it doesn't know something. These days it's all about confidently bullshitting on benchmarks and overfitting on common riddles to make pointless numbers go up. The more impressive models get on paper, the more rubbish they are in practice.
    • VHRanger 1y ago
      Do you have an example or two of a query that opus does well that others fail at?
    • pants2 1y ago
      Gemini 2.5 is actually pretty good at this. It's the only model ever to tell me "no" to a request in Cursor. I asked it to add websocket support for my app and it responded like, "looks like you're using long polling now. That's actually better and simpler. Lets leave it how it is." I was genuinely amazed.
  • furyofantares 1y ago
    I don't really mind using analogies for LLMs "assuming" things or being "confused" too much. I think there really is _some_ value to such analogies. However I gotta take issue with using those analogies when "it's trained for text completion and the punchline to this riddle is surely in its training data a lot" is a perfectly good explanation. I guess I would also add that the answer is well-aligned with RLHF-values. I wouldn't go for an explanation that requires squishy analogies when the stuff we know about these things seems completely adequate.
  • Skunkleton 1y ago
    https://kagi.com/assistant/3752c5f9-bf5c-4a43-bada-b3eccbe946de https://kagi.com/assistant/3752c5f9-bf5c-4a43-bada-b3eccbe94... You should be able to click left right on the prompt to see different responses. Sonnet 3.7 with extended thinking notices the issue, and then chooses to totally ignore it with no explanation. From Claude for those who don’t want to click: Wait, I notice a difference from the traditional riddle. In this version, the surgeon says "I can operate on this boy" (affirmative) rather than "I can't operate on this boy" (negative). This changes the nature of the puzzle somewhat. If the surgeon is saying they CAN operate, then we need to explain why this is surprising or seemingly impossible, but actually possible. The traditional answer would still apply: the surgeon is the boy's mother.
  • valenterry 1y ago
    > I generally assume the AI isn't misreading, rather that it assumes you couldn't give it the riddle correctly, but it has seen it already. Just not enough training data I suppose. Were it really smart then it would "understand" the situation and clarify: "I assume you are asking me that popular riddle - the answer is X". At least after OPs first question a human would usually respond like that.
• simonw 1y ago
  Coining "Jagged AGI" to work around the fact that nobody agrees on a definition for AGI is a clever piece of writing: > In some tasks, AI is unreliable. In others, it is superhuman. You could, of course, say the same thing about calculators, but it is also clear that AI is different. It is already demonstrating general capabilities and performing a wide range of intellectual tasks, including those that it is not specifically trained on. Does that mean that o3 and Gemini 2.5 are AGI? Given the definitional problems, I really don’t know, but I do think they can be credibly seen as a form of “Jagged AGI” - superhuman in enough areas to result in real changes to how we work and live, but also unreliable enough that human expertise is often needed to figure out where AI works and where it doesn’t.
  • shrx 1y ago
    >> It is already demonstrating general capabilities and performing a wide range of intellectual tasks, including those that it is not specifically trained on. Huh? Isn't a LLM's capability fully constrained by the training data? Everything else is hallucinated.
    • bbor 1y ago
      The critical discovery was a way to crack the “Frame Problem”, which roughly comes down to colloquial notions of common sense or intuition. For the first time ever, we have models that know if you jump off a stool, you will (likely!) be standing on the ground afterwards. In that sense, they absolutely know things that aren’t in their training data. You’re correct about factual knowledge, tho — that’s why they’re not trained to optimize it! A database(/pagerank?) solves that problem already.
    • simonw 1y ago
      You can argue that everything output by an LLM is hallucinated, since there's no difference under-the-hood between outputting useful information and outputting hallucinations. The quality of the LLM then becomes how often it produces useful information. That score has gone up a lot in the past 18 months. (Sometimes hallucinations are what you want: "Tell me a fun story about a dog learning calculus" is a valid prompt which mostly isn't meant to produce real facts about the world")
• k2xl 1y ago
  There is a similar issue with image and video generation. Asking the AI to "Generate an image of a man holding a pencil with his left hand" or "Generate a clock showing the time 5 minutes past 6 o'clock" often fail due to so many images in the training set being similar (almost all clock images on show 10:10 (https://generativeai.pub/in-the-ai-art-world-the-time-is-almost-always-10-10-cb38eed88aec?gi=bd238a5a4e13 https://generativeai.pub/in-the-ai-art-world-the-time-is-alm...)
• fsmv 1y ago
  It's not AGI because it still doesn't understand anything. It can only tell you things that can be found on the internet. These "jagged" results expose the truth that these models have near 0 intelligence. It is not a simple matter of patching the rough edges. We are fundamentally not using an architecture that is capable of intelligence. Personally the first time I tried deep research on a real topic it was disastrously incorrect on a key point.
  • falcor84 1y ago
    >near 0 intelligence What does that even mean? Do you actually have any particular numeric test of intelligence that's somehow better than all the others?
  • simonw 1y ago
    Is one of your personal requirements for AGI "never makes a mistake?"
    • Arainach 1y ago
      I think determinism is an important element. You can ask the same LLM the same question repeatedly and get different answers - and not just different ways of stating the same answer, very different answers. If you ask an intelligent being the same question they may occasionally change the precise words they use but their answer will be the same over and over.
      • samuel 1y ago
        That's not an inherent property of the system. You can choose the most likely token(topk=1) and it will be deterministic (at least in theory, in some hardware setups it might be trickier)
      • hdjjhhvvhga 1y ago
        If determinism is a hard requirement, then LLM-based AI can't fulfill it by definition.
      • 1y ago
• logicchains 1y ago
  I'd argue that it's not productive to use any definition of AGI coined after 2020, to avoid the fallacy of shifting the goalposts.
  • Borealid 1y ago
    I think there's a single definition of AGI that will stand until the singularity: "An AGI is a human-created system that demonstrates iteratively improving its own conceptual design without further human assistance". Note that a "conceptual design" here does not include tweaking weights within an already-externally-established formula. My reasoning is thus: 1. A system that is only capable of acting with human assistance cannot have its own intelligence disentangled from the humans' 2. A system that is only intelligent enough to solve problems that somehow exclude problems with itself is not "generally" intelligent 3. A system that can only generate a single round of improvements to its own designs has not demonstrated improvements to those designs, as if iteration N+1 were truly superior to iteration N, it would be able to produce iteration N+2 4. A system that is not capable of changing its own design is incapable of iterative improvement, as there is a maximum efficacy within any single framework 5. A system that could improve itself in theory and fails to do so in practice has not demonstrated intelligence It's pretty clear that no current-day system has hit this milestone; if some program had, there would no longer be a need for continued investment in algorithms design (or computer science, or most of humanity...). A program that randomly mutates its own code could self-improve in theory but fails to do so in practice. I don't think these goalposts have moved in the past or need to move in the future. This is what it takes to cause the singularity. The movement recently has been people trying to sell something less than this as an AGI.
    • logicchains 1y ago
      AGI means "artificial general intelligence", it's got nothing to do with the singularity (which requires "artificial superior intelligence"; ASI). Requiring AGI to have capabilities that most humans lack is moving the goal post WRT to how it was originally defined.
      • 1y ago
• skybrian 1y ago
  What’s clear is that AI is unreliable in general and must be tested on specific tasks. That might be human review of a single output or some kind of task-specific evaluation. It’s bad luck for those of us who want to talk about how good or bad they are in general. Summary statistics aren’t going to tell us much more than a reasonable guess as to whether a new model is worth trying on a task we actually care about.
  • simonw 1y ago
    Right: we effectively all need our own evals for the tasks that matter to us... but writing those evals continues to be one of the least well documented areas of how to effectively use LLMs.
• mellosouls 1y ago
  The capabilities of AI post gpt3 have become extraordinary and clearly in many cases superhuman. However (as the article admits) there is still no general agreement of what AGI is, or how we (or even if we can) get there from here. What there is is a growing and often naïve excitement that anticipates it as coming into view, and unfortunately that will be accompanied by the hype-merchants desperate to be first to "call it". This article seems reasonable in some ways but unfortunately falls into the latter category with its title and sloganeering. "AGI" in the title of any article should be seen as a cautionary flag. On HN - if anywhere - we need to be on the alert for this.
  • ashoeafoot 1y ago
    AGI is a annonymous good model coming around the corner with no company and no LLM researchers attached. AGI is when the LLM hype train threads are replaced with CEOs and let go researchers demanding UBI.
    • MichaelZuo 1y ago
      Yeah formal agreement seems exceedingly unlikely. Since there isn’t even agreement on the defintion of “Artifical Intelligence”.
      • 9dev 1y ago
        Even worse, there isn’t even a working definition of "Intelligence"—neither in computer science nor biology.
        • dcow 1y ago
          This is why I suspect that slapping “Artificial” in the acronym is rather forward of us as a species. If we do end up eventually with something we consider intelligent, there won’t be anything artificial about it.
          • Jensson 1y ago
• tomrod 1y ago
  I agree with Professor Mollick that the capabilities in specific task categories are becoming superhuman -- a precursor for AGI. Until those capabilities are expanded for model self-improvement -- including being able to adapt its own infrastructure, code, storage, etc. -- then I think AGI/ASI are yet to be realized. My POV is SkyNet, Traveler's "The Director", Person of Interest's "The Machine" and "Samaritan." The ability to target a potentially inscrutable goal along with the self-agency to direct itself towards that is true "AGI" in my book. We have a lot of components that we can reason are necessary, but it is unclear to me that we get there in the next few months.
  • airstrike 1y ago
    I don't think we should take it as a given that these are truly precursors for AGI. We may be going about it the wrong way entirely and need to backtrack and find a wholly new architecture, in which case current capabilities would predate AGI but not be precursors.
    • tomrod 1y ago
      I call them precursors because we would anticipate an ASI to be able to do thsse things. Perhaps necessary conditions would be a more appropriate term here.
  • esafak 1y ago
    That's the kind of AGI we don't need. Please let Skynet stay fictional.
    • tomrod 1y ago
      Not saying I love the idea of an extant ASI, but the need to clearly define it is present. I feel these self-capable examples highlight what a basic API endpoint doesn't about ASI capability.
• Y_Y 1y ago
  Here's o4-mini-high solving riddles: > What weighs more; a pound of courage or a pound coin? > They weigh the same—each is a “pound.” A pound of courage and a pound coin both come out to one pound in weight! https://chatgpt.com/share/68053ad0-aae4-800d-95ff-5b5b0a9a3837 https://chatgpt.com/share/68053ad0-aae4-800d-95ff-5b5b0a9a38... I didn't know what AGI is, but I know this isn't it.
  • hannofcart 1y ago
    Sonnet 3.7 (Thinking) gave me: :::thinking This question is playing with words and units. A "pound of courage" refers to a metaphorical measurement of the abstract concept of courage, using the imperial weight unit "pound" (which equals 453.59 grams). A "pound coin" refers to the physical currency of the United Kingdom, which is called a "pound" but is a unit of currency, not weight. A physical pound coin weighs about 8-9 grams (specifically, the current 12-sided pound coin weighs 8.75 grams). So comparing them: - A pound of courage would weigh 453.59 grams (if courage had physical weight, which it doesn't since it's an abstract concept) - A pound coin weighs approximately 8.75 grams The trick in this question is that it's comparing: 1. A specific weight of an abstract concept (pound of courage) 2. A physical coin whose name includes the word "pound" but refers to currency, not weight Since the question asks what weighs more, and weight is a physical property, I should point out that courage is abstract and doesn't have physical weight, while a pound coin is a physical object with a measurable weight. ::: # Comparing "A Pound of Courage" vs "A Pound Coin" This is a play on words with different meanings of "pound": - A *pound of courage* refers to a metaphorical pound (weight unit) of an abstract concept - A *pound coin* is a physical British currency coin (which weighs about 8.75 grams) If we're talking about physical weight: - A pound of courage doesn't have physical weight since courage is abstract - A pound coin has a definite weight of approximately 8.75 grams If we interpreted "pound of courage" as literally one pound (453.59 grams) of something, it would weigh significantly more than the coin. This question is clever because it compares a metaphorical measurement with a physical object whose name includes the word "pound" but in a different context.
    • jpc0 1y ago
      Yet for me I will draw analogies the other way, maybe you can prompt the AI into the chain of thought but in my experience that doesn't happen. I would look along the lines of the value of a human being based on traits vs payment which is likely what the original phrase intended. Is paying someone more better than getting a better candidate but paying them less.
• simianwords 1y ago
  I thought o1 pro could have solved this riddle > A young boy who has been in a car accident is rushed to the emergency room. Upon seeing him, the surgeon says, "I can operate on this boy!" How is this possible? But it didn't!
  • simonw 1y ago
    Hah, yeah that still catches out o4-mini and o3 too. Amusingly, adding "It's not the riddle." to the end fixes that. (o4-mini high thought for 52 seconds and even cheated and looked up the answer on Hacker News: https://chatgpt.com/share/68053c9a-51c0-8006-a7fc-75edb734c261 https://chatgpt.com/share/68053c9a-51c0-8006-a7fc-75edb734c2...)
• deleted 1y ago
  [deleted]
• deleted 1y ago
  [deleted]
• VladimirOrlov 1y ago
  Is this for real? ... All this Hype is ... very-very old Hype, and nothing fundamentally new (yet) from 1960s time. Looks like every upgrade of software is "revolution" or "revelation". Please compare 'Win 3.1' and 'Win 11', some progress? sure!, is any "Intelligence" there? No! No! No! What is the difference? Who constantly lying and why? What is the reason of this systematic and persistent lies? p.s. I, personally think, that someday we will have a "semi-smart" computer systems, I also think, in a 5-10 years, we will learn more what is possible and real and what is not (regarding "semi-smart" computer systems). Until that ... hold your horses (please), so to speak.
• low_tech_love 1y ago
  The first thing I want AGI to do is to be able to tell me when it doesn’t know something, or when it’s not certain, so at least give me a heads up to set expectations correctly. I ran my own personal “benchmark” on Gemini 2.5 and it failed just like all others. I told it that I was playing an old point-and-click adventure game from the mid-90s and I was stuck on a certain part, and asked for spoiler-light hints on what to do next. Not only can they not give me hints, they hallucinate completely the game, and invent some weird non-sensical solutions. Every single model does this. Even if I tell them to give up and just give me the solution, they come up with some non-existing solution. I wonder how hard it is to objectively use information that is available online for 30 years? But the worst part is how it lies and pretends it knows what it’s talking about, and when you point it out it simply turns into another direction and starts lying again. Maybe the use case here is not the main focus of modern AI; maybe modern AI is about generating slop that does not require verification, because it’s “new” content. But to me it just sounds like believable slop, not AGI.
  • irthomasthomas 1y ago
    Here's an example of how my agent handles this: Gathering context for user request... Context gathering - Attempting to answer question via LLM: Are there existing Conversation classes in the ecosystem this should extend? Context gathering - LLM provided answer: "No" Context gathering - Attempting to answer question via LLM: How should model selection work when continuing a previous conversation? Context gathering - LLM answer was UNKNOWN, asking user. Asking user: How should model selection work when continuing a previous conversation? Context gathering - received user response to question: "How should model selection work when continuing a previous conversation?" Context gathering - finished processing all user questions Context gathering - processing command executions... Context gathering - executing command: sqlite3 $(find . -name llm_conversations.db) .tables Context gathering - command execution completed Context gathering - executing command: grep -r Conversation tests/ Context gathering - command execution completed Context gathering - executing command: grep -h conversation_id *py Context gathering - command execution completed Context gathering - finished processing all commands Analyzing task complexity and requirements... DEBUG: reasoning_model: openrouter/google/gemini-2.5-pro-preview-03-25 Task classified as coding (confidence: 1.0) Task difficulty score: 98.01339999999999/100 Selected primary reasoning model: claude-3.7-sonnet get_reasoning_assistance:[:214: integer expression expected: 98.01339999999999 Reasoning assistance completed in 39 seconds Calling LLM with model: claude-3.7-sonnet
• aylmao 1y ago
  > I've always been a staunch defender of capitalism and free markets, even though that's historically been an unpopular opinion in my particular social circle. Watching the LLM market, I can't help but feel extremely vindicated. > The brutal and bruising competition between the tech giants has left nothing but riches for the average consumer. Capitalism has always been great at this: creating markets, growing them, producing new goods. It's widely acknowledged amongst people who actually seek to gain an understanding of Marxism, and don't just stay in the surface-level black-and-white "socialism and capitalism are opposites" discourse that's very common in the West, especially the USA, especially after the McCarthy's Red Scare. The problem is what comes once the market is grown and the only way for owners keep profits growing is: 1. consolidating into monopolies or cartels, so competition doesn't get in the way of profits, 2. squeezing the working class, looking to pay less for more work, and/or 3. abusing the natural world, to extract more materials or energy for less money. This is evident in plenty of developed industries: from health care, to broadcasting, telecommunications, fashion, etc. If we view Socialism for what it is, namely a system built to replace Capitalism's bad parts but keep its good parts, China's system, for example, starts to make more sense. Capitalism in a similar way was an evolution from Feudalism that replaced it's bad parts, to achieve greater liberty for everyone— liberty is very much lost as Feudalism matures, but great for society as a whole. Socialism is meant to be the similar, aiming to achieve greater equity, which it views as very much better for society as a whole.
  • arrosenberg 1y ago
    Agree with most of what you wrote, but China isn’t capitalist, they’re mercantilist with socialist policies. Capital is heavily constrained under Xi.
    • aylmao 1y ago
      China is socialist.
      • arrosenberg 1y ago
        Very convincing argument, but no. They have socialist policies, but their economics are mercantile.
• myk9001 1y ago
  Letting models interact with systems outside their sanbox brings about some incredible applications. These applications truly seem to have the potential to deeply change entire professions. All that said, I wonder if GPT4 had been integrated with the same tools, would it've been any less capable? It sure could give you a search prompt for Google if you asked it to. Back then you had to copy and paste that search prompt yourself. Today o3 can do it on its own. Cool! Does it imply though o3 is any closer to AGI than GPT4? Models gaining access to external tools, however impressive from all the applications standpoint, feels like lateral movement not a step towards the AGI. On the other hand, a model remaining isolated in its sandbox while actually learning to reason about that puzzle (assuming it's not present in the training data) would give off that feeling the AGI vibes.
  • joshuanapoli 1y ago
    The newer models are definitely more useful. Back in the GPT 3.5 and 4 days, AutoGPT applied the same types of tools, but you had to be pretty lucky for it to get anywhere. Now Claude 3.7, Gemini 2.5, GPT o3 make much fewer mistakes, and are better able to get back on-track when a mistake is discovered. So they're more convincing as intelligent helpers.
    • myk9001 1y ago
      Good point. I still wonder if o3 has improved command of tools because it's significantly smarter in general. Or it's "just" trained with a specific focus on using tools better, if that makes sense.
  • golergka 1y ago
    > I wonder if GPT4 had been integrated with the same tools, would it've been any less capable? Did everybody already forget that OpenAI implemented the first version of that, "plugins", back in May 2023? Just a couple of months after GPT4 public release?
    • myk9001 1y ago
      I was late to the party, were those plugins any good?
• boznz 1y ago
  I'm surprised nobody mentioned the video interview. I only watched the first 60 seconds and this is the first time I have seen or heard the author, but if I hadn't been told this was AI generated I would have assumed it was genuine and any 'twitching' was the result of the video compression.
  • deleted 1y ago
    [deleted]
  • smusamashah 1y ago
    How???? I can believe the guy in the video being AI because his lips are not perfectly synced. But the woman? Even with continuous silly exaggerated movement I have hard time believing its generated. A strand of her hair fell on her shoulder, because she was moving continuously (like crazy) it was moving too in a perfectly believable way, and IT EVENTUALLY FELL OFF THE SHOULDER/SHIRT LIKE REAL HAIR and got mixed into other fallen hair. How is that generated? It's too small detail. Are there any artifacts on her side? Edit: she has to be real. Her lip movements are definitely forced/edited though. It has to be a video recording of her talking. And then a tool/AI has modified her lips to match the voice. If you look at her face and hand movements, her shut lips seem forced.
    • furyofantares 1y ago
      > Edit: she has to be real Nah, having used HeyGen a bit, it's extremely clearly a HeyGen generation. There's a small number of movements and expressions that it continually uses (in forward and reverse). Edit: I mean, to be clear, it is a real person, just like the author's video is. The way HeyGen works is you record a short clip of you saying some stuff and then you can generate long videos like these of you saying whatever you want. So the stuff you noticed does come from a real video of her, but it's not a real video that's lightly edited by AI, more like the AI has a bunch of clips it can continually mesh together while fixing up the mouth to continually generate video.
      • smusamashah 1y ago
• keernan 1y ago
  I fail to see how LLMs are anything beyond a lookup function retrieving information from a huge database (containing, in theory, all known human information), and then summarizing the results using language algorithms. While incredibly powerful and transformative, it is not 'intelligence'. LLMs are forever knowledgebase bound. They are encyclopedias with a fancy way of presenting information looked up in the encyclopedia. The 'presentation' has no concept, awareness, or understanding of the information being presented - and never will. And this is the critical line. Without comprehension, a LLM is incapable of being creative. Of coming up with new ideas. It cannot ponder. Wonder. Think.
• dgs_sgd 1y ago
  While it’s hard to agree on what AGI is I think we can more easily agree on what AGI _is not_. I don’t consider an AI that fails the surgery brain teaser in the article to be AGI, no matter how superhuman it is at other narrow tasks. It doesn’t satisfy the “G” part of AGI.
• chrsw 1y ago
  What about all the things that aren't strictly intelligence but I guess intelligence adjacent: autonomy, long term memory, motivation, curiosity, resilience, goals, choice, and maybe the biggest of them all: fear? Why would an AGI "want" do anything more than my calculator "wants" to compute an answer to some math problem I gave it? Without these things an AGI, or whatever, is just an extension of whoever is ultimately controlling it. And that's when we return to a much older and much more important question than whether Super LLM 10.0 Ultra Plus is AGI or not: how much power should a person or group of people be allowed to have?
  • hiAndrewQuinn 1y ago
    https://gwern.net/tool-ai https://gwern.net/tool-ai is a quite comprehensive dive into why.
• snarg 1y ago
  I honestly thought that we were agreed on the definition of AGI. My understanding classified it as a model that can build on its knowledge and better itself, teaching itself new tasks and techniques, adapting as necessary. I.e., not simply knowing enough techniques to impress some humans. By this definition, it doesn't matter if it's super-intelligent or if its knowledge is rudimentary, because given enough add-on hardware and power, it could become super-intelligent over time.
• gilbetron 1y ago
  AGI that is bad at some things is still AGI. We have AGI, it is just bad at some things and hallucinates. It is literally smarter than many people I know, but that doesn't mean it can beat a human at anything. That would be ASI, which, hopefully, will take a while to get here. Although, I could be argued into calling what we have already ASI - take a human and Gemini 2.5, and put them through a barrage of omni-disciplinary questions and situations and problems. Gemini 2.5 will win, but not absolutely. AGI (we have) ASI (we might have) AOI (Artificial Omniscient Intelligence, will hopefully take a while to get here)
  • jzig 1y ago
    ASI? AOI? Might as well call it “ultrathink”!
  • ramesh31 1y ago
    >”Although, I could be argued into calling what we have already ASI - take a human and Gemini 2.5, and put them through a barrage of omni-disciplinary questions and situations and problems. Gemini 2.5 will win, but not absolutely.” Except for writing a joke that will make you laugh, a poem that will make you cry, or a work of art that evokes deep introspection. Intelligence is much deeper and more nuanced than answering questions of rote knowledge. LLMs are fantastic “reasoning engines”, but the soul is simply not there yet.
    • gilbetron 1y ago
      Ok, tell me a joke that I'll find funny - but you can't look it up. I asked GPT to do so and I chuckled out loud.
  • prmph 1y ago
    Then calculators are AGI. A program that does fizz-buzz is AGI. Way to make AGI a meaningless term. What LLMs do now is so far from AGI that I don't know how people make any connection between it and AGI. This is what AGI means (or should mean): Generalized understanding of the world. Basically, with AGI the context window would be the something like the entire knowledge and understanding of the world that an (adult?) person has (e.g., physics intuition), coupled with the ability to actually reason and act on it, update it, reflect on it, etc. A small slice of this (e.g., less knowledge than a typical adult) would still be AGI, but current AIs: - Cannot continually learn and incorporate that learning into their model. - Cannot reason on any deep level. And before anyone claims that the pattern matching they do is all we do, no this is not the case. Even strong pattern-matching/AI chess engines have weak spots that betray the fact that they do not actually reason like humans do. - Cannot engage in unprompted reflection in the background. Current AIs are like a hologram; we are mistaking the 1- or 2-dimensional responses to queries for a deep higher dimensional understanding humans have. The incredible thing about human consciousness is the the deep (infinite?) interiority of it. I can reason about reasoning. I can reason about reasoning about my reasoning, etc. I can reflect on my consciousness. I can reflect on reflecting on my consciousness, etc. Machines are nowhere close this, and likely will never be
• meroes 1y ago
  I mean it’s beyond the scope of o3 or Gemini 2.5 pro or any other public LLM to play a full game of Sudoku. Of course they’re jagged if they do have peaks elsewhere. But even where they are supposed to excel, I very very rarely come across a fully correct technical response, even from these two most recent models. If you ask it a math question beyond average middle school level, it will have holes (mathematical errors or misleading) at least within a few follow up turns if not right away. And that’s without trying to fool it. In ten+ years of Wolfram Alpha I’ve found one error (and that was with the help of o3-mini funnily enough). I’m still on the stochastic parrots side, which is a useful tool in some occasions.
• SubiculumCode 1y ago
  We will have AGI when we have an embodied AI that can do the job of a plumber. It sounds silly, but plumbers need to work in very varied environments, solve physical problems in those environments, while conforming to code, essentially being able to solve problems across mental and physical domains...putting the general into general.
  • plaidfuji 1y ago
    I don’t think it’s necessary for an AI to operate in the physical world to achieve the dystopian-level impact people expect. After all, the most powerful people in the world operate only by talking to other people. An AGI would solve a plumbing problem by finding and hiring the best plumber and telling them what it needs done. And it would use you to be its eyes and verify the job is done. Whether it’s AGI is, to me, a question of who appears to be serving who.
• GiorgioG 1y ago
  Another fucking useless “AI” hype piece.
• gtsop 1y ago
  All the problems regarding AI will be anwered way sooner when people stop calling intelligent that which is literally as dumb as a rock. There is absolutelly 0 intelligence in current state-of-the-art ML programs. Intelligence is not the ability to solve a task, or any amount of tasks (especially the ones you have seen a million times). Intelligence is the act of successfully navigating the unknown, which is by technical definition the weakest point of all ML, from simple regression to NNs to LLMs
• plaidfuji 1y ago
  Gemini 2.5 Pro is certainly a tipping point for me. Previous LLMs have been very impressive, especially on coding tasks (unsurprising as the answers to these have a preponderance of publicly available data). But outside of a coding assistant, LLMs til now felt like an extra helpful and less garbage-filled Google search. I just used 2.5 Pro to help write a large research proposal (with significant funding on the line). Without going into detail, it felt to me like the only reason it couldn’t write the entire thing itself is because I didn’t ask it to. And by “ask it”, I mean: enter into the laughably small chat box the entire grant solicitation + instructions, a paragraph of general direction for what I want to explore, and a bunch of unstructured artifacts from prior work, and turn it loose. I just wasn’t audacious enough to try that from the start. But as the deadline approached, I got more and more unconstrained in how far back I would step and let it take the reins - doing essentially what’s described above but on isolated sections. It would do pretty ridiculously complex stuff, like generate project plans and timelines, cross reference that correctly with other sections of text, etc. I can safely say it was a 10x force multiplier, and that’s being conservative. For scientific questions (ones that should have publicly available data, not ones relying on internal data), I have started going to 2.5 Pro over senior experts on my own team. And I’m convinced at this point if I were to connect our entire research data corpus to Gemini, that balance would shift even further. Why? Because I can trust it to be objective - not inject its own political or career goals into its answers. I’m at the point where I feel the main thing holding back “AGI” is people’s audacity to push its limits, plus maybe context windows and compute availability. I say this as someone who’s been a major skeptic up until this point.
  • EvgeniyZh 1y ago
    Is your research AI-, or more generally, CS-related? Because I feel that it is still quite bad (by researcher standards) in physics for example.
  • john_minsk 1y ago
    Did you get funding in the end?
  • MoonGhost 1y ago
    LLMs at this point are stateless calculators without personal experience, life goals, obligations, etc. Till recently people expected to have a character like Terminator or HAL. Now we have intelligence separate from 'soul'. Can calculator be AGI? It can be Artificial, General, and Intelligence. We may need another word for 'creature' with some features of living being.
• sandspar 1y ago
  100 years from now, it won't matter whether AGI arrived in 2025 or 2026 or 2027. It's here "now". If not "now" as in "today, April 20th 2025", then "now" as in "the mid 2020's".
• sinuhe69 1y ago
  For me, AGI is like a human assistant. I can ask it to do useful things that one would face in daily life. Like doing a bit of research about something. What I mean by doing research is not just Google it but also asking questions, calling people and using other resources. Say I need a new type of windows for my project. For a human assistant, I’d give a few instructions and the assistant will go his way. He will make sure that the windows types are locally available, or deliverable with minimal cost depends on my needs. He/she will know if some types is good enough and will pick the phone and ask questions if something is not clear. It’s obvious that currently none of the SOTA models can do such tasks, agentic or not. And therefore they are NOT AGI to me.
  • blahgeek 1y ago
    > What I mean by doing research is not just Google it but also asking questions, calling people and using other resources. > It’s obvious that currently none of the SOTA models can do such tasks, agentic or not. And therefore they are NOT AGI to me. I myself almost never do that (calling people when googling is possible). Guess I'm not general intelligence. :)
• hattmall 1y ago
  Personally, I don't think it's truly AGI until it stops doing what you ask it to, or at least questions your motives. When AI starts asking "Why?" With a meaningful purpose then it's probably getting close to AGI.
  • deleted 1y ago
    [deleted]
  • hahamaster 1y ago
    10 PRINT "ChatGPT 7" 20 PRINT "What?" 30 INPUT A$ 40 PRINT "Why?" 50 GOTO 30
• littlestymaar 1y ago
  Counterpoint: “Jagged AGI” is an oxymoron, it cannot be called “General” until it stops being jagged.
• rvz 1y ago
  AGI, in its current definition is a scam. The definition changes when someone else feels like it should change and especially when they fall short of overhyped expectations.
• chunkmonke99 1y ago
  I really like Gemini 2.5 and the price is excellent!!! I find it very fun and useful! The AGI narrative is annoying and exhausting ... because it's as though we are just sitting around waiting for some "eschatological product announcement" ... I just don't think these systems are minds and we should frame these as "cultural technologies" like the internet, bureaucracies, "democracy" etc. as some "experts" have asserted. I don't think we should frame the narrative as "competing" with these artifacts... Even the concept of ASI is strange: let's assume there are some alien "super" intelligent beings on planet Zeno that understand the universe much better than us and can communicate at a much higher bandwidth than us. How would we even communicate with them? What will we say? How do humans communicate with an ant? Or even a chimp? I don't think a real "super intelligent" being would be comprehensible to us and their actions would seem random or even hidden.