3 ms·
Coining "Jagged AGI" to work around the fact that nobody agrees on a definition for AGI is a clever piece of writing: > In some tasks, AI is unreliable. In oth
by simonw 2y ago
Coining "Jagged AGI" to work around the fact that nobody agrees on a definition for AGI is a clever piece of writing:
> In some tasks, AI is unreliable. In others, it is superhuman. You could, of course, say the same thing about calculators, but it is also clear that AI is different. It is already demonstrating general capabilities and performing a wide range of intellectual tasks, including those that it is not specifically trained on. Does that mean that o3 and Gemini 2.5 are AGI? Given the definitional problems, I really don’t know, but I do think they can be credibly seen as a form of “Jagged AGI” - superhuman in enough areas to result in real changes to how we work and live, but also unreliable enough that human expertise is often needed to figure out where AI works and where it doesn’t.
- shrx 2y ago>> It is already demonstrating general capabilities and performing a wide range of intellectual tasks, including those that it is not specifically trained on. Huh? Isn't a LLM's capability fully constrained by the training data? Everything else is hallucinated.
- bbor 2y agoThe critical discovery was a way to crack the “Frame Problem”, which roughly comes down to colloquial notions of common sense or intuition. For the first time ever, we have models that know if you jump off a stool, you will (likely!) be standing on the ground afterwards. In that sense, they absolutely know things that aren’t in their training data. You’re correct about factual knowledge, tho — that’s why they’re not trained to optimize it! A database(/pagerank?) solves that problem already.
- simonw 2y agoYou can argue that everything output by an LLM is hallucinated, since there's no difference under-the-hood between outputting useful information and outputting hallucinations. The quality of the LLM then becomes how often it produces useful information. That score has gone up a lot in the past 18 months. (Sometimes hallucinations are what you want: "Tell me a fun story about a dog learning calculus" is a valid prompt which mostly isn't meant to produce real facts about the world")
- codr7 2y agoIsn't it the case that the latest models actually hallucinate more than the ones that came before? Despite best efforts to prevent it.
- simonw 2y agoThe o3 model card reports a so far unexplained uptick in hallucination rate from o1 - on page 4 of https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f3722c1/o3-and-o4-mini-system-card.pdf https://cdn.openai.com/pdf/2221c875-02dc-4789-800b-e7758f372... That is according to one specific internal OpenAI benchmark, I don't know if it's been replicated externally yet.
- verdverm 2y agoWhy not call it AJI instead of AGI then? Certainly jagged does not imply general It seems to me the bar for "AGI" has been lowered to measuring what tasks it can do rather than the traits we normally associate with general intelligence. People want it to be here so bad they nerf the requirements...
- bbor 2y agoWell I think the point being made is an instrumental one: it’s general enough to matter, so we should use the word “general” to communicate that to laypeople. Re:”traits we associate with general intelligence”, I think the exact issue is that there is no scientific (ie specific*consistent) list of such traits. This is why Turing wrote his famous 1950 paper and invoked the Imitation Game; not to detail how one could test for a computer that’s really thinking(/truly general), but to show why that question isn’t necessary in the first place.
- verdverm 2y agoI still disagree, being good at a number of tasks does not make it intellectual. Certainly creativity is missing, it has no internal motivation, and it will answer the same simple question both right and wrong, depending on unknown factors. What if we reverse the framing from "it can do these tasks, therefore it must be..." to "it lacks these traits, therefore it is not yet..." While I do not disagree that the LLMs have become advanced enough to do a bunch of automation, I do not agree they are intelligent or actually thinking. I'm with Yann Lecun when he says that we won't reach AGI until we move beyond transformers.
- parodysbird 2y agoAnd based on the actual Imitation Game in Turing's paper, we are no where close and I don't think we will be close for quite some time.
- iknowstuff 2y agoAJI lol love it.
- nearbuy 2y ago
- qsort 2y agoI don't think that's a particularly honest line of thinking though. It preempts the obvious counterargument, but very weakly so. Calculators are different, but why? Can an ensemble of a calculator, a Prolog interpreter, Alexnet and Stockfish be considered "jagged superintelligence"? They are all clearly superhuman, and yet require human experience to be wielded effectively. I'm guilty as charged of having looked at GPT 3.5 and having thought "it's meh", but more than anything this is showing that debating words rather than the underlying capabilities is an empty discussion.
- famouswaffles 2y ago>Calculators are different, but why? Can an ensemble of a calculator, a Prolog interpreter, Alexnet and Stockfish be considered "jagged superintelligence"? Those are all different things with little to nothing to do with each other. It's like saying what if I ensemble a snake and cat ? What does that even mean ? GPT-N or whatever is a single model that can do many things, no ensembling required. That's the difference between it and a calculator or stockfish.
- AstralStorm 2y agoThat is not true, the model is modular, thus an ensemble. Uses DallE for graphics and specialized tokenizer models for sound. If you remove those tools, or cut its access to search databases, it becomes quite less capable. A human would often still manage to do it without some data still, perhaps with less certainty, while GPT has more problems than that without others filling in the holes.
- famouswaffles 2y ago>Uses DallE for graphics and specialized tokenizer models for sound. chatgpt no longer uses dalle for image generation. I don't understand your point about the tokenization. It doesn't make the model become an ensemble. It's also just beside the point. Even if you restrict the modalities to text alone, these models are still general alone in ways a calculator is not.
- deleted 2y ago[deleted]