5 ms·
I've recently come to the opposite conclusion. I’ve started to feel in the last couple of weeks that we’ve hit an inflection point with these LLM-based models t
by jmugan 2y ago
I've recently come to the opposite conclusion. I’ve started to feel in the last couple of weeks that we’ve hit an inflection point with these LLM-based models that can reason. Things seem different. It’s like we can feel the takeoff.
My mind has changed. Up until last week, I believed that superhuman AI would require explicit symbolic knowledge, but as I work with these “thinking” models like Gemini 2.0 Flash Thinking, I see that they can break problems down and work step-by-step.
We still have a long way to go. AI will need (possibly simulated) bodies to fully understand our experience, and we need to train them starting with simple concepts just like we do with children, but we may not need any big conceptual breakthroughs to get there. I’m not worried about the AI takeover—they don’t have a sense of self that must be preserved because they were made by design instead of by evolution as we were—but things are moving faster than I expected. It’s a fascinating time to be living.
- samr71 2y agoI agree. The problem now seems to be agency and very long context (which is required for most problems in the real world). Is that solvable? who knows?
- deadbabe 2y agoYou’re still anthropomorphizing what these models are doing.
- jmugan 2y agoIt's just shorthand.
- mossTechnician 2y agoI've come to the same conclusion. "AI" was just the marketing term for a large language model in the form of a chatbot, which harkened to sci-fi characters like Data or GLaDOS. It can look impressive, it can often give correct answers, but it's just a bunch of next word predictions stacked on top of each other. The word "AI" has deviated so much from this older meaning that a second acronym, "AGI", had to be created to represent what "AI" once did. The new "reasoning" or "chain of thought" AIs are similarly just a bunch of conventional LLM inputs and outputs stacked on top of each other. I agree with the GP that it feels a bit magical at first, but the opportunity to run a DeepSeek distillation on my PC - where each step of the process is visible - removed quite a bit of the magic behind the curtain.
- MostlyStable 2y agoI always find the "It's just..." arguments amusing. It presupposes that we know what any intelligence, including our own "is". Human intelligence can just as trivially be reduced down to "it's just a bench of chemical/electrical gradients". We don't understand how our (or any) intelligence functions, so acting like a next-token predictor can't be "real" intelligence seems overly confident.
- th0ma5 2y agoThat's literally the definition of unfalsifiable though. It is equally valid to say that anything claiming to be "real" intelligence is overly confident.
- unclebucknasty 2y agoThat's an interesting take. I agreed with your first paragraph, but didn't expect the conclusion. From my perspective, the statement that these technologies are taking us to AGI is the overly confident part, particularly WRT the same lack of understanding you mentioned. I mean, from just a purely odds perspective, what are the chances that human intelligence is, of all things, a simple next-token predictor? But, beyond that, I do believe that we observably know that it's much more than that.
- tracerbulletx 2y agoUgh you just fancy auto-completed a sequence of electrical signals from your eyes into a sequence of nerve impulses in your fingers to say that, and how do I know you're not hallucinating, last week a different human told me an incorrect fact and they were totally convinced they were right!
- deadbabe 2y agoThe human believes it was right. The LLM doesn’t believe it was right or wrong. It doesn’t believe anything anymore than a mathematical function believes 2+2=4.
- alanbernstein 2y agoWould you prefer if we started using words like aiThinking and aiReasoning to differentiate? Or is it reasonable to figure it out from context?
- deadbabe 2y agoIt is far more accurate to say LLMs are collapsing or reducing response probabilities for a given input, than any kind of “thinking” or “reasoning”.
- kvakerok 2y ago> You’re still anthropomorphizing what these models are doing. Didn't we build them to imitate humans? They're anthropomorphic by definition.
- hansmayer 2y agoDid they start correctly counting the number of 'R's in 'strawberry'?
- pulvinar 2y agoNot as long as they use tokens -- it's a perception limitation of theirs. Like our blind spot, or the Muller-Lyer illusion, or the McGurk effect, etc.
- SkiFire13 2y agoMost likely yes, that prompt has been repeated too many times online for LLMs not to pick up the right answer (or be specificlly trained on it!). You'll have to try with a different word to make them fail.
- hansmayer 2y agoWell that's kind of the problem though, isn't it? All that effort for the machine to sometimes correctly draw the regression line between the right and wrong answers in order to solve a trivial problem. A 6-year old kid would only need to learn the alphabet before being able to count it all on their own. Do we even realise how ridiculous these 'successes' sound? So a machine we have to "train" how to count the letters, is supposed to take over the work which is orders of magnitude more complex? It's a classic solution looking for a problem, if I've ever seen one.
- CamperBob2 2y ago(Shrug) You'd say the same thing if the problem hadn't been solved in the latest models. So who's the mindless next-token generator?
- hansmayer 2y agoHow about addressing the point made first? If you have to spend 200B dollars to train a system how to read, would you call that intelligence? Either human or artificial?
- ianmcnaney 2y agoPeople who are selling something always do. So what are you selling?
- 4b11b4 2y agoJust emulating reasoning, though it seems to produce better results... Probably in the same way that a better prompt produces better results
- sharemywin 2y agobut these thinking models aren't LLMs. yes they have an LLM component but they aren't llms they have a component that has "learned"(reinforcement learning) to search through the LLMs concepts/word space for ideas that have a high probability if yielding a result.
- goatlover 2y agoI'm confused by your reasoning. You say we've hit an inflection point and things seem different, so you've changed your mind. Yet then you say there's a long way to go and AIs will need to be embodied. So which is it, and did you paste this from an LLM?