5 ms·
How Does Claude 4 Think? – Sholto Douglas and Trenton Bricken
- fadsiooie3 1y ago[dead]
- cratermoon 1y agoIt doesn't. LLMs don't think.
- TheNewsIsHere 1y agoIt’s insane that this needs to continually be said.
- anonzzzies 1y agoIt is, but half of humanity cannot think either so that doesn't help evaluating AI.
- JKCalhoun 1y agoPut "think" in quotes then. I'm only 30 minutes into the interview and find it fascinating — learning some esoterica about how LLMs work. A lot of counter-intuitive stuff too about how larger models keep concepts more closely together, how we may be an order of magnitude or two away from the size of a human brain…
- simonw 1y agoI'll bite. Define "think". (And, to be fair, I try to always use the terms "reasoning" and "thinking" in scare-quotes when I'm writing about LLMs. But honestly I mostly do that to avoid tedious arguments about how "they're not actually thinking"!)
- cratermoon 1y agoI'd love for the AI hypesters to give their definition. They don't because any definition of "thinking" that would apply to their constructions would be laughed out of the room.
- arghwhat 1y agoI'd love to hear your definition of human thinking. No one else knows how that works, so it would most certainly be worth sharing, especially what the discriminator for whether something "thinks" would be. While avoiding being "laughed out of the room", of course.
- esafak 1y agoThe ability to extrapolate from facts. I don't think self awareness, emotions, or senses are required. You should give your definition too, so we can debate where computers fall short. See also: Levels of AGI for Operationalizing Progress on the Path to AGI (https://arxiv.org/abs/2311.02462 https://arxiv.org/abs/2311.02462)
- bigyabai 1y agoI'll steelman the other side, even as someone that detests AI and marketing hype. There isn't any harm in calling AI-generated text "thought" in the same way we aren't debasing humans by saying that crows or worms think too. It doesn't confer sophistication, sentience or even intelligence onto the thing that is thinking. It just means that some sort of process is taking an input and creating a distinct output as a product of consideration. There's no rule anywhere that says thought has to be intangible, or that it can't be represented as an algebraic formula. We can peer very easily into your train of thought here. You won't produce evidence to logically justify your stance, you can't take a position of authority on the subject, and basically rely purely on pathos to sell a fear of LLMs. Aristotle would call your rhetoric AI-generated if he lived to see the modern age.
- loudmax 1y agoAlong the same lines, it's often useful to think of an LLM's "understanding" a question or a context. Sometimes responses from smaller models show a shallower "understanding" of the question than you'd get from a bigger model. Obviously, we shouldn't anthropomorphize too much here, and even the most powerful LLMs don't "understand" or "reason" or "think" the way humans do. But whatever they're doing, it's at least analogous to what we do. These concepts are genuinely useful for making better use of these tools.
- simonw 1y agoYeah, that's my position as well. I hesitated on using the term "reasoning", but it's honestly a really good shortcut for describing that thing where the recent models can "think step by step" about a problem before providing their final answer.
- emp17344 1y agoNo one can provide a technical definition of what it means to think. That doesn’t mean LLMs are thinking. This style of argument, which often boils down to “you can’t solidly define the thought process, therefore LLMs can think!” is a fallacy. Why would we expect LLMs to be capable of thought in the first place? The fact that these models have devoured the contents of the entire internet and still aren’t AGI is an indication that LLMs won’t develop a capacity for cognition.
- nullstyle 1y ago> Why would we expect LLMs to be capable of thought in the first place? "We" didn't have the expectation, but upon interacting with LLMs, we can see and feel and hear that thought is taking place, and have realized that maybe the special place we held for human thought isn't so special as we had hoped. > The fact that these models have devoured the contents of the entire internet and still aren’t AGI is an indication that LLMs won’t develop a capacity for cognition. Why do you assume that training on the internet would lead to AGI?
- AstroBen 1y ago> upon interacting with LLMs, we can see and feel and hear that thought is taking place uh.. no we can't?
- nullstyle 1y agoThen you aren't paying attention, IMO. An example for the dense: A friend of mine used his conversations with ChatGPT to get through a tough spot in his marriage. The distinction between what happened in ChatGPT's internal processing and what would have happened in the internal processing of the brain of a family therapist is a meaningless distinction. My friend was able to benefit from the thoughtwork of ChatGPT.
- AstroBen 1y agoCalling people dense isn't helping them engage with you
- keybored 1y ago3000 BC to now: All vexing philosophical problems on things we do as easily as breathe but can’t seem to conceptualize rigourosly Now: If you can’t come up with a good definition on the spot I don’t see why my graph dump can’t do it as well
- whytaka 1y agoEven if one were to deny humans the will to "act", to think - as a verb, as some action being committed by a willing being or an automaton - by my definition, requires some weighing or surveying of possible answers. If I were to ask you a question and you were to blurt out an answer by reflex or by unguided stream of consciousness, I can accuse you of not thinking. The kind of thinking I'm referring to you here is one where you take pause to let go of prejudices and consider alternatives before answering. I'd say that LLMs are at best simulating reflexive streams of consciousness. Even with chain-of-thought, it never pauses to actually "think". But maybe even our own pauses are just internal chains of thought. Look at me be a reflexive stream of consciousness.
- sebzim4500 1y agoObviously he's using a different definition of think than you are. Your comment is as useless as me correcting an American for using a different spelling of 'colour' than I am used to.
- keybored 1y agoMetaphors taking on a life of their own makes programmers feel like they understand linguistics (my programming language is like a language) and cognitive science (look, the advanced chat bot is chatting like a thinking person). That’s not the same as using one wowel too much when writing words.
- monkaiju 1y agoThe hype machine just loves humanizing these little toys...
- icedchai 1y agoThey don't "think" like a human, but on the other hand, they don't "think" like a calculator, either.
- binary132 1y agoThis is one of those fancy new Veo productions, right?
- dcre 1y agoThese interviews are very, very good. Hearing these people talk casually in their own idiom about their work is the most efficient way I know of to get a feel for how things work at the cutting edge inside the big labs. That is useful whether you like what they’re doing or want to oppose it effectively. Interview with the same two people from a year ago: https://www.dwarkesh.com/p/sholto-douglas-trenton-bricken https://www.dwarkesh.com/p/sholto-douglas-trenton-bricken
- thundergolfer 1y agoThey’re not actually talking casually, as OpenAI and Anthropic run a tight ship around PR. If either of these two say something the PR team doesn’t like it gets edited out. Trenton and Sholto are very much “talking their book”. They’re doing it well, but it’s highly filtered and partial chat. source: know a podcast episode which got removed because an OpenAI employee used “black box” to refer to NNs.
- dcre 1y agoThat’s interesting, though it doesn’t undermine the point that it’s information-dense, especially if you are coming at it from outside the field.
- fumeux_fume 1y agoJust started listening to the Anthropic interview, and a few things jumped out at me. First, the claim that one of the main things holding LLMs back is a lack of expert feedback. To me, that just means the models are guessing—because they don’t have knowledge like humans do, they rely on pattern-matching, not understanding. If the user doesn’t know the answer, the LLM can’t help. That’s not just a minor limitation—it’s foundational. Framing it as a feedback issue is a way of sidestepping the deeper problem. Second, the speculation about Claude winning a Pulitzer or a Nobel Prize. I get the underlying point—they're wondering whether LLMs are better at creative or scientific work. But couching it in terms of prestigious awards just adds to the hype. Why not just say “creative vs. scientific tasks”? Framing it as “what will it win first?” cheapens what those prizes represent and makes the model seem far more capable than it actually is. Third, one of them claims a friend at a drug company says they’re about to release a drug discovered by AI. But when pressed for details, it turns out to be pure hearsay. It’s a textbook example of the kind of vague hype that surrounds LLMs—bold claims with no real substance when you dig. That said, I appreciate that the host, despite being quite friendly with the guests, actually pushes back and holds them to their claims. That's not very common in AI discussions, and I respect it.
- terhechte 1y agoJust at note that > If the user doesn’t know the answer, the LLM can’t help. That’s not just a minor limitation—it’s foundational. doesn't mean the LLM is not useful. My favorite use case for LLMs is them doing something I know 100% how to solve myself. However, they do it much faster and I can 100% understand their solution and give feedback in case it is wrong. No surprises. I much prefer this as it allows me to work on multiple tasks in parallel.
- martingalex2 1y agoThis seems like an extreme of example of "all models are wrong, but some are useful".
- pfortuny 1y agoRight, you’ve just described a calculator.
- Der_Einzige 1y agoTo think that Trenton invented a far better LLM sampler than top_p/top_k back in 2019 (tail free sampling - https://www.trentonbricken.com/Tail-Free-Sampling/ https://www.trentonbricken.com/Tail-Free-Sampling/) and it and pretty much all other better samplers are still not being used in production in 2025. Trenton please if you are listening give us an explanation for this!
- zackangelo 1y agoAre you familiar with min_p sampling? Kind of funny that it was introduced randomly on Reddit a couple of years ago instead of in a journal or something[0]. But I believe it's widely implemented and used now. [0] https://www.reddit.com/r/LocalLLaMA/comments/17vonjo/your_settings_are_probably_hurting_your_model_why/ https://www.reddit.com/r/LocalLLaMA/comments/17vonjo/your_se...
- Der_Einzige 1y agoI'm one of the authors on the paper about min_p sampling :)
- vessenes 1y agoAs always Dwarkesh gives the best interview in tech. The whole episode is worth a listen, but some of the highlights for me were: Learning that an earlier Opus was highly committed to animal welfare and they’re not sure why; other versions and models have not been Hearing that their mechanistic interpretability groups recently beat an internal red team quickly, using Claude’s help Speculation on de-risking AI growth for nation states that aren’t building frontier models Real real profound comment near the end noting that even if we get no more algorithmic improvements ever, current tech is sufficient to replace most white collar jobs, given enough data. That last one is a super profound and interesting point to me. We’re at the inflection point where white collar jobs will at the very least be implemented by this tech and overseen by humans, and the current tech and economics make it desirable to do so for companies that pay for a lot of white collar work. Feels like it’s time to buckle up. Not AI2027 buckle up, but still time to buckle up.
- mritchie712 1y ago> an earlier Opus was highly committed to animal welfare and they’re not sure why; other versions and models have not been I caught that too, very interesting. Many people treat their pets as part of their family. Many people make statements online like: "I like dogs / cats more than most people". So it's not surprising that this sentiment would be picked up in training data. I'd wager animals on average are thought of more highly on a site like Reddit than humans are. i.e. the average sentiment of a post on cats is more positive than a post about a human. Cats, in the training data, may be perceived as "more important" or "better" than humans. It is surprising Opus holds on to this tighter than other models.
- vessenes 1y agoThe MechInterp guys talk a lot about concept overloading. I wonder if “kind to animals” randomly overlaid with “helpful and not harmful” concepts during training and they got locked in together. All highly speculative obv.
- SubiculumCode 1y agoI recently asked an AI to write a scientific review for me via the deep research feature on a field of study with which I have some familiarity. While the review did not reveal new deep insights, the review was accurate (!!) and cross-cutting, well written, as requested, only sourced from pubmed, and the review was perfectly styled in the form I wanted...writing this would have taken me a couple of days. I don't know how I feel about it. Writing basic literature reviews is tedious, on the other hand, it is approaching the line where I can feel the temptation to let it work while I sip a cocktail. At what point will I be replaced by the open science literature we made?
- arecurrence 1y agoThis is one of the most interesting interviews I've ever read/listened to. Reminds me of when I first heard a Lex Fridman interview (the style is completely different but it hits on a lot of material that is interesting purely due to the openness of the interviewee to talk about whatever and how the interviewer drives the conversation). If you are at all interested in the current challenges being grappled on in this space, this does a great job of illuminating some of them. Many many interesting passages in here and the text transcript has links to relevant papers when their topics are brought up. Really like that aspect and would love to see that done a lot more often.
- crypto420 1y agoAll of this, as well as the crazy weird behaviors by o3 around it's hallucinations and Claude on deceiving users - is pointing to an interesting quote I saw about scaling RL in LLMs: https://x.com/jxmnop/status/1922078186864566491 https://x.com/jxmnop/status/1922078186864566491 "the AI labs spent a few years quietly scaling up supervised learning, where the best-case outcome was obvious: an excellent simulator of human text now they are scaling up reinforcement learning, which is something fundamentally different. and no one knows what happens next" I tend to believe this. AlphaGo and AlphaZero, which were both trained with RL at scale, led to strategies that have never been seen before. They were also highly specialized neural networks for a very specific task, which is quite different from LLMs, which are quite general in their capabilities. Scaling RL on LLMs could lead to models that have very unpredictable behaviors and properties on a variety of tasks. This is all going to sound rather hyperbolic - but I think we're living in quite unprecedented times, and I am starting to believe Kurzweil's vision of the Singularity. The next 10-20 years are going to be very unpredictable. I don't quite know what the answer will be, but I believe scaling mechanistic interpretability will probably yield some breakthroughs into how these models approach problems.