4 ms·
Calling it "just search" is like calling a compiler "just string manipulation". Not false, but aggressively missing the point.
by johnisgood 9mo ago
Calling it "just search" is like calling a compiler "just string manipulation". Not false, but aggressively missing the point.
- emp17344 9mo agoNo, “just search” is correct. Boosters desperately want it to be something more, but it really is just a tool.
- johnisgood 9mo agoYes, it is a tool. No, it is not "just search". Is your CPU running arbitrary code "just search over transistor states"? Calling LLMs "just search" is the kind of reductive take that sounds clever while explaining nothing. By that logic, your brain is "just electrochemical gradients".
- RhythmFox 9mo agoI mean, actually not a bad metaphor, but it does depend on the software you are running as to how much of a 'search' you could say the CPU is doing among its transistor states. If you are running an LLM then the metaphor seems very apt indeed.
- jvanderbot 9mo agoWhat would you add? To me it's "search" like a missile does "flight". It's got a target and a closed loop guidance, and is mostly fire and forget (for search). At that, it excels. I think the closed loop+great summary is the key to all the magic.
- bitwize 9mo agoWhich is kind of funny because my standard quip is that AI research, beginning in the 1950s/1960s, and indeed much of late 20th century computer tech especially along the Boston/SV axis, was funded by the government so that "the missile could know where it is". The DoD wanted smarter ICBMs that could autonomously identify and steer toward enemy targets, and smarter defense networks that could discern a genuine missile strike from, say, 99 red balloons going by.
- soulofmischief 9mo agoIt's a prediction algorithm that walks a high-dimensional manifold, in that sense all application of knowledge it just "search", so yes, you're fundamentally correct but still fundamentally wrong since you think this foundational truth is the end and beginning of what LLMs do, and thus your mental model does not adequately describe what these tools are capable of.
- jvanderbot 9mo agoMe? My mental model? I gave an analogy for Claude not a explanation for LLMs. But you know what? I was mentally thinking of both deep think / research and Claude code, both of which are literally closed loop. I see this is slightly off topic b/c others are talking about the LLM only.
- soulofmischief 9mo agoSorry, I should have said "analogy" and not "mental model", that was presumptuous. Maybe I also should have replied to the GP comment instead. Anyway, since we're here, I personally think giving LLMs agency helps unlock this latent knowledge, as it provides the agent more mobility when walking the manifold. It has a better chance at avoiding or leaving local minima/maxima, among other things. So I don't know if agentic loops are entirely off-topic when discussing the latent power of LLMs.
- 8note 9mo agoi dont disagree, but i also dont think thats an exciting result. every proboem can be described as a search for the right SOP, followed by execution of that SOP. an LLM to do the search, and the agent to execute the instructions can do everything under the sun
- gubicle 9mo ago[dead]
- deleted 9mo ago[deleted]
- maxilevi 9mo agoI don't mean search in the reductionist way but rather that its much better at translating, finding and mapping concepts if everything is provided vs creating from scratch. If it could truly think it would be able to bootstrap creations from basic principles like we do, but it really can't. Doesn't mean its not a great powerful tool.
- ordinaryatom 9mo ago> If it could truly think it would be able to bootstrap creations from basic principles like we do, but it really can't. alphazero?
- maxilevi 9mo agoI just said LLMs
- ordinaryatom 9mo agoYou are right that LLM and alphazero are different models, but given that alphazero demonstrated having the ability to bootstrap creations, we can't easily rule out LLM also has this ability?
- emp17344 9mo agoThis doesn’t make sense. They are fundamentally different things, so an observation made about Alphazero does not help you learn anything about LLMs.
- ordinaryatom 9mo agoI am not sure, self-play with LLMs self generated synthetic data is becoming a trendy topic in LLMs research.