3 ms·
LLMs exhibit very similar behavior to a search algorithm. Text query in -> relevant text out. I don’t say that search algorithms “learn” or “think” outside of
by dartos 2y ago
LLMs exhibit very similar behavior to a search algorithm.
Text query in -> relevant text out.
I don’t say that search algorithms “learn” or “think” outside of ML.
- famouswaffles 2y ago>Text query in -> relevant text out. Wait that's it ? I guess humans also exhibit similar behavior to a search algorithm in certain instances. Nothing about LLM inference seems particularly similar to search even with our limited understanding. All you're saying here is Input goes in > Output comes out. Well no shit.
- dartos 2y agoI think LLM inference is extremely similar to search. One of the killer features of LLMs is summarization (which can be thought of as searching a noisy set of data for the most relevant information) and document QA, which is also a search function. Even implementation wise, transformers encode information in their weights and include that relevant information in their response. Image generation models work the same way as google image search. Key word soup in -> relevant image out. They encode information seen in their training within their weights then filter down those weight over many layers and what’s leftover is relevant data. Idk how you’d look at how these models work and what they do and not see search.
- layer8 2y agoThe algorithm that learns is the training algorithm whose output is the LLM, not the inference algorithm employed when using the finished model.
- dartos 2y agoSure, but the whole process is referred to as “AI” Nobody says the algo that chatgpt was made with learned information which it encoded into weights, even if that’s technically correct.