3 ms·

The entire vocabulary around machine learning is and always has been really weird. We don’t personify database interactions the same way we personify setting w

• layer8 2y ago
  We also personify synapses and axons in human brain tissue, though. My point is, while I agree with your first sentence to a degree, we shouldn’t judge the whole solely by its elementary parts. Clearly an LLM exhibits very different behavior from a conventional database.
  • dartos 2y ago
    LLMs exhibit very similar behavior to a search algorithm. Text query in -> relevant text out. I don’t say that search algorithms “learn” or “think” outside of ML.
    • famouswaffles 2y ago
      >Text query in -> relevant text out. Wait that's it ? I guess humans also exhibit similar behavior to a search algorithm in certain instances. Nothing about LLM inference seems particularly similar to search even with our limited understanding. All you're saying here is Input goes in > Output comes out. Well no shit.
      • dartos 2y ago
        I think LLM inference is extremely similar to search. One of the killer features of LLMs is summarization (which can be thought of as searching a noisy set of data for the most relevant information) and document QA, which is also a search function. Even implementation wise, transformers encode information in their weights and include that relevant information in their response. Image generation models work the same way as google image search. Key word soup in -> relevant image out. They encode information seen in their training within their weights then filter down those weight over many layers and what’s leftover is relevant data. Idk how you’d look at how these models work and what they do and not see search.
    • layer8 2y ago
      The algorithm that learns is the training algorithm whose output is the LLM, not the inference algorithm employed when using the finished model.
      • dartos 2y ago
        Sure, but the whole process is referred to as “AI” Nobody says the algo that chatgpt was made with learned information which it encoded into weights, even if that’s technically correct.
• ToucanLoucan 2y ago
  > The entire vocabulary around machine learning is and always has been really weird. I would argue it took a staggeringly weird turn around 2022/23. Machine learning has been around for a long time and only recently since OpenAI and it's slavish desire to harness true AI (which thanks to their horseshit now has to be called AGI) and Sam Altman in particular's delusional ramblings upon the topic that he clearly barely understands beyond it's ability to get his company fantastical amounts of capital has it truly gone off the rails. I cannot wait to watch this bubble pop.
  • dartos 2y ago
    I don’t think it was just post 2022. “Neural networks” “learned” data by being “trained” since they were first described in the late 1900s. The same language was used in Ian Goodfellow’s (excellent) 2012 text book “Deep Learning”
• danielmarkbruce 2y ago
  It has to be. Most people don't understand the basic math involved and hence you can't explain it in concrete terms (neither what it's doing or how it's doing it) so you have to sort of make up analogies. It's an impossible task.
  • dartos 2y ago
    Yeah, maybe it’s just unfortunate that “learn” and “think” are such fuzzy subjects.