3 ms·

On the contrary, I think you are overestimating it; and I don't blame you for doing so. The overwhelming majority of narratives surrounding LLMs have shared th

On the contrary, I think you are overestimating it; and I don't blame you for doing so. The overwhelming majority of narratives surrounding LLMs have shared the same character flaw: personification. Almost no one writes explicitly about what an LLM itself does. An LLM models patterns from text. That's it. Not a single thing more. Personification is a great tool for explaining what you already know. The mistake is to draw conclusions about what you don't know from the personified character instead of the thing itself. Yet, that is exactly what is being done for LLMs. The first mistake is in the name: "Language Learning Model". That's the intended purpose, yes, but it's a misleading description for what the tool actually does. I propose we instead call them, "Text Learning Models". An LLM models all of the patterns in can find in the text it is given. It doesn't categorize those patterns, or have any notion of their importance to humans. An LLM doesn't do a single explicit thing. Everything an LLM does is implicit. An LLM models the patterns in the text, and exhibits them. The patterns in the text do all the exciting stuff. Everything interesting about LLMs is human generated. Without human authored text, the model is worthless. Without human authored prompts, the model is a black box. Every interesting thing an LLM does, one or more humans told it to do.

seydor 4y ago
That's very misleading. The current large LLMs learn tons and tons of "reference maps" that map how words (tokens) are referring (attending) to other tokens within a certain context. In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. It's not memorization, nor simple pattern matching, but something that allows them to be creative
- thomastjeffery 4y ago
  The LLM does not know what a word or a context are. It doesn't have that kind of categorization in its model or anywhere else. Tokens don't even match word boundaries: they are intentionally misaligned to better accommodate contractions. > In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. Exactly. Alongside those patterns, they model all of the other patterns that are present in the text. There is no categorization between these patterns: the word "syntax" has no bearing on the model whatsoever, except where it literally exists in the text being modeled. LLMs do not parse. They tokenize, but not to match any predefined grammar. It's not the LLM itself "being creative". It's the content of the text: the patterns that are modeled. Those patterns exist independent of the LLM. They come from the complex human behavior that is, "writing".
  - seydor 4y ago
    I dont actually see the point of using human-derived notions of 'language', 'syntax' , 'know', 'category' and even 'thinking' or 'parsing' when making precise arguments. Those are all vague, subjective notions that we made for ourselves but nature does not need to fit to our mold. The fact is, LLMs are the best quantitative model of the human communication code, so good that it can replicate it quite convincingly. Better than any model we had made before using our imprecise subjective notions, which were quite like alchemy. Whether it will prove to be 'thinking' or not, remains to be seen in the future. > the word "syntax" has no bearing on the model whatsoever, We don;t know that. One would have to look into the model weights for that and this analysis does not exist yet. If one can provide a decent definition of syntax, it may be found to reside somewhere in its weights. same for other subjective notions.
    - thomastjeffery 4y ago
      > I dont actually see the point of using human-derived notions of...'thinking'... > Whether it will prove to be 'thinking' or not, remains to be seen in the future. Maybe you should take your own advice. You are really missing my point. Humans use language, syntax, thinking, etc. We know what those patterns are. LLMs do not. LLMs don't approach text with the same models that humans use: they model the text directly. Any pattern, whether it be a word or sentence or even a pattern that humans have no name for at all: the LLM models that pattern. What we humans call "syntax", "word", "idea", etc.; LLMs don't. They don't have any names for any things. They just have the collection of things in front of them, and implicitly model that collection. An LLM doesn't think objectively about its model, either. It only constructs continuations by modeling prompts. It is the content of the prompt itself that determines the continuation, not "thought".
meh8881 4y ago
LLM stands for Large Language model. Not Language Learning Model. It absolutely categorizes those patterns as you can ask it for descriptions of things or request things via descriptions. It does identify the importance to humans. Because all of those things are also just patterns about patterns. Everything an LLM does may be implicit, but it takes a tiny effort to engineer that implicit capability in to something explicit.
- thomastjeffery 4y ago
  > LLM stands for Large Language model. Not Language Learning Model. I'm not sure how I missed that. Good to know. My point still stands: it is the text being modeled, not language. > It absolutely categorizes those patterns as you can ask it for descriptions of things or request things via descriptions When you "ask it for descriptions", you are presenting a text prompt (that contains your question) to be merged into the LLM's model. After merging, the LLM presents a continuation. The categories you are talking about are patterns of text. Those patterns are part of the model, and the LLM doesn't know them any more than it knows any other pattern. The categories I am talking about would be higher-level: a metadata structure of explicitly named patterns. That does not exist. LLMs don't have any notion of "what" a specific text pattern is, or why humans care. > It does identify the importance to humans. Again: not it. Humans make that identification. The identification itself, as written in text somewhere in the training corpus, is made available as a pattern to the LLM. The answer you get is a continuation from the content of the model, not some objective thought that the LLM is doing. > Everything an LLM does may be implicit, but it takes a tiny effort to engineer that implicit capability in to something explicit. Yes, and that engineering is a collection of human intentions. This changes nothing about the distinction I am making between an LLMs behavior, and the human behavior that was encoded into the text it is modeling. Most of the engineering is to construct the training corpus itself. Adding "weights" is effectively the same; except the model is being edited directly. None of these efforts change the fundamental structure and behavior that the LLM itself is made of.
  - meh8881 4y ago
    So you agree that the LLM has patterns And you agree that all those things are patterns But you are arguing that they’re not… doing the patterns explicitly enough? Please. The bar you’re describing is already something we can half ass and something we will be able to do explicitly very soon. The LLM can write code. If we just let it delegate the execution of that code to something that isn’t an LLM you can generate your explicit ontologies very easily. imo this is so obviously near term it’s uninteresting to fuss over it not being here in the oresent

seydor 4y ago

That's very misleading. The current large LLMs learn tons and tons of "reference maps" that map how words (tokens) are referring (attending) to other tokens within a certain context. In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. It's not memorization, nor simple pattern matching, but something that allows them to be creative

thomastjeffery 4y ago
The LLM does not know what a word or a context are. It doesn't have that kind of categorization in its model or anywhere else. Tokens don't even match word boundaries: they are intentionally misaligned to better accommodate contractions. > In the process they learn a map that may be the equivalent of what we humans call "syntax", and they apply it to completely different contexts and inputs. Exactly. Alongside those patterns, they model all of the other patterns that are present in the text. There is no categorization between these patterns: the word "syntax" has no bearing on the model whatsoever, except where it literally exists in the text being modeled. LLMs do not parse. They tokenize, but not to match any predefined grammar. It's not the LLM itself "being creative". It's the content of the text: the patterns that are modeled. Those patterns exist independent of the LLM. They come from the complex human behavior that is, "writing".
- seydor 4y ago
  I dont actually see the point of using human-derived notions of 'language', 'syntax' , 'know', 'category' and even 'thinking' or 'parsing' when making precise arguments. Those are all vague, subjective notions that we made for ourselves but nature does not need to fit to our mold. The fact is, LLMs are the best quantitative model of the human communication code, so good that it can replicate it quite convincingly. Better than any model we had made before using our imprecise subjective notions, which were quite like alchemy. Whether it will prove to be 'thinking' or not, remains to be seen in the future. > the word "syntax" has no bearing on the model whatsoever, We don;t know that. One would have to look into the model weights for that and this analysis does not exist yet. If one can provide a decent definition of syntax, it may be found to reside somewhere in its weights. same for other subjective notions.
  - thomastjeffery 4y ago
    > I dont actually see the point of using human-derived notions of...'thinking'... > Whether it will prove to be 'thinking' or not, remains to be seen in the future. Maybe you should take your own advice. You are really missing my point. Humans use language, syntax, thinking, etc. We know what those patterns are. LLMs do not. LLMs don't approach text with the same models that humans use: they model the text directly. Any pattern, whether it be a word or sentence or even a pattern that humans have no name for at all: the LLM models that pattern. What we humans call "syntax", "word", "idea", etc.; LLMs don't. They don't have any names for any things. They just have the collection of things in front of them, and implicitly model that collection. An LLM doesn't think objectively about its model, either. It only constructs continuations by modeling prompts. It is the content of the prompt itself that determines the continuation, not "thought".

meh8881 4y ago

LLM stands for Large Language model. Not Language Learning Model. It absolutely categorizes those patterns as you can ask it for descriptions of things or request things via descriptions. It does identify the importance to humans. Because all of those things are also just patterns about patterns. Everything an LLM does may be implicit, but it takes a tiny effort to engineer that implicit capability in to something explicit.

thomastjeffery 4y ago
> LLM stands for Large Language model. Not Language Learning Model. I'm not sure how I missed that. Good to know. My point still stands: it is the text being modeled, not language. > It absolutely categorizes those patterns as you can ask it for descriptions of things or request things via descriptions When you "ask it for descriptions", you are presenting a text prompt (that contains your question) to be merged into the LLM's model. After merging, the LLM presents a continuation. The categories you are talking about are patterns of text. Those patterns are part of the model, and the LLM doesn't know them any more than it knows any other pattern. The categories I am talking about would be higher-level: a metadata structure of explicitly named patterns. That does not exist. LLMs don't have any notion of "what" a specific text pattern is, or why humans care. > It does identify the importance to humans. Again: not it. Humans make that identification. The identification itself, as written in text somewhere in the training corpus, is made available as a pattern to the LLM. The answer you get is a continuation from the content of the model, not some objective thought that the LLM is doing. > Everything an LLM does may be implicit, but it takes a tiny effort to engineer that implicit capability in to something explicit. Yes, and that engineering is a collection of human intentions. This changes nothing about the distinction I am making between an LLMs behavior, and the human behavior that was encoded into the text it is modeling. Most of the engineering is to construct the training corpus itself. Adding "weights" is effectively the same; except the model is being edited directly. None of these efforts change the fundamental structure and behavior that the LLM itself is made of.
- meh8881 4y ago
  So you agree that the LLM has patterns And you agree that all those things are patterns But you are arguing that they’re not… doing the patterns explicitly enough? Please. The bar you’re describing is already something we can half ass and something we will be able to do explicitly very soon. The LLM can write code. If we just let it delegate the execution of that code to something that isn’t an LLM you can generate your explicit ontologies very easily. imo this is so obviously near term it’s uninteresting to fuss over it not being here in the oresent