3 ms·

So you agree that the LLM has patterns And you agree that all those things are patterns But you are arguing that they’re not… doing the patterns explicitly en

So you agree that the LLM has patterns And you agree that all those things are patterns But you are arguing that they’re not… doing the patterns explicitly enough? Please. The bar you’re describing is already something we can half ass and something we will be able to do explicitly very soon. The LLM can write code. If we just let it delegate the execution of that code to something that isn’t an LLM you can generate your explicit ontologies very easily. imo this is so obviously near term it’s uninteresting to fuss over it not being here in the oresent

thomastjeffery 4y ago
I'm talking about problem domain and approach. "Language" is a subdomain of "written text", which is a subdomain of "all possible text permutations". LLMs model "written text". Approach is either explicit or implicit. Constructed or inferred. You may be familiar with parsing: that's the explicit approach. Parsers are made out of predetermined grammar patterns. They categorize text into a model (AST) that was predetermined by the grammar rules. This approach is essentially a function from "all possible permutations of text" to "a known language model". It also maps "written language" to "predictable machine instructions". Parsing works for "context-free" (code) grammars, because the patterns of grammar are already known. Parsing fails at "context-dependent" (natural language) grammars because the patterns of grammar are ambiguous. LLMs take the implicit approach: they start completely blind, and model every pattern they can find in the text. An LLM has no category for "language grammar pattern". It does not constrain itself to the domain of "language". This approach maps "the behavior of someone writing text" to "patterns". The difference in problem domain introduces ambiguity: LLMs can't categorize truth from lie. The difference in approach removes intentional behavior: LLMs don't translate text into predictable machine behavior. LLMs model the patterns of behavior from the training corpus text, then model some more from a prompt, then show the resulting pattern. What an LLM does should not be expected to ever match what a parser does: they are completely different approaches working in completely different domains.
- meh8881 4y ago
  And I’m yawning because getting an LDM to construct explicit ontologies and then evaluate text against them seems trivial given it can code. It seems doubtful that such a hurdle will remain unsolved for more than a year. Like I said, you could do a poor man’s version of it today by instructing it on how to use a natural language API to call services.

thomastjeffery 4y ago

I'm talking about problem domain and approach. "Language" is a subdomain of "written text", which is a subdomain of "all possible text permutations". LLMs model "written text". Approach is either explicit or implicit. Constructed or inferred. You may be familiar with parsing: that's the explicit approach. Parsers are made out of predetermined grammar patterns. They categorize text into a model (AST) that was predetermined by the grammar rules. This approach is essentially a function from "all possible permutations of text" to "a known language model". It also maps "written language" to "predictable machine instructions". Parsing works for "context-free" (code) grammars, because the patterns of grammar are already known. Parsing fails at "context-dependent" (natural language) grammars because the patterns of grammar are ambiguous. LLMs take the implicit approach: they start completely blind, and model every pattern they can find in the text. An LLM has no category for "language grammar pattern". It does not constrain itself to the domain of "language". This approach maps "the behavior of someone writing text" to "patterns". The difference in problem domain introduces ambiguity: LLMs can't categorize truth from lie. The difference in approach removes intentional behavior: LLMs don't translate text into predictable machine behavior. LLMs model the patterns of behavior from the training corpus text, then model some more from a prompt, then show the resulting pattern. What an LLM does should not be expected to ever match what a parser does: they are completely different approaches working in completely different domains.

meh8881 4y ago
And I’m yawning because getting an LDM to construct explicit ontologies and then evaluate text against them seems trivial given it can code. It seems doubtful that such a hurdle will remain unsolved for more than a year. Like I said, you could do a poor man’s version of it today by instructing it on how to use a natural language API to call services.