6 ms·

Almost every predicate in natural languages is an arbitrary association of properties in the world. There isn't any property a person has "bald", nor are there

Almost every predicate in natural languages is an arbitrary association of properties in the world. There isn't any property a person has "bald", nor are there "tree"s in the world. What bundles of properties the vast majority of language names are a product of historical and contingent associations we've made for practical reasons. Likewise, languages do not have the same distributional structure. There is no reason an alien language would use discrete tokens to name properties, nor co-locate tokens by linear position in a 'sentence'. Historical human languages did not; using, eg., the full 2D structure of the clay tablet. To suppose that it is the glyphs and their colocation which somehow bare meaning is a nonesensical superstition. The world is what our words mean, and it is we, in that world, who provide them their meaning. We can do so with arbitrary linguistic structures.

FeepingCreature 2y ago
Right, but neural networks at a certain level of scale begin to compress the perception described by the text rather than the text per se. The lexical features are abstracted away and the network begins to understand meaning directly. Of course, you'd still have differentiation by different perceptions and focus on different parts of reality, but I see no reason why that should not likewise abstract out at even higher levels. Ultimately, the whole point of language is that it's about reality, and there is only one reality no matter how many different cultures you feed in. The network isn't doing magic - it's just doing exactly the same thing that we're doing with language. The reason why language is useful is the same reason it's learnable.
- mjburgess 2y ago
  If you want to understand how statistical AI systems work, then you should understand the basics of formalism. They are curve-fitting algorithms that approximate the probability distribution of a hisotrical dataset. That's all they are. By analogising to any animal or human mind, you aren't describing anythign that acutally exists. A neutral network algorithm isnt neural and it isnt a network. It's a statistical curve-fitting algorith. One oughtnt study trees to understand a decision tree either. This language is entirely metaphorical. There are no neurones in an NN, there are just summation entires in a matrix. This matrix comprises weights, which define the orientation and scale of the line pieces which form the curve being fit to the data.
  - FeepingCreature 2y ago
    They're, in particular, multi-level curve-fitting algorithms. A formalism which, I'm sure you are aware, can express any bounded computation. In particular, depending on how you set up the tape, neural networks are comfortably Turing complete. We're not talking about 2-layer perceptrons anymore here. In my opinion, you should look less at the formalism and more at the empirically demonstrated performance.
    - mjburgess 2y ago
      I'm not aware that it can express any bounded computation. But such is irrelevant anyway: there is no function from the distributional structure of text tokens to reasoning. There is nothing to approximate, and this is a system trained just to approximate some presumed function: our mental capacities arent patterns in glyphs. Whatever property you might imagine a transformer architecture to have (and it is vastly fewer than the set needed for general computation), the problem here is that it's being applied to approximate the structure of historical text data which isnt being generated from such a function. Indeed, there is no function which generates text data, there's a very large number of independent generating processes that give rise to the distribution of text. The phrase "the war in ukraine" acquires a different semantics over 2010-2030 in a radically different way than, "I liked that film" does. The capacities which produce distributions of text tokens are highly varied, complex, invovle a vast array of our mental processes, and so on. There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. The structure of language is conventional, and rests on our familiarity with such conventions. Otherwise, let's end all science, everything to be known about the world derives from how "e" occurs alongside "lectron"
      - stevenhuang 2y ago
        Computability is of critical relevance. Our mental capacities can be patterns. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding > There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. There's nothing in evolutionary fitness that necessitates intelligence or reasoning ability either. Yet here we are. Sorry but if you don't see the connections then you need to do some reading on theory of mind, cognition, information theory, physics, philosophy. All of the fundamental basis are met to allow reasoning to emerge in LLMs. It is clear now where your confusion lies, and why you are led to believe so strongly that LLMs cannot reason: it is because you are an ML practitioner you overweight your expertise yet you don't know what you don't know, and have foundational gaps in your knowledge. You lack the context in these other fields. If you had them, your position should be closer to agnostic than this strong belief of yours that LLMs in their current form cannot reason.
canjobear 2y ago
Languages do in fact have similar distributional structure, so that it is possible to learn how to translate words without supervision: https://arxiv.org/abs/2203.04863 https://arxiv.org/abs/2203.04863 In principle languages are arbitrary, but in practice they’re describing the same world and the same concepts end up being useful.
- mjburgess 2y ago
  Those are on embedding vectors, not on words. They use embeddings created from translated sources, eg., wikipedia articles. Yes, if you construct an embedding vector on texts-A, and another on texts-B where (A, B) are translations of each other, then an "unsupervised" algorithm really will give you the dizzying heights of a little above coin-flip accuracy on highly engineered self-selected benchmarks. They evaluate by by looking at the most in-use words in each vocab.. so you take the most in-use words on translations of Wikipedia, whose frequency is decided by the need of translation.. and then you use that to evaluate. It is blindingly obvious that the structure and frequency of heirglphys on tombs, Chinese glyphs in poetry, and latin in medieval liturgical literature are not distributed by Reality.. written in this order by God so that the Langauge of Reality is what places "d" alongside "oor". We already know this to be the case. The assumption of its opposite is rank pseudoscience.
  - FeepingCreature 2y ago
    > It is blindingly obvious that the structure and frequency of heirglphys on tombs, Chinese glyphs in poetry, and latin in medieval liturgical literature are not distributed by Reality.. written in this order by God so that the Langauge of Reality is what places "d" alongside "oor". We already know this to be the case. The assumption of its opposite is rank pseudoscience. I mean, on the first level, of course they're entirely determined by reality in the sense that the human brain is a real, physical object. But also on a second level they're still entirely determined by reality because the shape of the human mind is also entirely determined by reality. What is the mind for except reflecting reality? What is language for except communicating it? Sure that reality is warped, filtered, reduced and biased, but the data is still in there. That's in large part why LLMs need such ludicrously large training runs. I don't think letter frequency is objectively determined, but we know for a fact (many studies!) that the features that large language models learn are far, far above the scale of letters. Even arguing about phrases isn't engaging with the current state of the art. We're not talking about Markov chains here.

FeepingCreature 2y ago

Right, but neural networks at a certain level of scale begin to compress the perception described by the text rather than the text per se. The lexical features are abstracted away and the network begins to understand meaning directly. Of course, you'd still have differentiation by different perceptions and focus on different parts of reality, but I see no reason why that should not likewise abstract out at even higher levels. Ultimately, the whole point of language is that it's about reality, and there is only one reality no matter how many different cultures you feed in. The network isn't doing magic - it's just doing exactly the same thing that we're doing with language. The reason why language is useful is the same reason it's learnable.

mjburgess 2y ago
If you want to understand how statistical AI systems work, then you should understand the basics of formalism. They are curve-fitting algorithms that approximate the probability distribution of a hisotrical dataset. That's all they are. By analogising to any animal or human mind, you aren't describing anythign that acutally exists. A neutral network algorithm isnt neural and it isnt a network. It's a statistical curve-fitting algorith. One oughtnt study trees to understand a decision tree either. This language is entirely metaphorical. There are no neurones in an NN, there are just summation entires in a matrix. This matrix comprises weights, which define the orientation and scale of the line pieces which form the curve being fit to the data.
- FeepingCreature 2y ago
  They're, in particular, multi-level curve-fitting algorithms. A formalism which, I'm sure you are aware, can express any bounded computation. In particular, depending on how you set up the tape, neural networks are comfortably Turing complete. We're not talking about 2-layer perceptrons anymore here. In my opinion, you should look less at the formalism and more at the empirically demonstrated performance.
  - mjburgess 2y ago
    I'm not aware that it can express any bounded computation. But such is irrelevant anyway: there is no function from the distributional structure of text tokens to reasoning. There is nothing to approximate, and this is a system trained just to approximate some presumed function: our mental capacities arent patterns in glyphs. Whatever property you might imagine a transformer architecture to have (and it is vastly fewer than the set needed for general computation), the problem here is that it's being applied to approximate the structure of historical text data which isnt being generated from such a function. Indeed, there is no function which generates text data, there's a very large number of independent generating processes that give rise to the distribution of text. The phrase "the war in ukraine" acquires a different semantics over 2010-2030 in a radically different way than, "I liked that film" does. The capacities which produce distributions of text tokens are highly varied, complex, invovle a vast array of our mental processes, and so on. There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. The structure of language is conventional, and rests on our familiarity with such conventions. Otherwise, let's end all science, everything to be known about the world derives from how "e" occurs alongside "lectron"
    - stevenhuang 2y ago
      Computability is of critical relevance. Our mental capacities can be patterns. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding > There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. There's nothing in evolutionary fitness that necessitates intelligence or reasoning ability either. Yet here we are. Sorry but if you don't see the connections then you need to do some reading on theory of mind, cognition, information theory, physics, philosophy. All of the fundamental basis are met to allow reasoning to emerge in LLMs. It is clear now where your confusion lies, and why you are led to believe so strongly that LLMs cannot reason: it is because you are an ML practitioner you overweight your expertise yet you don't know what you don't know, and have foundational gaps in your knowledge. You lack the context in these other fields. If you had them, your position should be closer to agnostic than this strong belief of yours that LLMs in their current form cannot reason.

canjobear 2y ago

Languages do in fact have similar distributional structure, so that it is possible to learn how to translate words without supervision: https://arxiv.org/abs/2203.04863 https://arxiv.org/abs/2203.04863 In principle languages are arbitrary, but in practice they’re describing the same world and the same concepts end up being useful.

mjburgess 2y ago
Those are on embedding vectors, not on words. They use embeddings created from translated sources, eg., wikipedia articles. Yes, if you construct an embedding vector on texts-A, and another on texts-B where (A, B) are translations of each other, then an "unsupervised" algorithm really will give you the dizzying heights of a little above coin-flip accuracy on highly engineered self-selected benchmarks. They evaluate by by looking at the most in-use words in each vocab.. so you take the most in-use words on translations of Wikipedia, whose frequency is decided by the need of translation.. and then you use that to evaluate. It is blindingly obvious that the structure and frequency of heirglphys on tombs, Chinese glyphs in poetry, and latin in medieval liturgical literature are not distributed by Reality.. written in this order by God so that the Langauge of Reality is what places "d" alongside "oor". We already know this to be the case. The assumption of its opposite is rank pseudoscience.
- FeepingCreature 2y ago
  > It is blindingly obvious that the structure and frequency of heirglphys on tombs, Chinese glyphs in poetry, and latin in medieval liturgical literature are not distributed by Reality.. written in this order by God so that the Langauge of Reality is what places "d" alongside "oor". We already know this to be the case. The assumption of its opposite is rank pseudoscience. I mean, on the first level, of course they're entirely determined by reality in the sense that the human brain is a real, physical object. But also on a second level they're still entirely determined by reality because the shape of the human mind is also entirely determined by reality. What is the mind for except reflecting reality? What is language for except communicating it? Sure that reality is warped, filtered, reduced and biased, but the data is still in there. That's in large part why LLMs need such ludicrously large training runs. I don't think letter frequency is objectively determined, but we know for a fact (many studies!) that the features that large language models learn are far, far above the scale of letters. Even arguing about phrases isn't engaging with the current state of the art. We're not talking about Markov chains here.