3 ms·

They're, in particular, multi-level curve-fitting algorithms. A formalism which, I'm sure you are aware, can express any bounded computation. In particular, dep

They're, in particular, multi-level curve-fitting algorithms. A formalism which, I'm sure you are aware, can express any bounded computation. In particular, depending on how you set up the tape, neural networks are comfortably Turing complete. We're not talking about 2-layer perceptrons anymore here. In my opinion, you should look less at the formalism and more at the empirically demonstrated performance.

mjburgess 2y ago
I'm not aware that it can express any bounded computation. But such is irrelevant anyway: there is no function from the distributional structure of text tokens to reasoning. There is nothing to approximate, and this is a system trained just to approximate some presumed function: our mental capacities arent patterns in glyphs. Whatever property you might imagine a transformer architecture to have (and it is vastly fewer than the set needed for general computation), the problem here is that it's being applied to approximate the structure of historical text data which isnt being generated from such a function. Indeed, there is no function which generates text data, there's a very large number of independent generating processes that give rise to the distribution of text. The phrase "the war in ukraine" acquires a different semantics over 2010-2030 in a radically different way than, "I liked that film" does. The capacities which produce distributions of text tokens are highly varied, complex, invovle a vast array of our mental processes, and so on. There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. The structure of language is conventional, and rests on our familiarity with such conventions. Otherwise, let's end all science, everything to be known about the world derives from how "e" occurs alongside "lectron"
- stevenhuang 2y ago
  Computability is of critical relevance. Our mental capacities can be patterns. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding > There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. There's nothing in evolutionary fitness that necessitates intelligence or reasoning ability either. Yet here we are. Sorry but if you don't see the connections then you need to do some reading on theory of mind, cognition, information theory, physics, philosophy. All of the fundamental basis are met to allow reasoning to emerge in LLMs. It is clear now where your confusion lies, and why you are led to believe so strongly that LLMs cannot reason: it is because you are an ML practitioner you overweight your expertise yet you don't know what you don't know, and have foundational gaps in your knowledge. You lack the context in these other fields. If you had them, your position should be closer to agnostic than this strong belief of yours that LLMs in their current form cannot reason.
  - mjburgess 2y ago
    lol, well when I've written my PhD on those areas we can return to whether I'm an expert on them or not. One does not form "connections" between them as in some wide-eye conspiracy theorist.. "predictive coding" has little to do with "prediction" in the ML sense. The latter concerned with making a quantitive estimate of some variable by summarising historical data. What we are doing when we revise a "mental model" is done by counter-factual simulation of possible future states. Statistical AI models conditional probability structures, and computes predictions as an expectation over weighted summarised historical data. This is not a means of performing counter-factual simulation. One trivial, sadly empirical, way to see this is to note that each marginal token generated is of constant time and energy use. Yet trivially, reasoning and a variety of other mental capacities should require abitarily different time to run. Eg., simulating a complex scenario is necessarily more intensive than a simple one, and so on. Yet I am quite annoyed that we need such dumb observations to make this point. It speaks of a profound ignorance of "theory of mind, cognition, information theory, physics, philosophy" and especially neurology and zoology which are your most significant missing terms. It is no real mystery what the structure of various mental capacities involves; nor any mystery what s(Ws(Ws(WX+B)+B...)...)...) computes. Even involving anything beyond trivial applied statistics and trivial results in science shouldnt be required here. This stuff is very obvious.
- FeepingCreature 2y ago
  > Indeed, there is no function which generates text data, there's a very large number of independent generating processes that give rise to the distribution of text. The phrase "the war in ukraine" acquires a different semantics over 2010-2030 in a radically different way than, "I liked that film" does. But those semantics are revealed in the greater context of the phrase! That's why it's so important that transformers can attend to large context ranges; that's what lets them learn the greater semantic patterns to begin with. And at the limit, at a scale greater than phrases, I simply reject the idea that the same article - the same book - can have totally different meanings depending on context. Language isn't just shaped by context, it shapes context itself. Because language cannot be considered without context, it reveals information about that context, and in fact any compression of language ultimately requires modelling the person and even society that produced it. That's what the network learns. If your words don't have meaning beyond themselves, what are you even talking about?
  - mjburgess 2y ago
    > If your words don't have meaning beyond themselves, what are you even talking about? ...err... of course? That's the whole point. "Context" here isnt other words. It's the world. The meaning of words is the world. When I say, "I like what you're wearing" i'm not summarising a history of prior texts; it has nothnig to do with any statistical operation over historical documents. It has entirely to do with what you're wearing. Langauge use is a side-effect of being embedded in reality, directly attentive to it, and so on. Words are mere symptoms of how we are situated in the world. LLMs merely replay these back to us. They are not in the world. They cannot, in principle, ever mean, "I like what you're wearing"
    - FeepingCreature 2y ago
      I'd say exactly the opposite. It's because human speech is about the world, and LLM speech is about human speech, that LLM speech is about the world, by transitivity. "The ball fell to the <floor>", the LLM predicts, ultimately, not because of any feature of the human brain, or any feature of the English language, but because of gravity. That the causal arrow passes through a human brain does not make this any less true! Because human speech is inextricable from physical reality, language models likewise learn to model reality. They learn this poorly, incrementally, making missteps on the way - granted! But that they learn it at all shows that there's more going on than statistical modelling - or that statistical modelling of language is more than it sounds like.

mjburgess 2y ago

I'm not aware that it can express any bounded computation. But such is irrelevant anyway: there is no function from the distributional structure of text tokens to reasoning. There is nothing to approximate, and this is a system trained just to approximate some presumed function: our mental capacities arent patterns in glyphs. Whatever property you might imagine a transformer architecture to have (and it is vastly fewer than the set needed for general computation), the problem here is that it's being applied to approximate the structure of historical text data which isnt being generated from such a function. Indeed, there is no function which generates text data, there's a very large number of independent generating processes that give rise to the distribution of text. The phrase "the war in ukraine" acquires a different semantics over 2010-2030 in a radically different way than, "I liked that film" does. The capacities which produce distributions of text tokens are highly varied, complex, invovle a vast array of our mental processes, and so on. There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. The structure of language is conventional, and rests on our familiarity with such conventions. Otherwise, let's end all science, everything to be known about the world derives from how "e" occurs alongside "lectron"

stevenhuang 2y ago
Computability is of critical relevance. Our mental capacities can be patterns. https://en.m.wikipedia.org/wiki/Predictive_coding https://en.m.wikipedia.org/wiki/Predictive_coding > There's literally almost nothing in the distribution of text tokens that corresponds to any features of these processes. There's nothing in evolutionary fitness that necessitates intelligence or reasoning ability either. Yet here we are. Sorry but if you don't see the connections then you need to do some reading on theory of mind, cognition, information theory, physics, philosophy. All of the fundamental basis are met to allow reasoning to emerge in LLMs. It is clear now where your confusion lies, and why you are led to believe so strongly that LLMs cannot reason: it is because you are an ML practitioner you overweight your expertise yet you don't know what you don't know, and have foundational gaps in your knowledge. You lack the context in these other fields. If you had them, your position should be closer to agnostic than this strong belief of yours that LLMs in their current form cannot reason.
- mjburgess 2y ago
  lol, well when I've written my PhD on those areas we can return to whether I'm an expert on them or not. One does not form "connections" between them as in some wide-eye conspiracy theorist.. "predictive coding" has little to do with "prediction" in the ML sense. The latter concerned with making a quantitive estimate of some variable by summarising historical data. What we are doing when we revise a "mental model" is done by counter-factual simulation of possible future states. Statistical AI models conditional probability structures, and computes predictions as an expectation over weighted summarised historical data. This is not a means of performing counter-factual simulation. One trivial, sadly empirical, way to see this is to note that each marginal token generated is of constant time and energy use. Yet trivially, reasoning and a variety of other mental capacities should require abitarily different time to run. Eg., simulating a complex scenario is necessarily more intensive than a simple one, and so on. Yet I am quite annoyed that we need such dumb observations to make this point. It speaks of a profound ignorance of "theory of mind, cognition, information theory, physics, philosophy" and especially neurology and zoology which are your most significant missing terms. It is no real mystery what the structure of various mental capacities involves; nor any mystery what s(Ws(Ws(WX+B)+B...)...)...) computes. Even involving anything beyond trivial applied statistics and trivial results in science shouldnt be required here. This stuff is very obvious.
FeepingCreature 2y ago
> Indeed, there is no function which generates text data, there's a very large number of independent generating processes that give rise to the distribution of text. The phrase "the war in ukraine" acquires a different semantics over 2010-2030 in a radically different way than, "I liked that film" does. But those semantics are revealed in the greater context of the phrase! That's why it's so important that transformers can attend to large context ranges; that's what lets them learn the greater semantic patterns to begin with. And at the limit, at a scale greater than phrases, I simply reject the idea that the same article - the same book - can have totally different meanings depending on context. Language isn't just shaped by context, it shapes context itself. Because language cannot be considered without context, it reveals information about that context, and in fact any compression of language ultimately requires modelling the person and even society that produced it. That's what the network learns. If your words don't have meaning beyond themselves, what are you even talking about?
- mjburgess 2y ago
  > If your words don't have meaning beyond themselves, what are you even talking about? ...err... of course? That's the whole point. "Context" here isnt other words. It's the world. The meaning of words is the world. When I say, "I like what you're wearing" i'm not summarising a history of prior texts; it has nothnig to do with any statistical operation over historical documents. It has entirely to do with what you're wearing. Langauge use is a side-effect of being embedded in reality, directly attentive to it, and so on. Words are mere symptoms of how we are situated in the world. LLMs merely replay these back to us. They are not in the world. They cannot, in principle, ever mean, "I like what you're wearing"
  - FeepingCreature 2y ago
    I'd say exactly the opposite. It's because human speech is about the world, and LLM speech is about human speech, that LLM speech is about the world, by transitivity. "The ball fell to the <floor>", the LLM predicts, ultimately, not because of any feature of the human brain, or any feature of the English language, but because of gravity. That the causal arrow passes through a human brain does not make this any less true! Because human speech is inextricable from physical reality, language models likewise learn to model reality. They learn this poorly, incrementally, making missteps on the way - granted! But that they learn it at all shows that there's more going on than statistical modelling - or that statistical modelling of language is more than it sounds like.