4 ms·

> Any model that trivially depends upon statistics could not do causal reasoning, it would become exponentially less likely over time. At long output lengths, p

> Any model that trivially depends upon statistics could not do causal reasoning, it would become exponentially less likely over time. At long output lengths, practically impossible. This is handwaving. Yes, a system that is fundamentally based on statistics will require more and more data and compute power to be able to continue to function over longer and longer output lengths. But you don't know a priori what the shape of that curve is, or how far along it current LLMs are (maybe their creators have some idea, but I suspect not even they truly understand where on that curve the current systems are). Thus, there's no reason to assume that the system is "generaliz[ing] to the underlying problem" at all. And in fact, I'd argue that not only is there no reason to do so, there are strong reasons to assume that it is not doing that.

tysam_and 4y ago
Nope. It's an argument from Lyapunov exponents, translated into layman's terms. Edit: I noticed we could be reading this in different directions -- I'm reading the OP's post as treating LLMs as large Markov-chain-memorizing models, which is where the statistics argument comes into play. Heck, the curse of dimensionality alone makes the memorization aspect of it intractable, so there is a sort of compression, only what kind of compression. I agree that statistically it's going to approach what happens in the real world on the text as things get larger, but simple hallucinated chains of text in an autoregressive matter severely breaks the causal regime. I think that was where I was coming from originally, please do let me know if I've misunderstood, however. Second Edit: w.r.t. to the underlying manifold generalization, yes, this is a natural consequence of the operators used. directly measuring how much it's happening is intractable but by the nature of the system operating under them we are actually somewhere on that compressive spectrum of generalization.

tysam_and 4y ago

Nope. It's an argument from Lyapunov exponents, translated into layman's terms. Edit: I noticed we could be reading this in different directions -- I'm reading the OP's post as treating LLMs as large Markov-chain-memorizing models, which is where the statistics argument comes into play. Heck, the curse of dimensionality alone makes the memorization aspect of it intractable, so there is a sort of compression, only what kind of compression. I agree that statistically it's going to approach what happens in the real world on the text as things get larger, but simple hallucinated chains of text in an autoregressive matter severely breaks the causal regime. I think that was where I was coming from originally, please do let me know if I've misunderstood, however. Second Edit: w.r.t. to the underlying manifold generalization, yes, this is a natural consequence of the operators used. directly measuring how much it's happening is intractable but by the nature of the system operating under them we are actually somewhere on that compressive spectrum of generalization.