4 ms·

>Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think th

>Any tractable amount of data with the former [statistics] can’t approximate an ounce of the latter [causal/reasoning structure]. I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. In a predictive language model, at some point the best way to model a text corpus that indirectly references the "A then B" causal structure is to just model that structure and reference it as needed.

ClumsyPilot 6y ago
Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that. The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature in 2 hours you might have travelled across town, across continents, or to the moon. Statistical approach to such problems is dumb.
- hackinthebochs 6y ago
  But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plausible data. A human would behave similarly when forced to write a story while lacking critical information. >Statistical approach to such problems is dumb. Well, expecting your model to extract a spatial representation of the world from text is a dumb approach indeed. We interact with the spatial information much more directly. But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.
  - ClumsyPilot 6y ago
    The statistical argument has limitations, for instance, when there are more pieces of data to record than there are atoms in the universe. Then it falls firmly into impossible category. > But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input. i don't think this is true at all, many animals have dedicated 'hardware' for navigation that can sense magnetic fields, etc. We seem to be born with spatial awareness that is far beyond what GPT will ever be capable of.
ssivark 6y ago
> I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. Yes, but suppose A implied B only if C were true. And in the training corpus C were always true (hence learned A=>B) but in the test corpus suppose C is not true, then the learned statistical rule is wrong. The problem is that to cover all these possibilities will require not a billion documents but something like exp(billion) documents (exaggerating, but to give you a feel for combinatorial explosions) as @ClumsyPilot explains. And problem is the real world always keeps “changing” (Eg: whether C is valid or not) compared to the examples you trained on. This is referred to in polite company as “distribution shift” where your underlying statistical distribution keeps evolving.
- hackinthebochs 6y ago
  But this isn't an issue for statistically modelling causal relationships specifically, this is a core problem of modelling causal relationships at all. The fact that GPT-3 is sensitive to the real world changing, or to having insufficient information to form a universally accurate model says nothing interesting about GPT-3.

ClumsyPilot 6y ago

Because if you have a working concept of time, space, and modes of transport, you are aware that a a person has been driving for 2 hours, you can easily deduce the handful of possible towns they might arrive at. Indeed we have software that does that. The statistical model will die to combinatorial explosion between billions of possible combinations of locations, times, and modes of transport. In various literature in 2 hours you might have travelled across town, across continents, or to the moon. Statistical approach to such problems is dumb.

hackinthebochs 6y ago
But this isn't pointing to a fundamental limitation of statistical models, only a limitation of the text corpus. If you had a billion pages of text written about some town and the text included descriptions of travel distances and locations, the model should eventually develop a good representation of the town and relative locations. But of course without such a seed of spatial information, it will just make up plausible data. A human would behave similarly when forced to write a story while lacking critical information. >Statistical approach to such problems is dumb. Well, expecting your model to extract a spatial representation of the world from text is a dumb approach indeed. We interact with the spatial information much more directly. But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input.
- ClumsyPilot 6y ago
  The statistical argument has limitations, for instance, when there are more pieces of data to record than there are atoms in the universe. Then it falls firmly into impossible category. > But our ability to navigate is fundamentally just a process of capturing regularities in our sensory input. i don't think this is true at all, many animals have dedicated 'hardware' for navigation that can sense magnetic fields, etc. We seem to be born with spatial awareness that is far beyond what GPT will ever be capable of.

ssivark 6y ago

> I don't know why you think this is true. If statistically B follows A to a high degree, then a sufficiently advanced statistical model will represent "A then B" in some manner. Yes, but suppose A implied B only if C were true. And in the training corpus C were always true (hence learned A=>B) but in the test corpus suppose C is not true, then the learned statistical rule is wrong. The problem is that to cover all these possibilities will require not a billion documents but something like exp(billion) documents (exaggerating, but to give you a feel for combinatorial explosions) as @ClumsyPilot explains. And problem is the real world always keeps “changing” (Eg: whether C is valid or not) compared to the examples you trained on. This is referred to in polite company as “distribution shift” where your underlying statistical distribution keeps evolving.

hackinthebochs 6y ago
But this isn't an issue for statistically modelling causal relationships specifically, this is a core problem of modelling causal relationships at all. The fact that GPT-3 is sensitive to the real world changing, or to having insufficient information to form a universally accurate model says nothing interesting about GPT-3.