4 ms·

The problem is that new knowledge comes not only from combination but discovery, and discovery fundamentally requires access to and interaction with the actual

by lsy 3y ago

The problem is that new knowledge comes not only from combination but discovery, and discovery fundamentally requires access to and interaction with the actual world. And as of yet we have not created a way that computers can directly access the world, all I/O is mediated, etc. The more pedantic argument would be that what an LLM can write about the world is fundamentally limited to what has thus far been captured in human writing, and any structural properties thereof. To your point, prediction may not be bound by what the data explicitly shows, but it is necessarily bound by the data and its implicit and explicit structure. But to argue that the LLM can make any discovery a person can, or interact fluidly with the world as it exists, would require one to believe that every property of the world is either explicitly captured in human-written text or in the structure of that text, which I think would be pretty difficult to argue.

Davidzheng 3y ago
I think discovery for llm is possible both through its knowledge of the world from training data and also from some fields which are theoretical enough that interaction is not needed for discover--e.g pure mathematics but also many applied science
osmarks 3y ago
"Directly access"? Computers can use cameras and robotics hardware and such. It is not as if humans are special because of using eyes and muscles instead. Also, text does say a lot about the world. It includes descriptions of physics experiments and people's best knowledge of the laws, for instance.
vidarh 3y ago
People are hooking up LLM's with "access to the world" all over the place. Some of that will increasingly result in output that gets fed into training set. I'd be shocked if there aren't people running fairly large-scale research on letting LLMs using various patterns like ReAct to augment synthetic training data as we discuss this, because that'd be near the top of my list of things to do if I had the resources to turn around and train a large-scale model on those outputs, because testing ways of accelerating the training by letting the models build augmented training datasets themselves is such an obvious step.
- lsy 3y ago
  What does it take to "hook up" an LLM to a camera and get it to output something cogent? We have a set of human-selected discrete photographs and a set of human-written text labels for those photographs, then a model that is trained to assign those text labels to other photographs. Then a new photo is put into the model, a set of labels is emitted, and those labels are processed by the LLM as text (or more accurately as token integers). The same principle holds for other sensors and their data. All of these inputs are either human-generated or based directly on the human-generated input. So the argument is merely extended to say that "all human-generated text and photographs are sufficient to fully describe the world" which again I think would be hard to argue for. The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated.
  - vidarh 3y ago
    > What does it take to "hook up" an LLM to a camera and get it to output something cogent? The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time. But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actions, and open ended problems to do things like web searches allows for discovery in the same way that a lot of human discovery happens, by posing questions, engaging in discovery and feed their findings back into further training. The only thing stopping us from doing that at scale right now is that training costs are still high enough that people are being conservative and taking the well known paths of feeding in cleaned and well labelled data. But we've already seen a lot of efforts in using LLMs to generate new training data for LLMs, and letting them drive that process without human intervention is an obvious next step (which will run into a huge number of failures, but the successes will keep accelerating future advances). > The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated. For humans to be anything more than a "mechanical transformer from its input to its output" (assuming you're using "transformer" in the general sense here rather than in the sense of the specific LLM architecture) presumes not just a non-deterministic, non-materialistic world, but one that fundamentally breaks our understanding of both logic and physics. There's no evidence to suggest we are anything more than well trained, well-structured automatons. And on that basis, while there's every reason to acknowledge that there are additional big leaps to be made, this constant stream of assumptions that there's some inherent barrier between us and LLMs rather than just a set of refinements is pretty much a religious belief and nothing more.

by lsy 3y ago

Davidzheng 3y ago

I think discovery for llm is possible both through its knowledge of the world from training data and also from some fields which are theoretical enough that interaction is not needed for discover--e.g pure mathematics but also many applied science

osmarks 3y ago

"Directly access"? Computers can use cameras and robotics hardware and such. It is not as if humans are special because of using eyes and muscles instead. Also, text does say a lot about the world. It includes descriptions of physics experiments and people's best knowledge of the laws, for instance.

vidarh 3y ago

People are hooking up LLM's with "access to the world" all over the place. Some of that will increasingly result in output that gets fed into training set. I'd be shocked if there aren't people running fairly large-scale research on letting LLMs using various patterns like ReAct to augment synthetic training data as we discuss this, because that'd be near the top of my list of things to do if I had the resources to turn around and train a large-scale model on those outputs, because testing ways of accelerating the training by letting the models build augmented training datasets themselves is such an obvious step.

lsy 3y ago
What does it take to "hook up" an LLM to a camera and get it to output something cogent? We have a set of human-selected discrete photographs and a set of human-written text labels for those photographs, then a model that is trained to assign those text labels to other photographs. Then a new photo is put into the model, a set of labels is emitted, and those labels are processed by the LLM as text (or more accurately as token integers). The same principle holds for other sensors and their data. All of these inputs are either human-generated or based directly on the human-generated input. So the argument is merely extended to say that "all human-generated text and photographs are sufficient to fully describe the world" which again I think would be hard to argue for. The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated.
- vidarh 3y ago
  > What does it take to "hook up" an LLM to a camera and get it to output something cogent? The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time. But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actions, and open ended problems to do things like web searches allows for discovery in the same way that a lot of human discovery happens, by posing questions, engaging in discovery and feed their findings back into further training. The only thing stopping us from doing that at scale right now is that training costs are still high enough that people are being conservative and taking the well known paths of feeding in cleaned and well labelled data. But we've already seen a lot of efforts in using LLMs to generate new training data for LLMs, and letting them drive that process without human intervention is an obvious next step (which will run into a huge number of failures, but the successes will keep accelerating future advances). > The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated. For humans to be anything more than a "mechanical transformer from its input to its output" (assuming you're using "transformer" in the general sense here rather than in the sense of the specific LLM architecture) presumes not just a non-deterministic, non-materialistic world, but one that fundamentally breaks our understanding of both logic and physics. There's no evidence to suggest we are anything more than well trained, well-structured automatons. And on that basis, while there's every reason to acknowledge that there are additional big leaps to be made, this constant stream of assumptions that there's some inherent barrier between us and LLMs rather than just a set of refinements is pretty much a religious belief and nothing more.