3 ms·

What does it take to "hook up" an LLM to a camera and get it to output something cogent? We have a set of human-selected discrete photographs and a set of human

by lsy 3y ago

What does it take to "hook up" an LLM to a camera and get it to output something cogent? We have a set of human-selected discrete photographs and a set of human-written text labels for those photographs, then a model that is trained to assign those text labels to other photographs. Then a new photo is put into the model, a set of labels is emitted, and those labels are processed by the LLM as text (or more accurately as token integers). The same principle holds for other sensors and their data. All of these inputs are either human-generated or based directly on the human-generated input. So the argument is merely extended to say that "all human-generated text and photographs are sufficient to fully describe the world" which again I think would be hard to argue for. The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated.

vidarh 3y ago
> What does it take to "hook up" an LLM to a camera and get it to output something cogent? The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time. But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actions, and open ended problems to do things like web searches allows for discovery in the same way that a lot of human discovery happens, by posing questions, engaging in discovery and feed their findings back into further training. The only thing stopping us from doing that at scale right now is that training costs are still high enough that people are being conservative and taking the well known paths of feeding in cleaned and well labelled data. But we've already seen a lot of efforts in using LLMs to generate new training data for LLMs, and letting them drive that process without human intervention is an obvious next step (which will run into a huge number of failures, but the successes will keep accelerating future advances). > The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated. For humans to be anything more than a "mechanical transformer from its input to its output" (assuming you're using "transformer" in the general sense here rather than in the sense of the specific LLM architecture) presumes not just a non-deterministic, non-materialistic world, but one that fundamentally breaks our understanding of both logic and physics. There's no evidence to suggest we are anything more than well trained, well-structured automatons. And on that basis, while there's every reason to acknowledge that there are additional big leaps to be made, this constant stream of assumptions that there's some inherent barrier between us and LLMs rather than just a set of refinements is pretty much a religious belief and nothing more.
- lsy 3y ago
  An LLM can't generate new training data for itself. Mathematically speaking, the output of the LLM is governed by the same statistics embedded in its parameters, and a large enough training set of LLM output should have zero impact on those parameters. The difference is that training on human output contains the implicit assumption that the human output is "correct" in some way. So if the LLM gets more human-generated training data, it can come closer to "correctness". An LLM training on its own output is merely reinforcing its weights. Maybe to put it more simply, there is no LLM in the world that can determine for itself whether a proposition or piece of training data is sound without a person saying so at some point, either implicitly or explicitly. Even efforts to give LLMs some non-human-mediated reward function are merely people attempting to encode our own principles. While people are no doubt physical, we have a biological imperative to survival and well-being that governs our sense-making faculties, and to which nobody has yet proposed an analogue for LLMs. Our "transformations" are not only governed by other people's "training data" but by our urge to be able to skillfully cope with a world in which we are biologically invested.

by lsy 3y ago

vidarh 3y ago

> What does it take to "hook up" an LLM to a camera and get it to output something cogent? The same as it takes to teach a human to recognise objects and output something cogent, except for machine learning models the effort does not need to be replicated from scratch each time. But you're entirely missing the point of the comment you replied to, which is that giving them feedback loops and ability to carry out actions, and open ended problems to do things like web searches allows for discovery in the same way that a lot of human discovery happens, by posing questions, engaging in discovery and feed their findings back into further training. The only thing stopping us from doing that at scale right now is that training costs are still high enough that people are being conservative and taking the well known paths of feeding in cleaned and well labelled data. But we've already seen a lot of efforts in using LLMs to generate new training data for LLMs, and letting them drive that process without human intervention is an obvious next step (which will run into a huge number of failures, but the successes will keep accelerating future advances). > The point of the article is that an LLM is simply a mechanical transformer from its input to its output, and the specifics of that transformation are fully determined by its training data, which is all human-generated. For humans to be anything more than a "mechanical transformer from its input to its output" (assuming you're using "transformer" in the general sense here rather than in the sense of the specific LLM architecture) presumes not just a non-deterministic, non-materialistic world, but one that fundamentally breaks our understanding of both logic and physics. There's no evidence to suggest we are anything more than well trained, well-structured automatons. And on that basis, while there's every reason to acknowledge that there are additional big leaps to be made, this constant stream of assumptions that there's some inherent barrier between us and LLMs rather than just a set of refinements is pretty much a religious belief and nothing more.

lsy 3y ago
An LLM can't generate new training data for itself. Mathematically speaking, the output of the LLM is governed by the same statistics embedded in its parameters, and a large enough training set of LLM output should have zero impact on those parameters. The difference is that training on human output contains the implicit assumption that the human output is "correct" in some way. So if the LLM gets more human-generated training data, it can come closer to "correctness". An LLM training on its own output is merely reinforcing its weights. Maybe to put it more simply, there is no LLM in the world that can determine for itself whether a proposition or piece of training data is sound without a person saying so at some point, either implicitly or explicitly. Even efforts to give LLMs some non-human-mediated reward function are merely people attempting to encode our own principles. While people are no doubt physical, we have a biological imperative to survival and well-being that governs our sense-making faculties, and to which nobody has yet proposed an analogue for LLMs. Our "transformations" are not only governed by other people's "training data" but by our urge to be able to skillfully cope with a world in which we are biologically invested.