4 ms·

And it also suffers from the tired assumption that GPT-3 (or any language models) should, or are designed to in any way, give reasonable answers[1]. All GPT-3 d

And it also suffers from the tired assumption that GPT-3 (or any language models) should, or are designed to in any way, give reasonable answers[1]. All GPT-3 does is give likely continuations, given the training corpus. The prompts here are too short, and it could likely just be writing mediocre fiction continuations. Fiction tends to not be reasonable much of the time (to create story conflict). > "To understand why, it helps to think about what systems like GPT-3 do. They don’t learn about the world—they learn about text and how people use words in relation to other words. What it does is something like a massive act of cutting and pasting, stitching variations on text that it has seen, rather than digging deeply for the concepts that underlie those texts." This is another pet peeve of mine. It has long been shown experimentally[1] that neural networks such as image recognition and text prediction networks such as GPT-3 do understand deep concepts that underlie texts (not perfectly yet, of course), from emergent abstractions and similar cognitive tools employed by human brains. [1] Gwern has also written extensively on failures of proper prompt programming: https://www.gwern.net/GPT-3#prompts-as-programming https://www.gwern.net/GPT-3#prompts-as-programming [2] For example, using feature map and kernel visualization. In object classification or detection CNNs, specialized filters arise for detecting common observed object classes, like faces. Moreover, there is a hierarchical assembly of objects from elementary components (e.g. from lines, to limbs, to humans). Deep visualization toolbox: https://www.youtube.com/watch?v=AgkfIQ4IGaM https://www.youtube.com/watch?v=AgkfIQ4IGaM See this comment: https://news.ycombinator.com/item?id=24195009 https://news.ycombinator.com/item?id=24195009 for an extended discussion.

sjg007 6y ago
The have a layer that represents a face sure.. but that doesn’t mean it’s a deep understanding. It’s just an activation pattern.
- gnramires 6y ago
  It's an activation pattern, but it's not "just an activation pattern". The face activation relies on previous layers, the detection of each component. We can conjecture probably fair to say human brain object recognition (and other subconscious processes) use similar principles. All the components required to efficiently "understand", say a face, are there (not the shown visualization and architecture is for AlexNet, by now a very old and primitive model). I don't think we can ask for much more. What transformers do differently from CNNs is attention/recurrence. They have modifiable internal state, while feedforward models just have the feedforward state that can't temporally be reused (which is what we mean by 'algorithm'). This is a feature of logical thinking (and our own logical thinking), but I suspect most of what is meant by understanding the world is already contained in the internal structure captured by those models. Most of I understanding, as far as I can tell, comes from both this structural, intuitive inference (that CNNs and language models do), allied with our ability to think -- that is, talk to ourselves -- and thus build explanations and models on the fly, still reliant on the structural, intuitive understanding that comes from just very large networks generating abstract representations, classifications, etc.
skybrian 6y ago
The question is how you know you’ve found a prompt that will give reliable results, at least for questions in some domain? Although it’s possible such a prompt exists, it’s not clear that anyone has found one, and based on architecture, there is some reason to expect that apparently-promising prompts will prove unreliable if investigated seriously. It’s fine to do some exploratory experiments to find a good prompt, but this needs to be followed up with a separate, rigorous evaluation phase with a fixed prompt. If you keep changing the prompt then you will never know what you have and can easily be fooled by randomness.
deleted 6y ago
[deleted]

sjg007 6y ago

The have a layer that represents a face sure.. but that doesn’t mean it’s a deep understanding. It’s just an activation pattern.

gnramires 6y ago
It's an activation pattern, but it's not "just an activation pattern". The face activation relies on previous layers, the detection of each component. We can conjecture probably fair to say human brain object recognition (and other subconscious processes) use similar principles. All the components required to efficiently "understand", say a face, are there (not the shown visualization and architecture is for AlexNet, by now a very old and primitive model). I don't think we can ask for much more. What transformers do differently from CNNs is attention/recurrence. They have modifiable internal state, while feedforward models just have the feedforward state that can't temporally be reused (which is what we mean by 'algorithm'). This is a feature of logical thinking (and our own logical thinking), but I suspect most of what is meant by understanding the world is already contained in the internal structure captured by those models. Most of I understanding, as far as I can tell, comes from both this structural, intuitive inference (that CNNs and language models do), allied with our ability to think -- that is, talk to ourselves -- and thus build explanations and models on the fly, still reliant on the structural, intuitive understanding that comes from just very large networks generating abstract representations, classifications, etc.

skybrian 6y ago

The question is how you know you’ve found a prompt that will give reliable results, at least for questions in some domain? Although it’s possible such a prompt exists, it’s not clear that anyone has found one, and based on architecture, there is some reason to expect that apparently-promising prompts will prove unreliable if investigated seriously. It’s fine to do some exploratory experiments to find a good prompt, but this needs to be followed up with a separate, rigorous evaluation phase with a fixed prompt. If you keep changing the prompt then you will never know what you have and can easily be fooled by randomness.

deleted 6y ago

[deleted]