6 ms·

GPT-3 didn't even need the `gloober Blue` prompt as part of `Answer:`. It 0-shotted (no other training examples required) it straight out of the box. Prompt:

GPT-3 didn't even need the `gloober Blue` prompt as part of `Answer:`. It 0-shotted (no other training examples required) it straight out of the box. Prompt: Every turn each gloober flomps the same gloobers. If a yellow gloober flomps a blue gloober then the blue gloober turns yellow, otherwise, gloobers stay the same color. 1. gloober Roy flomps gloober Zoo and gloober Bat 2. gloober Zoo flomps gloober Bat and gloober Crystal 3. gloober Crystal flomps gloober X-Ray 4. gloober Bat flomps gloober Blue If gloober Zoo is the only gloober that is yellow, and all other gloobers are blue, will gloober Blue turn yellow, and, if so, at what turn? Answer: Continuation: gloober Blue will turn yellow on turn 4. > I find it a little unsettling that GPT-3 is able to do this well. I agree and think that this is definitely the right reaction.

mbil 4y ago
Reminds me of a paper that I saw the other day: "On the Paradox of Learning to Reason from Data"[0] [0]: http://starai.cs.ucla.edu/papers/ZhangArxiv22.pdf http://starai.cs.ucla.edu/papers/ZhangArxiv22.pdf
- foota 4y ago
  This is fantastic, thanks for sharing!
nikkwong 4y ago
I know that GPT-3 is not AGI nor modeled properly to ever represent what we would consider to be AGI; but this feels freakishly AGI-ish. This feels like the model is 'reasoning' in a way that I would not expect based on my intuition of how GPT-3 works. As it's been explained to me in lay terms; GPT is great at predicting the next word based on a chunk of previous context. This feels like it's doing much more than that as it's understanding the context and parameters that relate to a question embedded in the text, which seems unrelated to the original instruction of GPT-3 as I understand. Can anyone explain?
- joe-collins 4y ago
  I think the sentences in this example are, odd vocabulary notwithstanding, fairly typical descriptions of logical implication, which I expect has non-trivial representation in the training set. GPT-3 is probably capable of matching the pattern of the tokens (sentence structure), even if the content of the tokens (spelling/vocabulary) is novel.
  - uup 4y ago
    I doubt it. There are billions of parameters and weights all combining in non trivial ways to produce the output. I don’t think you can describe any of its workings with simple terms like “doing token substitution” even if that’s what it looks like to us when we look at the output.
    - joe-collins 4y ago
      Try it. Reduce the tokens to single characters but keep the structure. Does it work? Maybe single characters looks too much like an abbreviation and that throws it off, because those are found in sufficiently different contexts. Exchange to two characters, three. Throw similar inputs at it and find the point where it reliably "gets it".
- mjburgess 4y ago
  A NN is a means of compressing a training dataset into a mathematical representation (weights) which can be exploited for pattern finding. The representation of the training data is where 95% of the "magic" comes from. We intelligently prepare it, so no intelligence is required in its use. On any given input, its predictions can "look" how we expect an intelligent response to "look". The job is to disprove this intuition, which is usually trivial. This cannot be done by looking at a single response. Consider walking into a room and predicting a few people's personality traits correctly based on their star sign. This shows nothing.
  - YorkshireSeason 4y ago
    This sounds like you consider what has been called "feature engineering" in older ML research as being fundamental to modern ML successes. But the trajectory has been the opposite direction, away from feature engineering, and instead towards letting the transformer architecture learn as much as possible, including interesting features. Most research on deep net architecture, is about how to train it effectively on huge stacks of GPUs and TPUs. That's how/why transformers were invented.
ALittleLight 4y ago
I've been messing with the prompt a little bit and tried to get it to spell out what it thinks is happening. (I changed the name of gloober Blue to gloober Lovely just to make it easier to read). Continuation is at the asterisk. Every turn each gloober flomps the same gloobers. If a yellow gloober flomps a blue gloober then the blue gloober turns yellow, otherwise, gloobers stay the same color. 1. gloober Roy flomps gloober Zoo 2. gloober Roy flomps gloober Bat 3. gloober Zoo flomps gloober Bat 4. gloober Zoo flomps gloober Crystal 5. gloober Crystal flomps gloober X-Ray 6. gloober Bat flomps gloober Lovely gloober Zoo is the only yellow gloober. Write the turn and which gloobers are yellow. 1. Yellow gloobers - Zoo. 2. Yellow gloobers -* Zoo, Bat. 3. Yellow gloobers - Zoo, Bat, Crystal. 4. Yellow gloobers - Zoo, Bat, Crystal, X-Ray. 5. Yellow gloobers - Zoo, Bat, Crystal, X-Ray, Lovely. 6. Yellow gloobers - Zoo, Bat, Crystal, X-Ray, Lovely. Other than thinking the spread of yellow is limited to one gloober at a time, this seems right.
- deleted 4y ago
  [deleted]
- codetrotter 4y ago
  > Other than thinking the spread of yellow is limited to one gloober at a time, this seems right. Yup. Here are the results I get when manually applying these rules myself. Here I mention only the flomps that change gloobers to become yellow. Turn 1: Zoo is yellow. Turn 2: Zoo flomps Bat, turning Bat yellow. Zoo flomps Crystal, turning Crystal yellow. Turn 3: Crystal flomps X-Ray, turning X-Ray yellow. Bat flomps Lovely, turning Lovely yellow. In other words it would be: 1. Yellow gloobers - Zoo. 2. Yellow gloobers - Zoo, Bat, Crystal. 3. Yellow gloobers - Zoo, Bat, Crystal, X-Ray, Lovely. So it’s got the order correct, and like you said it’s just that it thinks only one gloober is flomped each turn.
hhhfvk 4y ago
Since GPT-3 is prone to guessing, maybe it's just guessing it's the last step. What happens if you try a misdirection and insert a 5th step?
tveita 4y ago
> It 0-shotted (no other training examples required) it straight out of the box. It 0-shotted it... but incorrectly, right? Gloober Zoo flomps gloober Bat on turn 1. Gloober Bat flomps gloober Blue on turn 2. Answering '4' seems like a total guess, maybe based on there being four steps, or 'Blue' being mentioned in step 4. The replies to your post seem to assume that the answer is correct, but it is actually demonstrating a lack of step-wise reasoning.
- londons_explore 4y ago
  I think you and OP have a different definition of [subject verb object]... My reading is that on line 1, no action happens, because Roy isn't yellow.
  - HL33tibCe7 4y ago
    > If gloober Zoo is the only gloober that is yellow
  - tveita 4y ago
    The order seems clear to me: "If a yellow gloober flomps a blue gloober then the blue gloober turns yellow," I still don't see how 4 could be a correct answer, could you post a plausible sequence of turns where Blue turns yellow on turn 4?
    - londons_explore 4y ago
      List of yellow gloobers after each turn. 1. Zoo 2. Zoo, Bat, Crystal 3. Zoo, Bat, Crystal, X-Ray 4. Zoo, Bat, Crystal, X-Ray, Blue
      - tveita 4y ago
        The text says "Every turn each gloober flomps the same gloobers." If I understand you correctly, you are applying only one of the steps each turn, in order, which contradicts the instructions by flomping different gloobers each turn. It is also guessable in multiple ways which is unfortunate. "There are four steps specified so I guess it happens on turn four." is one invalid way of getting the same answer.

mbil 4y ago

Reminds me of a paper that I saw the other day: "On the Paradox of Learning to Reason from Data"[0] [0]: http://starai.cs.ucla.edu/papers/ZhangArxiv22.pdf http://starai.cs.ucla.edu/papers/ZhangArxiv22.pdf

foota 4y ago
This is fantastic, thanks for sharing!

nikkwong 4y ago

I know that GPT-3 is not AGI nor modeled properly to ever represent what we would consider to be AGI; but this feels freakishly AGI-ish. This feels like the model is 'reasoning' in a way that I would not expect based on my intuition of how GPT-3 works. As it's been explained to me in lay terms; GPT is great at predicting the next word based on a chunk of previous context. This feels like it's doing much more than that as it's understanding the context and parameters that relate to a question embedded in the text, which seems unrelated to the original instruction of GPT-3 as I understand. Can anyone explain?

joe-collins 4y ago
I think the sentences in this example are, odd vocabulary notwithstanding, fairly typical descriptions of logical implication, which I expect has non-trivial representation in the training set. GPT-3 is probably capable of matching the pattern of the tokens (sentence structure), even if the content of the tokens (spelling/vocabulary) is novel.
- uup 4y ago
  I doubt it. There are billions of parameters and weights all combining in non trivial ways to produce the output. I don’t think you can describe any of its workings with simple terms like “doing token substitution” even if that’s what it looks like to us when we look at the output.
  - joe-collins 4y ago
    Try it. Reduce the tokens to single characters but keep the structure. Does it work? Maybe single characters looks too much like an abbreviation and that throws it off, because those are found in sufficiently different contexts. Exchange to two characters, three. Throw similar inputs at it and find the point where it reliably "gets it".
mjburgess 4y ago
A NN is a means of compressing a training dataset into a mathematical representation (weights) which can be exploited for pattern finding. The representation of the training data is where 95% of the "magic" comes from. We intelligently prepare it, so no intelligence is required in its use. On any given input, its predictions can "look" how we expect an intelligent response to "look". The job is to disprove this intuition, which is usually trivial. This cannot be done by looking at a single response. Consider walking into a room and predicting a few people's personality traits correctly based on their star sign. This shows nothing.
- YorkshireSeason 4y ago
  This sounds like you consider what has been called "feature engineering" in older ML research as being fundamental to modern ML successes. But the trajectory has been the opposite direction, away from feature engineering, and instead towards letting the transformer architecture learn as much as possible, including interesting features. Most research on deep net architecture, is about how to train it effectively on huge stacks of GPUs and TPUs. That's how/why transformers were invented.

ALittleLight 4y ago

I've been messing with the prompt a little bit and tried to get it to spell out what it thinks is happening. (I changed the name of gloober Blue to gloober Lovely just to make it easier to read). Continuation is at the asterisk. Every turn each gloober flomps the same gloobers. If a yellow gloober flomps a blue gloober then the blue gloober turns yellow, otherwise, gloobers stay the same color. 1. gloober Roy flomps gloober Zoo 2. gloober Roy flomps gloober Bat 3. gloober Zoo flomps gloober Bat 4. gloober Zoo flomps gloober Crystal 5. gloober Crystal flomps gloober X-Ray 6. gloober Bat flomps gloober Lovely gloober Zoo is the only yellow gloober. Write the turn and which gloobers are yellow. 1. Yellow gloobers - Zoo. 2. Yellow gloobers -* Zoo, Bat. 3. Yellow gloobers - Zoo, Bat, Crystal. 4. Yellow gloobers - Zoo, Bat, Crystal, X-Ray. 5. Yellow gloobers - Zoo, Bat, Crystal, X-Ray, Lovely. 6. Yellow gloobers - Zoo, Bat, Crystal, X-Ray, Lovely. Other than thinking the spread of yellow is limited to one gloober at a time, this seems right.

deleted 4y ago
[deleted]
codetrotter 4y ago
> Other than thinking the spread of yellow is limited to one gloober at a time, this seems right. Yup. Here are the results I get when manually applying these rules myself. Here I mention only the flomps that change gloobers to become yellow. Turn 1: Zoo is yellow. Turn 2: Zoo flomps Bat, turning Bat yellow. Zoo flomps Crystal, turning Crystal yellow. Turn 3: Crystal flomps X-Ray, turning X-Ray yellow. Bat flomps Lovely, turning Lovely yellow. In other words it would be: 1. Yellow gloobers - Zoo. 2. Yellow gloobers - Zoo, Bat, Crystal. 3. Yellow gloobers - Zoo, Bat, Crystal, X-Ray, Lovely. So it’s got the order correct, and like you said it’s just that it thinks only one gloober is flomped each turn.

hhhfvk 4y ago

Since GPT-3 is prone to guessing, maybe it's just guessing it's the last step. What happens if you try a misdirection and insert a 5th step?

tveita 4y ago

> It 0-shotted (no other training examples required) it straight out of the box. It 0-shotted it... but incorrectly, right? Gloober Zoo flomps gloober Bat on turn 1. Gloober Bat flomps gloober Blue on turn 2. Answering '4' seems like a total guess, maybe based on there being four steps, or 'Blue' being mentioned in step 4. The replies to your post seem to assume that the answer is correct, but it is actually demonstrating a lack of step-wise reasoning.

londons_explore 4y ago
I think you and OP have a different definition of [subject verb object]... My reading is that on line 1, no action happens, because Roy isn't yellow.
- HL33tibCe7 4y ago
  > If gloober Zoo is the only gloober that is yellow
- tveita 4y ago
  The order seems clear to me: "If a yellow gloober flomps a blue gloober then the blue gloober turns yellow," I still don't see how 4 could be a correct answer, could you post a plausible sequence of turns where Blue turns yellow on turn 4?
  - londons_explore 4y ago
    List of yellow gloobers after each turn. 1. Zoo 2. Zoo, Bat, Crystal 3. Zoo, Bat, Crystal, X-Ray 4. Zoo, Bat, Crystal, X-Ray, Blue
    - tveita 4y ago
      The text says "Every turn each gloober flomps the same gloobers." If I understand you correctly, you are applying only one of the steps each turn, in order, which contradicts the instructions by flomping different gloobers each turn. It is also guessable in multiple ways which is unfortunate. "There are four steps specified so I guess it happens on turn four." is one invalid way of getting the same answer.