4 ms·

OK, I wanted to be sure because GPT was getting it quicker than I did.

somenameforme 4y ago
Interesting. Can you share the input/output? Because it'd be quite amusing if ChatGPT 3 can get it, but ChatGPT 4 cannot. Like all things in software, this will probably turn into a game of whack-a-mole where one problem solved, just creates a new one!
- SanderNL 4y ago
  It's important to note that it doesn't do "correct". It can do whatever, it doesn't care about truth. Setting the stage is very important. If you don't it doesn't know if it's doing improv on stage or academic research. Input: You are Math AI. An advanced pattern finding AI. This is your sequence: 0, done, 2, free, 4, hive, ... ? Find a sensible logical pattern and explain. Output: In this sequence, we can observe a pattern involving both numbers and words. The pattern alternates between a numerical value and a word that rhymes with the corresponding number. Here's the sequence again: 0, done, 2, free, 4, hive, ... Let's analyze the pattern: The numerical values are increasing by 2: 0, 2, 4, ... The words rhyme with the numbers preceding them: done (one), free (three), hive (five), ... Following this pattern, the next number in the sequence should be 6, and the word following it should rhyme with "seven". A possible word that rhymes with "seven" is "heaven". Therefore, the next two elements in the sequence are: 6, heaven
  - somenameforme 4y ago
    Interesting! That's not at all what I get with GPT 4! --- * Me: "You are Math AI. An advanced pattern finding AI. This is your sequence: 0, done, 2, free, 4, hive, ... ? Find a sensible logical pattern and explain." --- * GPT4: "I apologize for any confusion, but my capabilities as a language model are not restricted to math or pattern finding. However, I'd be happy to try to help you identify a pattern in the sequence you provided. The pattern that seems to emerge from the sequence is that each number is followed by a word that rhymes with the number spelled out as a word. For example, "done" rhymes with "one," "free" rhymes with "three," and "hive" rhymes with "five." Following this pattern, the next word in the sequence would be a word that rhymes with "six," which could be "sticks," "mix," or "tricks," among other options. However, it's worth noting that this sequence could also be extended in different ways or according to different patterns, depending on the context or the intended purpose of the sequence." --- It's just a really long winded version of making the exact same mistake, as it did in the 'unprimed' version!
    - SanderNL 4y ago
      That's indeed interesting. By the way, it found another pattern: [d, f, h] => j. Your starting letters were sequentially meaningful too. According to this pattern the word "jazz" would be acceptable. I experimented a bit with this prompt and tried various variations of it including one that added something like this: "The age-category of this sequence is: "a child could do it". You are to solve this by pretending to be a young human with average skills and perceptions. You should assume average common sense in your answer." Outputs are like this: ===== "Now, let's look at the words. I noticed that the words have something in common with the numbers that come right before them. They are related to the numbers in this way: 'done' - It sounds like "one" when you say it aloud. 'free' - It sounds like "three" when you say it aloud. 'hive' - It sounds like "five" when you say it aloud." "So, the pattern in the sequence is alternating between even numbers and words that rhyme with the odd numbers. Here's the whole pattern: 0 (even number), 'done' (rhymes with 'one'), 2 (even number), 'free' (rhymes with 'three'), 4 (even number), 'hive' (rhymes with 'five')" ===== I agree with you that language models have vastly different failure modes than humans. Often extremely weird, but to even get close to their potential you do have to probe a bit deeper than just asking it stuff without context. It just semi-randomly walks some semantic space and not all of those branches are fruitfully related to our initial problem. It doesn't know at all what we are doing, there is nothing going on "under the hood". Edit: to be completely clear I'm not arguing it "does know" this answer or something. I don't know if it even "knows", but I do know a thing or two how to get better answers. Current models still are very clearly limited so sooner or later you'll hit it the limit and they start what I call "averaging": you'll get blurry pictures of what you want.

Interesting. Can you share the input/output? Because it'd be quite amusing if ChatGPT 3 can get it, but ChatGPT 4 cannot. Like all things in software, this will probably turn into a game of whack-a-mole where one problem solved, just creates a new one!

SanderNL 4y ago
It's important to note that it doesn't do "correct". It can do whatever, it doesn't care about truth. Setting the stage is very important. If you don't it doesn't know if it's doing improv on stage or academic research. Input: You are Math AI. An advanced pattern finding AI. This is your sequence: 0, done, 2, free, 4, hive, ... ? Find a sensible logical pattern and explain. Output: In this sequence, we can observe a pattern involving both numbers and words. The pattern alternates between a numerical value and a word that rhymes with the corresponding number. Here's the sequence again: 0, done, 2, free, 4, hive, ... Let's analyze the pattern: The numerical values are increasing by 2: 0, 2, 4, ... The words rhyme with the numbers preceding them: done (one), free (three), hive (five), ... Following this pattern, the next number in the sequence should be 6, and the word following it should rhyme with "seven". A possible word that rhymes with "seven" is "heaven". Therefore, the next two elements in the sequence are: 6, heaven
- somenameforme 4y ago
  Interesting! That's not at all what I get with GPT 4! --- * Me: "You are Math AI. An advanced pattern finding AI. This is your sequence: 0, done, 2, free, 4, hive, ... ? Find a sensible logical pattern and explain." --- * GPT4: "I apologize for any confusion, but my capabilities as a language model are not restricted to math or pattern finding. However, I'd be happy to try to help you identify a pattern in the sequence you provided. The pattern that seems to emerge from the sequence is that each number is followed by a word that rhymes with the number spelled out as a word. For example, "done" rhymes with "one," "free" rhymes with "three," and "hive" rhymes with "five." Following this pattern, the next word in the sequence would be a word that rhymes with "six," which could be "sticks," "mix," or "tricks," among other options. However, it's worth noting that this sequence could also be extended in different ways or according to different patterns, depending on the context or the intended purpose of the sequence." --- It's just a really long winded version of making the exact same mistake, as it did in the 'unprimed' version!
  - SanderNL 4y ago
    That's indeed interesting. By the way, it found another pattern: [d, f, h] => j. Your starting letters were sequentially meaningful too. According to this pattern the word "jazz" would be acceptable. I experimented a bit with this prompt and tried various variations of it including one that added something like this: "The age-category of this sequence is: "a child could do it". You are to solve this by pretending to be a young human with average skills and perceptions. You should assume average common sense in your answer." Outputs are like this: ===== "Now, let's look at the words. I noticed that the words have something in common with the numbers that come right before them. They are related to the numbers in this way: 'done' - It sounds like "one" when you say it aloud. 'free' - It sounds like "three" when you say it aloud. 'hive' - It sounds like "five" when you say it aloud." "So, the pattern in the sequence is alternating between even numbers and words that rhyme with the odd numbers. Here's the whole pattern: 0 (even number), 'done' (rhymes with 'one'), 2 (even number), 'free' (rhymes with 'three'), 4 (even number), 'hive' (rhymes with 'five')" ===== I agree with you that language models have vastly different failure modes than humans. Often extremely weird, but to even get close to their potential you do have to probe a bit deeper than just asking it stuff without context. It just semi-randomly walks some semantic space and not all of those branches are fruitfully related to our initial problem. It doesn't know at all what we are doing, there is nothing going on "under the hood". Edit: to be completely clear I'm not arguing it "does know" this answer or something. I don't know if it even "knows", but I do know a thing or two how to get better answers. Current models still are very clearly limited so sooner or later you'll hit it the limit and they start what I call "averaging": you'll get blurry pictures of what you want.