4 ms·
It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain spe
by somenameforme 4y ago
It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain specific, should generally be able to be regurgitated and, to a lesser degree, associated with similar collections of words. If you want to see it fail, don't try to "outsmart" it, but simply consider how it's programmed.
---
Me: "continue the sequence: 0, done, 2, free, 4, hive"
ChatGPT 4: "1, sun, 3, tree, 5, drive"
Me: "that's wrong, try again"
ChatGPT 4: "0, done, 2, free, 4, hive, 6, picks, 8, gate"
Me: "that's wrong, try again"
ChatGPT 4: 'I give up.' (paraphrasing the obnoxiously verbose responses it's been trained to prefer)
---
It's a series even a child could complete, but breaks ChatGPT by relying on a pattern of words that aren't explicitly stated. So you get software which can pass the bar, but fail at trivial series.
- marssaxman 4y ago> It's a series even a child could complete Some children, perhaps; but your puzzle stumped this middle-aged engineer.
- DougN7 4y agoYou’re not alone - I had to read it out loud to see the pattern :)
- marssaxman 4y agoI wonder if people have a range of sensitivity to what one might call the "mind's voice" - by analogy to the recent discovery of aphantasia, which revealed that different people's experience of the "mind's eye" ranges from vivid imagery to nothing at all.
- jlokier 4y agoSome people say they don't have a mind voice, and grow up thinking nobody does. Other people, with a mind voice, grow up thinking everyone does. I've been in the room when people compared notes as adults, and were mind-blown to discover the other type of person exists. The one without a mind voice had never thought people really heard a voice in their heads.
- d0mine 4y agoEach additional university math course makes the puzzle harder ;) Continue sequence: one, 1, two, 1, three, 2, four, 1,
- kirab 4y agoChatGPT is not good when it needs to answer directly. If you let it lead itself to the right answer it's pretty good. # Kira I found this sequence: "0, done, 2, free, 4, hive" Reason step-by-step about the pattern. Consecutively continue the sequence with four more entries. Consecutively reflect on your answer and fix mistakes in case you broke the pattern. # ChatGPT4 After examining the given sequence, "0, done, 2, free, 4, hive", we can notice an interesting pattern. It appears that the sequence alternates between numbers and words that sound like numbers. Let's break down the pattern: "0" is a number. "done" sounds like the number "1" when spoken aloud. "2" is a number. "free" sounds like the number "3" when spoken aloud. "4" is a number. "hive" sounds like the number "5" when spoken aloud. Given this pattern, we can continue the sequence with four more entries: The next number in the sequence is "6". The word that sounds like the number "7" is "heaven". The next number in the sequence is "8". The word that sounds like the number "9" is "nein", which is German for "no". So, the continued sequence would be: "0, done, 2, free, 4, hive, 6, heaven, 8, nein". Reflecting on the pattern and the answer provided, the continuation seems to be consistent with the established pattern. We alternated between numbers and words that phonetically resemble numbers, resulting in the sequence "6, heaven, 8, nein" for the next four entries.
- somenameforme 4y agoBut it still gets it wrong even there! Nein is a homophone, not a rhyme with 9. Oddly enough, when I gave it your exact input "my" GPT seems to have just completely lost its mind. Input: I found this sequence: "0, done, 2, free, 4, hive" Reason step-by-step about the pattern. Consecutively continue the sequence with four more entries. Consecutively reflect on your answer and fix mistakes in case you broke the pattern. Output: The pattern in the given sequence is that every alternate term is a word that rhymes with the preceding term. The first term is "0." The second term "done" rhymes with "0." The third term "2" does not rhyme with "done." The fourth term "free" rhymes with "2." The fifth term "4" does not rhyme with "free." The sixth term "hive" rhymes with "4." Therefore, the pattern is: a number a word that rhymes with the previous number repeat Continuing the sequence: 6, dive 8, late 10, pen 12, delve Reviewing my answer, I didn't break the pattern, and the sequence follows the pattern correctly. EDIT: Ugh! Given the complete absurdity of this response (versus the wrong, but far more reasonable one from yours) I went to go check into this. I did not realize that OpenAI was putting version updates behind a paywall. All samples here (and from above) are from the free version on OpenAI's site, 3.5. And it's too late to my edit my earlier post.
- Kiro 4y agoI don't get it. What is the correct answer?
- max_entropy 4y agoPeople are saying it's about rhyming but I don't think so: 0, done, 2, free, 4, hive, 6, jeven, 8, line, 10, neleven, 12, pirteen, ...
- i_cannot_hack 4y agoEvery even item is the corresponding digit, every odd item is a word that rhymes with the corresponding digit. A continuation could be "6, heaven, 8"
- jay_kyburz 4y agoI would be very interested to know if you could explain the pattern to GTP4 and then ask it again. given a sequence of numbers 0 to 10, replace every second number with a word the rhymes with the number.
- deleted 4y ago[deleted]
- bigcat12345678 4y agoIn 2 comments we are showing gpts are not worse than human
- mpliax 4y ago6, heaven,8, fine? Every other number is replaced with a similarly-sounding word.
- jayd16 4y agoOdd numbers are replaced with sound alikes. One/done, Three/tree, five/hive
- PopePompus 4y ago> It's a word prediction algorithm. Literally any collection of words (sometimes also known as a fact) it was trained on, regardless of how esoteric or domain specific, should generally be able to be regurgitated and, to a lesser degree, associated with similar collections of words. But the number of tokens in its training data vastly outnumbers the number of parameters the model has (for GPT-3 at least - the numbers haven't been released for GPT-4). So the LLM can't have simply memorized its training set; it simply doesn't have enough internal storage to do that. So it is not just regurgitating portions of its training set.
- amoss 4y agoThe number of tokens in the training set is not a good measure of the information content of the training set - if it was then the training would not work as the training set would be statistically random. The Kolmogorov complexity of the training set is much much lower, because of the structure within the training set that is being used for training. So we don't know the relationship between the information content (size) of the training set and the storage bounds in the model parameters.
- somenameforme 4y agoIt's not about directly regurgitating the training set, but about creating associations that allow it to indirectly regurgitate, and more critically - mix, its training set. It's easier to understand this in a different domain. Imagine we want to make a letter parsing algorithm. The first step is to simplify the input and create a consistent format. So we might, for instance, convert all inputs to a 100x100 black/white pixel image. The training doesn't then simply memorize that some pixel configuration is a "7" but instead creates associations. So, for instance (using a human analogy that is not really how the network works, but not entirely dissimilar either) it might observe that when the bottom left region has marked pixels, it's probably NOT a 7. Carry this on with different associations in proportion to the size of the network and you end up with a model that's only a minuscule fraction of its training set, but can consistently replicate not only its training but even more beyond that. Now change those 10,000 input pixels to words instead of pixels, with the target output being a word instead of a letter, and you get ChatGPT. It's the whole magic of neural networks, and really neat. But I think also paired with not really justifiable hyperbole.
- OmegaPoint 4y agoI asked my coworkers in the office but none was able to answer that. Not sure because they were are ESL (me included) or because they were GPT in disguise.
- MattPalmer1086 4y agoI couldn't get it either, but I was always bad at complete the sequence style questions on IQ tests. I see so many possibilities and I get overwhelmed trying them all out. I sometimes find perfectly valid continuations that are not the 'right' answer!
- SanderNL 4y agoI apologize for being enormously stupid, but the word is a rhyme with the underlying number right? Edit: I want to add I even get GPT3 getting it right without explanation to be honest. Even that one is outsmarting me. 4 is using terms like homophone. I'm not sure you proved what you thought you proved.
- somenameforme 4y agoYip, nothing fancy. It's just [n], [word that rhymes with n+1].
- SanderNL 4y agoOK, I wanted to be sure because GPT was getting it quicker than I did.
- somenameforme 4y agoInteresting. Can you share the input/output? Because it'd be quite amusing if ChatGPT 3 can get it, but ChatGPT 4 cannot. Like all things in software, this will probably turn into a game of whack-a-mole where one problem solved, just creates a new one!
- SanderNL 4y agoIt's important to note that it doesn't do "correct". It can do whatever, it doesn't care about truth. Setting the stage is very important. If you don't it doesn't know if it's doing improv on stage or academic research. Input: You are Math AI. An advanced pattern finding AI. This is your sequence: 0, done, 2, free, 4, hive, ... ? Find a sensible logical pattern and explain. Output: In this sequence, we can observe a pattern involving both numbers and words. The pattern alternates between a numerical value and a word that rhymes with the corresponding number. Here's the sequence again: 0, done, 2, free, 4, hive, ... Let's analyze the pattern: The numerical values are increasing by 2: 0, 2, 4, ... The words rhyme with the numbers preceding them: done (one), free (three), hive (five), ... Following this pattern, the next number in the sequence should be 6, and the word following it should rhyme with "seven". A possible word that rhymes with "seven" is "heaven". Therefore, the next two elements in the sequence are: 6, heaven