4 ms·
I was actually very impressed with some math questions it was able to answer for me: S: If flabb means three, snabb means four, and whabb mean five, what is fl
by SamBam 4y ago
I was actually very impressed with some math questions it was able to answer for me:
S: If flabb means three, snabb means four, and whabb mean five, what is flabb plus whabb?
If flabb means three, snabb means four, and whabb means five, then flabb plus whabb would be equal to three plus five, or eight. In this made-up numbering system, the words “flabb,” “snabb,” and “whabb” represent the numbers three, four, and five, respectively. When these numbers are added together, the result is the number eight.
S: If bbb means three, bbbb means four and bbbbb means five, how do you think you would write six?
If bbb means three, bbbb means four, and bbbbb means five, then six could potentially be represented by the word “bbbbbbb.” In this made-up numbering system, the number of "b"s in a word corresponds to the value of the number it represents. Therefore, to represent the number six, one possible way would be to add another “b” to the end of the word for five, which would give us “bbbbbbb” as the representation of six. This is just one possible way to represent six in this numbering system, and there may be other ways to do it as well.
S: Why do people argue about the answer of mathematical questions like “8 / 2(2+2)”?
There are a few reasons why people may argue about the answer to a mathematical question like “8 / 2(2+2)”. One reason is that there is more than one wav to interpret the question and arrive at a different answer. For example, some people may interpret the question as “8 / (2 * (2+2))” and arrive at an answer of 1, while others mav interpret it as “(8 / 2) * (2+2)” and arrive at an answer of 16.
- kevinventullo 4y agoThat is very cool, and surprising to me given I was also initially underwhelmed. One nitpick though: in the second question, although the explanation itself is clean, there appear to be seven b’s in its representation of the number six.
- wyldfire 4y ago> This is just one possible way to represent six in this numbering system, and there may be other ways to do it as well. Gotta hand it to ChatGPT. It's not wrong.
- BoppreH 4y agoI've heard it's because GPT's input is whole words/tokens, not characters. So it has little insight into their spelling or letter positions, unless it's explicitly mentioned in the training set (e.g. rhymes are easy, counting letters of rare words is hard). Though this is a specially bad example, surely it had bbbbbb somewhere in it's training set associated with "6".
- xg15 4y agoIt's even more interesting, I think. The tokens are a byte pair encoding [1] of the input string. So a short, frequent word might be represented as one token, but an infrequent word (such as "bbbbbbb") might be represented by several tokens, each of which might or might not correspond to a letter. This might also explain the weird "off-by-one" errors with the ROT13 task. [1] https://en.m.wikipedia.org/wiki/Byte_pair_encoding https://en.m.wikipedia.org/wiki/Byte_pair_encoding
- deleted 4y ago[deleted]
- wyldfire 4y ago> There are a few reasons why people may argue about the answer to a mathematical question like “8 / 2(2+2)”. One reason is that there is more than one wav to interpret the question and arrive at a different answer. For example, some people may interpret the question as “8 / (2 * (2+2))” and arrive at an answer of 1, while others mav interpret it as “(8 / 2) * (2+2)” and arrive at an answer of 16. Wow, this is kinda astonishing. Could it ruminate about the precedence of the operators? Also: the words "may" and "way" appear as "mav" and "wav" in your text. Is this a problem with your input to the HN comment or did the model really make these textual errors?
- LegitShady 4y agoits odd to me that it doesn't explicitly mention 'order of operations'.
- otabdeveloper4 4y ago8 / 2(2+2) was an internet meme, so not that astonishing. It should be in the training set somewhere.
- joshuahedlund 4y agoIt would be interesting to repeat the experiment with different integers.
- Lockal 4y agoReplace "2+2" with "2-2" and the answer is still the same. It does not read expressions, it does not see divisions by zero. It just outputs the training data, associated with parenthesis. It will still hallucinate "division by zero" fallacies when asked questions like: Given that a - b > 0, can we simplify? a / (a - b) = 1 / (a - b) a = 1
- tmp60beb0ed 4y agoRegarding the issue of "mav", it looks like the poster used screen capture followed by some image-to-text OCR routine (possibly tesseract). This feels like an OCR fail.
- deleted 4y ago[deleted]
- SketchySeaBeast 4y ago> If flabb means three, snabb means four, and whabb means five, then flabb plus whabb would be equal to three plus five, or eight. In this made-up numbering system, the words “flabb,” “snabb,” and “whabb” represent the numbers three, four, and five, respectively. When these numbers are added together, the result is the number eight. Was its training model a freshman university student trying to get through a 10 page essay?
- SamBam 4y agoActually, I cut some of the more verbose parts of all three answers...
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- aaron695 4y ago
- dalmo3 4y ago> bbbbbbb It's impressive that gpt was able to abstract the number system, but... There are 7 Bs there.
- hackerman123469 4y agoI caught that too!
- deleted 4y ago[deleted]