4 ms·

Wired: LLM are practically AGI Tired: ChatGPT thinks February 29th isn't a valid date.

AlienRobot 3y ago
That's because it's trained on 2021 data. No February 29 back then!
pants2 3y ago
For the record, both ChatGPT-4 and Gemini Ultra affirmed that it's a valid date. Gemini reasoned through it and GPT-4 ran python code to make sure 2024 was divisible by 4.
- daseiner1 3y ago
  Did it also check if 2024 is divisible by 100 but not 400?
- roland35 3y ago
  Interesting... But that isn't exactly true! Centuries that are not divisible by 4 don't count!
  - swores 3y ago
    I've yet to come across a form of 100 that isn't divisible by 4... since 25 usually still exists! But I do remember there being some weird niche rules about which years are or aren't leap years, so I'm guessing your comment is basically right just wrongly worded?
    - latexr 3y ago
      The rule is that leap years are the ones divisible by 4. Unless it’s also divisible by 100. Unless unless it’s divisible by 400. So 2000 was leap, but 2100, 2200, and 2300 won’t be, but 2400 will be.
      - swores 3y ago
        Ahh, so it's centuries that aren't divisible by 400 rather than that aren't divisible by 4, that makes more sense! Thanks for answering
        thaumasiotes 3y ago
        It's centuries that aren't divisible by 4. It isn't years that aren't divisible by 4.
    - inglor_cz 3y ago
      The GP formulated it in a somewhat unclear way. "Centuries" divisible by 4 probably meant "years" divisible by 400. So, 19th century (1900 is the last year) isn't divisible by 4 (19/4 is not integer), which is the same as saying that 1900 isn't divisible by 400. This is the main reform of the Gregorian calendar - leap days aren't introduced on xy00 years which aren't divisible by 400. This corrects the length of a year to 365.2425 days, which is fairly close to the real value of 364.2422 days. The original Julian calendar had year of 365.25 days, which aggregated an error of more than ten days over the centuries.
peteradio 3y ago
But akshually neither does my uncle Ned
pquki4 3y ago
My favorite prompt: asking "How many e's are there in the word egregious". Always says three (ChatGPT 3.5, 4, 4 turbo). When you ask it which three, it realizes its mistake and apologizes (or sometimes tells you where they are that is completely wrong). Looks like it just outputs gibberish for these things.
- devjab 3y ago
  It’s always gibberish, it’s just really good at guessing. I forgot what exactly it was I was doing. We were trying to get it to generate word lists of words ending with x or maybe it was starting with. For a marketing PoC and it made up oceans of words that not only didn’t start/end with x but mostly didn’t include x at all. Isn’t this also why it can pass CS exams and job interviews better than like 95% of us, but then can’t help you solve the most simple business process in the world. Because nobody has asked that question two billion times on its training data.
  - mewpmewp2 3y ago
    But also it doesn't see characters. It sees tokens. The only way it would be reliably able to solve it is if it had a lookup table of token to characters. Which it likely doesn't. You couldn't do it either unless you learned the exact matchings of all tokens to all characters in that token and their positions if you were given tokens as an input. You would have learned the meaning of the token, but not what the exact characters it represents.
    - yousif_123123 3y ago
      Even if it sees tokens, I don't think it's an impossible task. Certainly an advanced enough LLM should be able to decipher token meanings, to know that a word is made up of the individual character tokens regardless of how the full word is tokenized. Maybe something gpt5 can do (or there's some real technical limitation which I don't understand)
      - Izkata 3y ago

That's because it's trained on 2021 data. No February 29 back then!

For the record, both ChatGPT-4 and Gemini Ultra affirmed that it's a valid date. Gemini reasoned through it and GPT-4 ran python code to make sure 2024 was divisible by 4.

daseiner1 3y ago
Did it also check if 2024 is divisible by 100 but not 400?
roland35 3y ago
Interesting... But that isn't exactly true! Centuries that are not divisible by 4 don't count!
- swores 3y ago
  I've yet to come across a form of 100 that isn't divisible by 4... since 25 usually still exists! But I do remember there being some weird niche rules about which years are or aren't leap years, so I'm guessing your comment is basically right just wrongly worded?
  - latexr 3y ago
    The rule is that leap years are the ones divisible by 4. Unless it’s also divisible by 100. Unless unless it’s divisible by 400. So 2000 was leap, but 2100, 2200, and 2300 won’t be, but 2400 will be.
    - swores 3y ago
      Ahh, so it's centuries that aren't divisible by 400 rather than that aren't divisible by 4, that makes more sense! Thanks for answering
      - thaumasiotes 3y ago
        It's centuries that aren't divisible by 4. It isn't years that aren't divisible by 4.
  - inglor_cz 3y ago
    The GP formulated it in a somewhat unclear way. "Centuries" divisible by 4 probably meant "years" divisible by 400. So, 19th century (1900 is the last year) isn't divisible by 4 (19/4 is not integer), which is the same as saying that 1900 isn't divisible by 400. This is the main reform of the Gregorian calendar - leap days aren't introduced on xy00 years which aren't divisible by 400. This corrects the length of a year to 365.2425 days, which is fairly close to the real value of 364.2422 days. The original Julian calendar had year of 365.25 days, which aggregated an error of more than ten days over the centuries.

peteradio 3y ago

But akshually neither does my uncle Ned

pquki4 3y ago

My favorite prompt: asking "How many e's are there in the word egregious". Always says three (ChatGPT 3.5, 4, 4 turbo). When you ask it which three, it realizes its mistake and apologizes (or sometimes tells you where they are that is completely wrong). Looks like it just outputs gibberish for these things.

devjab 3y ago
It’s always gibberish, it’s just really good at guessing. I forgot what exactly it was I was doing. We were trying to get it to generate word lists of words ending with x or maybe it was starting with. For a marketing PoC and it made up oceans of words that not only didn’t start/end with x but mostly didn’t include x at all. Isn’t this also why it can pass CS exams and job interviews better than like 95% of us, but then can’t help you solve the most simple business process in the world. Because nobody has asked that question two billion times on its training data.
- mewpmewp2 3y ago
  But also it doesn't see characters. It sees tokens. The only way it would be reliably able to solve it is if it had a lookup table of token to characters. Which it likely doesn't. You couldn't do it either unless you learned the exact matchings of all tokens to all characters in that token and their positions if you were given tokens as an input. You would have learned the meaning of the token, but not what the exact characters it represents.
  - yousif_123123 3y ago
    Even if it sees tokens, I don't think it's an impossible task. Certainly an advanced enough LLM should be able to decipher token meanings, to know that a word is made up of the individual character tokens regardless of how the full word is tokenized. Maybe something gpt5 can do (or there's some real technical limitation which I don't understand)
    - Izkata 3y ago