5 ms·

The point still is that considering they 1) named the model strawberry, 2) released a promo video showing the model solving this exact problem successfully and

by paxys 2y ago

The point still is that considering they 1) named the model strawberry, 2) released a promo video showing the model solving this exact problem successfully and 3) put a strawberry joke as an example prompt on the landing page, you'd have expected them to actually have fixed it. Otherwise why even bring it up so many times?

GaggiX 2y ago
Well they did in the testing I have seen.
zamadatix 2y ago
The author's battle is that it doesn't solve the class problem correctly 100% of the time like a one off hack does, not that it does nothing for it at all. https://i.imgur.com/Ar3rlJ1.png https://i.imgur.com/Ar3rlJ1.png
- paxys 2y ago
  This was the case with 4o as well. It sometimes solved it, sometimes didn't.
  - zamadatix 2y ago
    One could say that down to a markov chain with noise, the perspective is the new model named after the problem solves it significantly more reliably than the previous without a problem specific hack. It's also worth noting the current model is the lower scoring o1-preview, not o1.
viraptor 2y ago
People who understand where the problem comes from know to ignore it and understand the irony of the naming. Others have a few seconds of fun. The problem and solving it is almost irrelevant.
- paxys 2y ago
  An AI that people consider to be AGI, or at least on a path to AGI, not being able to count the number of letters in a word is "almost irrelevant"?
  - viraptor 2y ago
    Yes. Nobody is seriously going to ask the system to count letters in a word they just typed. It could be interesting for crosswords and similar things, but adding "use python" solves those already. Bringing up the strawberry is like saying: "people consider computers to be great at dealing with numbers, but they can't even add 0.1 and 0.2 correctly". You learn about this limitation once, understand how to deal with it, get on with your life.
    - paxys 2y ago
      "Nobody is seriously going to..." People will throw any problem at it they want to. That's the entire point of general intelligence.
- lupire 2y ago
  Answering simple computational questions correctly is not almost irrelevant.
  - viraptor 2y ago
    If the question is simpler than any reasonable usecase, then it's basically irrelevant. It's as irrelevant as asking a math genius what's 1+1 and getting answer "a bazillion". Was it wrong - yes. Was everyone's time wasted - also yes. Openai people know this is something models don't answer right. And they barely care to attempt fixing it - it's basically a joke at this point. But it's irrelevant because nobody pays OpenAI to ask about spelling of words they just typed. If they actually had that need, then "use python" or similar approaches work just fine. The model could be also taught to call a function to get the token-to-spelling mapping it needed.
  - bdjsiqoocwk 2y ago

by paxys 2y ago

GaggiX 2y ago

Well they did in the testing I have seen.

zamadatix 2y ago

The author's battle is that it doesn't solve the class problem correctly 100% of the time like a one off hack does, not that it does nothing for it at all. https://i.imgur.com/Ar3rlJ1.png https://i.imgur.com/Ar3rlJ1.png

paxys 2y ago
This was the case with 4o as well. It sometimes solved it, sometimes didn't.
- zamadatix 2y ago
  One could say that down to a markov chain with noise, the perspective is the new model named after the problem solves it significantly more reliably than the previous without a problem specific hack. It's also worth noting the current model is the lower scoring o1-preview, not o1.

viraptor 2y ago

People who understand where the problem comes from know to ignore it and understand the irony of the naming. Others have a few seconds of fun. The problem and solving it is almost irrelevant.

paxys 2y ago
An AI that people consider to be AGI, or at least on a path to AGI, not being able to count the number of letters in a word is "almost irrelevant"?
- viraptor 2y ago
  Yes. Nobody is seriously going to ask the system to count letters in a word they just typed. It could be interesting for crosswords and similar things, but adding "use python" solves those already. Bringing up the strawberry is like saying: "people consider computers to be great at dealing with numbers, but they can't even add 0.1 and 0.2 correctly". You learn about this limitation once, understand how to deal with it, get on with your life.
  - paxys 2y ago
    "Nobody is seriously going to..." People will throw any problem at it they want to. That's the entire point of general intelligence.
lupire 2y ago
Answering simple computational questions correctly is not almost irrelevant.
- viraptor 2y ago
  If the question is simpler than any reasonable usecase, then it's basically irrelevant. It's as irrelevant as asking a math genius what's 1+1 and getting answer "a bazillion". Was it wrong - yes. Was everyone's time wasted - also yes. Openai people know this is something models don't answer right. And they barely care to attempt fixing it - it's basically a joke at this point. But it's irrelevant because nobody pays OpenAI to ask about spelling of words they just typed. If they actually had that need, then "use python" or similar approaches work just fine. The model could be also taught to call a function to get the token-to-spelling mapping it needed.
- bdjsiqoocwk 2y ago