4 ms·

My experience with O1 has been very different. I wouldn't even say it's performing at a "good undergrad" level for me. For example, I asked a pretty simple que

My experience with O1 has been very different. I wouldn't even say it's performing at a "good undergrad" level for me. For example, I asked a pretty simple question here and it got completely confused: https://moorier.com/math-chat-1.png https://moorier.com/math-chat-1.png https://moorier.com/math-chat-2.png https://moorier.com/math-chat-2.png https://moorier.com/math-chat-3.png https://moorier.com/math-chat-3.png (Full chat should be here: https://chatgpt.com/share/66e5d2dd-0b08-8011-89c8-f6895f321733 https://chatgpt.com/share/66e5d2dd-0b08-8011-89c8-f6895f3217...)

jghn 2y ago
Anecdata, but I've been finding O1 to be worse than 4o & Claude 3.5 Sonnet. To add insult to injury, it's slower & chattier.
- anujsjpatel 2y ago
  And sometimes it just bugs out and doesn't give any response? Faced that twice now, it "thought" for like 10-30s then no answer and I had to click regenerate and wait for it again.
  - jghn 2y ago
    I've seen it take over a couple of minutes, at which point I switched to Claude. And have seen reports of it taking even longer. So it may be that you didn't wait long enough.
abdullahkhalids 2y ago
Thinking about training LLMs on geometry. A lot of information in the sources would be contained in the diagrams accompanying the text. This model is not multi-modal, so maybe it wasn't trained on the accompanying diagrams at all. I would really like if people check on a set of geometry and a set of analysis questions and compare the difference.
- jazzyjackson 2y ago
  It will be trash. I'll have to dig up a chat I had the weekend GPT4 was released, I was musing about dodecahedron packing problems and GPT4 started with an assertion that a line through a sphere intersects the surface 3 times. Maybe if you fine tuned it on Euclid's Elements and then allowed it to run experiments with Mathematica snippets it could check its assumptions before spouting nonsense
almostgotcaught 2y ago
Why would they do this - make it speak like a customer service agent. The ideal experience here is short and succinct, not verbose and obsequious.
- ljlolel 2y ago
  Performs better on chat bot arena head to head
svdr 2y ago
Did you find out what the error was in computing the volume of the truncated icosidodecahedron?

jghn 2y ago

Anecdata, but I've been finding O1 to be worse than 4o & Claude 3.5 Sonnet. To add insult to injury, it's slower & chattier.

anujsjpatel 2y ago
And sometimes it just bugs out and doesn't give any response? Faced that twice now, it "thought" for like 10-30s then no answer and I had to click regenerate and wait for it again.
- jghn 2y ago
  I've seen it take over a couple of minutes, at which point I switched to Claude. And have seen reports of it taking even longer. So it may be that you didn't wait long enough.

abdullahkhalids 2y ago

Thinking about training LLMs on geometry. A lot of information in the sources would be contained in the diagrams accompanying the text. This model is not multi-modal, so maybe it wasn't trained on the accompanying diagrams at all. I would really like if people check on a set of geometry and a set of analysis questions and compare the difference.

jazzyjackson 2y ago
It will be trash. I'll have to dig up a chat I had the weekend GPT4 was released, I was musing about dodecahedron packing problems and GPT4 started with an assertion that a line through a sphere intersects the surface 3 times. Maybe if you fine tuned it on Euclid's Elements and then allowed it to run experiments with Mathematica snippets it could check its assumptions before spouting nonsense

almostgotcaught 2y ago

Why would they do this - make it speak like a customer service agent. The ideal experience here is short and succinct, not verbose and obsequious.

ljlolel 2y ago
Performs better on chat bot arena head to head

svdr 2y ago

Did you find out what the error was in computing the volume of the truncated icosidodecahedron?