4 ms·
Still not as good at math as gpt4o from benchmarks and also my experience
by Davidzheng 2y ago
Still not as good at math as gpt4o from benchmarks and also my experience
- Legend2440 2y agoThat's ok, I don't need my language model to do math. My calculator does that just fine.
- Davidzheng 2y agoMath is not just arithmetic
- ta988 2y agoIt is not just about math it is about tokenization and chain of thought.
- muzani 2y agome: "is 400000 the same as 4 * 100 * 100" gpt-4o: "Yes, 400,000 is the same as 4 X 100 × 1000." gpt-4: Yes, 400,000 is the same as 4 X 100 × 100. Here's the breakdown: 4 × 100 = 400 400 × 100 = 40,000 Oops, it looks like there was an error in my previous calculation. So, 4 × 100 X 100 actually equals 40,000, not 400,000. To make it 400,000, you would need to use 4 × 10 x 10,000 or adjust the numbers similarly. claude-3.5: No, 400000 is not the same as 4 * 100 * 100. Let's break it down: 4 * 100 * 100 = 4 * 10000 = 40000 400000 is ten times larger than 40000. Everyone keeps saying gpt-4o beats benchmarks and stuff, but this is consistently my experience with it. The benchmarks fall far off my every day experience.
- Davidzheng 2y agoThe thing is, gpt4(o) is the only model I've talked to that makes me feel like it understands calculus. Not like it's pretending and succeeding
- muzani 2y agoAh, I think it's just much better trained. At double the speed and half the price, I'd assume there's less computing power behind it. But likely the training covers some topics really well.
- Jverse 2y agoMy answer from gpt4o was correct and similar to your example from claude. Maybe you were just unlucky?
- muzani 2y agoThis kind of thing happens so frequently that I don't think so. My guess is the language auto detection messes it up for multilinguals and it works fine for people who are solely using English? Because it's certainly broken on voice input - I ask it a question in English and it will respond in Russian and such.
- deleted 2y ago[deleted]