4 ms·
For months now I've seen benchmarks for lots of models that beat the pants off Claude 3.5 Sonnet, but when I actually try to use those models (using Cline VSCod
by gman83 2y ago
For months now I've seen benchmarks for lots of models that beat the pants off Claude 3.5 Sonnet, but when I actually try to use those models (using Cline VSCode plugin) they never work as well as Claude for programming.
- joshuacc 2y agoPart of that is that Claude is exceptionally good at turn-based interactions compared to other models that are better at one-shot reasoning.
- raincole 2y agoAfter actually using DeepSeek-V3 for a while, the difference betwen it and Sonnet 3.5 is just glaring. My conclusion is that the hype around DeepSeek is either from 1) people who use LLM a lot more than a programmer can reasonably does so they're very price sensitive, like repackage service providers 2) astroturf.
- int_19h 2y agoThere's hype and there's hype. No, DeepSeek-V3 is not better than Sonnet. But it is drastically better than open-weights LLMs we've had before, so it is still a significant increase in "local AI power" - surely you can see why people are excited about that even if SOTA cloud models can still do better? I mean, even if it only just beats the original GPT-4 from two years ago, that still means that things are moving very fast.
- dimgl 2y agoIt's astroturf.