4 ms·
Astra, on average, despite its GPT 6 version bump, is not any better at coding than Sol. Some even argue its worse in practice due to the varying quality of its
by Culonavirus 13d ago
Astra, on average, despite its GPT 6 version bump, is not any better at coding than Sol. Some even argue its worse in practice due to the varying quality of its output.
https://x.com/theo/status/2097192907023458473 https://x.com/theo/status/2097192907023458473
(^ this "swearing at a model" thing has happened to me multiple times on Astra already)
If you have to "debate" the quality of a new Big Number model (and double and triple check your eyes and model setting switches when it pukes up complete garbage), that is NOT a good sign.
- heaney-555 13d ago>Astra, on average, despite its GPT 6 version bump, is not any better at coding than Sol. This is a ridiculous claim that is disproven by simply using it for more than 5 minutes. https://withspecific.com/benchmarks/real-swe https://withspecific.com/benchmarks/real-swe
- deleted 13d ago[deleted]
- rico735 11d agoThere is a theory that they are training them on those benchmark tests. No way to know if that's true, but there is also no reason to trust these companies to not do something like that. They lie a lot. The "how much time does it save a dev" tests seem a stronger way to measure success, but not seen one of those run for a while. Not bothered with Astra myself, but the demos I've seen people build don't seem any more impressive on the important stuff. Defaulting to three.js for games just feels smoke-and-mirrors to make them look better, as those games are still as unplayable for the same reasons they were in 2D.