2 ms·

I just started using opus 4.8 and it's still trash at very domain specific stuff and debugging. It went down a rabbit hole trying to coerce the algorithm to wor

by Neywiny 16d ago

I just started using opus 4.8 and it's still trash at very domain specific stuff and debugging. It went down a rabbit hole trying to coerce the algorithm to work for 1 data point, which dramatically overfit the code. Later it did 11 experiments to try and figure out how to do the math, each one yielding an accuracy no better than a coin flip and each one it asserted beforehand that it had it this time, 100% accuracy. We've got some time.

IshKebab 16d ago
That's last gen though. I agree it's mediocre. Try Astra. It's significantly better than Opus and even Fable. I'm not sure about earlier OpenAI models. I also thought we had time.
- Oreb 16d ago
  It seems to depend on the domain, and perhaps the tech stack? I’ve been disappointed by Astra, it’s worse than both Fable and Opus for me.
  - IshKebab 16d ago
    Yeah maybe. I dunno though, I'm using it in obscure SystemVerilog stuff. AI used to absolutely suck at that very recently but now it seems to have no problem. I think the key is that you give it all of the info - put all the docs, the LRM, tool manuals, etc. in a directory so it can read it if it wants (it really really wants).
- Neywiny 16d ago
  They say that every generation. https://www.youtube.com/watch?v=akNcZOoLKVk https://www.youtube.com/watch?v=akNcZOoLKVk
  - IshKebab 15d ago
    I didn't say it was AGI, I just said it wasn't trash and that it's significantly better than it used to be. Anyway just because AI sales people continually jump the gun doesn't mean AGI is never going to happen. It just means you just have to try the models yourself to see how good they really are, which I have done. Astra is really freaking good. (If you aren't paying.) I suggest you give it a try and base your opinion on that.