4 ms·
> Junior engineer learn, AI does not. This is technically true, but lets not act like we haven't seen immense improvement of both models are harnesses for thes
by dpoloncsak 5mo ago
> Junior engineer learn, AI does not.
This is technically true, but lets not act like we haven't seen immense improvement of both models are harnesses for these models in the past years. They may not be learning, but they are getting better
- nyrikki 5mo agoThey are getting better at historical data, not at the fundamental issue. As a recent example, I recently had to abandon the multiple LLM reviewer/verifier model I was using because zig 0.16 was released with major changes. I actually reverted back to full self hosted because the foundation models we’re trying too hard to revert to the older versions of the language. It is going to be a balancing act and there is fundamentally no way for LLMs to get around this. We will have to develop methods to do so, most likely by focusing agents on problems that are more static.
- askonomm 5mo agoI find great success in not relying on LLM's built-in knowledge, but giving it links to necessary docs/manuals and have it read that before doing anything.
- embedding-shape 5mo agoAlso, add "no assumptions or guesses" and if you use a model with really strong prompt adherence (most SOTA models), they'll figure out the right version first, then look up docs, then implement.
- nyrikki 5mo agoCurrently, with zig 0.16 the agent has to have access to the zig compiler and std library to even produce code that will compile. If you have zig installed, you can run ‘zig std’ to see that. You still have the limitations of attention etc… Even zed’s agent will leverage that built in tarball, but it doesn’t solve the problem, especially as some of the languages killer features are unavailable in C and other languages.
- smj-edison 5mo agoQuestion for you, since I also use Zig 0.16: how do you get it to use Zig idioms? I use Kimi 2.6, and I feel like whenever I try to get my agent to write modern Zig based on a C reference it decides to start writing everything in a C style (doesn't use defer, doesn't use opaque enums even when I explicitly tell it to, doesn't use Zig's error unions, swallows errors instead of asserting, and some more). It's quite frustrating, and a lot of catchable errors crop up until I've beat modern practices into it.
- nyrikki 5mo agoI don’t, it is mostly used for ideas, review, etc… Getting the agent to grep std, example code, comments that reference inaccessible security or bugs etc.. help a little. But for my needs, not refactoring would just be stepping over dollars to pick up pennies. But yes it is a problem.
- bigstrat2003 5mo agoPretend? I don't have to pretend, I haven't seen any real improvement. I wouldn't let the models of today write code one bit more than the models of several years ago, because they still suck at it.
- deleted 5mo ago[deleted]
- dpoloncsak 5mo agoModels several years ago would struggle to provide code that would compile, and need to be fed whatever errors were thrown to be able to resolve them. Today's models often output working code. I've had OpenClaw instances one shot simple static web-page HTML, Apache installation, and deployment. It may not meet modern standards or be as secure as you'd like, but fundamentally this is an improvement from previous models.
- antonios_makro 5mo agoAgreed, the "one-shot a static site" demo is the new "hello world" for agents. It's a real step up.
- goran-j 5mo agoyep, they need something a bit more challenging than printing two words on the screen
- antonios_makro 5mo ago:p