4 ms·

The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will

The curiosity is inefficient though. So many times I have to stop the agent and tell it to just fucking write the code and try compiling it. Otherwise it will fill its entire context tracing through the program logic to derive from the code itself whether the thing it is about to do would work. It completely fails to notice it can just… try.

ACCount37 3mo ago
It's tuned for the kinds of tasks where "just try" doesn't get good results. A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore". You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
- giovannibonetti 3mo ago
  Perhaps what is missing is a better memory/caching layer to avoid doing the same for explorations over and over again.
  - hoppp 3mo ago
    I use the human-in-the-loop for managing the context. Give it only what it needs and do things usually 1 file at a time. Feels like I'm a sort of manual tape editor, if the context was a tape fed into machine, I assemble that and then watch the machine output the results I need.
  - dpritchett 3mo ago
    That is the usual work of high end programmers, right? Growing codebases as consistent, dependable ontologies? I feel like most mainstream programming languages do this sort of work for their standard libraries and their official docs. Go and Python come to mind, but plenty others do this reasonably well to the point where one mostly doesn’t need to read the implementation code to effectively use the standard library itself.
- adastra22 3mo ago
  I get what you're saying, but these instances are of a different type. It is along the lines of "if I pass None for this parameter, will it default to X or return an error?" and it looks, but finds that the actual logic is distributed across multiple files. And so it quickly falls down that rabbit hole. All the while, it could have just done a two-line probe test and see what happens when it calls the API with "None" for that parameter. Or just assum it would act as expected and wire in debug logs in case it doesn't.
CSMastermind 3mo ago
Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.
- ifwinterco 3mo ago
  It’s got worse though right? Older models from before everyone went off the deep end with CoT don’t do this and just write the code with 1/10 the token usage. The downside is the code isn’t as good but it is produced a lot faster and more cheaply and often it’s actually fine. CoT has made LLMs better (say 50% improvement or something) but increases cost by an order of magnitude. That graph is going in the wrong direction and has been for a while now
  - NuclearPM 3mo ago
    CoT?
    - NatKarmios 3mo ago
      Chain of Thought, according to a quick search
hoppp 3mo ago
I think I use it differently. I still mainly stick to web UI. I write a good prompt, paste the code then copy the output code and place it into my project. So in the end I hand assemble and I only give it what it needs to know so no extra context wasted. The human in the loop is of course the secret sauce but this way I am highly efficient, no vibecode and I work really fast too. Everything is audited.
- jumpkick 3mo ago
  This is how I worked with LLMs originally, and I much preferred it. This gave me a much better understanding of the code that I was adding. But, there's no way to keep up with my team like this anymore. It's just too slow when everyone else is working directly in Claude Code.
  - hoppp 3mo ago
    If the entire team is vibe coding and there is no human audit then there is no way but to vibe code, for sure. I would also just vibe it if there is no responsibility, but if I do it that way I don't even care what happens with the project. I get so detached from it that I stop caring and if it has huge critical bugs..I just don't care anymore because it's not my responsibility or my code at all at that point. I'm just there to nudge things along. Just hook it up to Jira and let the managers add the features then pass it off to QA. Real engineering is fully automated at that point.
    - mancerayder 3mo ago
      Are the QA team bearing the brunt of the unexpected issues, bugs, performance etc or is it business-as-usual?
      - FridgeSeal 3mo ago
        Amusing that you think businesses still have QA teams.
zymhan 3mo ago
That's what the person you replied to is saying. You don't need this model.
hetspookjee 3mo ago
Next time it does that expensive scan, run order it to keep or update an index on the codebase. It really helps prevent these expansive scans if you have additional markdown files for LLM navigation.
- deleted 3mo ago
  [deleted]
byzantinegene 3mo ago
that's why i mostly use it for asynchronous work, the inefficiency is something i can bear with because the subscription costs are dirt cheap. if it's token-based, it wouldn't make financial sense.
- adastra22 3mo ago
  I need the full context window to get the work done though.

ACCount37 3mo ago

It's tuned for the kinds of tasks where "just try" doesn't get good results. A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore". You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.

giovannibonetti 3mo ago
Perhaps what is missing is a better memory/caching layer to avoid doing the same for explorations over and over again.
- hoppp 3mo ago
  I use the human-in-the-loop for managing the context. Give it only what it needs and do things usually 1 file at a time. Feels like I'm a sort of manual tape editor, if the context was a tape fed into machine, I assemble that and then watch the machine output the results I need.
- dpritchett 3mo ago
  That is the usual work of high end programmers, right? Growing codebases as consistent, dependable ontologies? I feel like most mainstream programming languages do this sort of work for their standard libraries and their official docs. Go and Python come to mind, but plenty others do this reasonably well to the point where one mostly doesn’t need to read the implementation code to effectively use the standard library itself.
adastra22 3mo ago
I get what you're saying, but these instances are of a different type. It is along the lines of "if I pass None for this parameter, will it default to X or return an error?" and it looks, but finds that the actual logic is distributed across multiple files. And so it quickly falls down that rabbit hole. All the while, it could have just done a two-line probe test and see what happens when it calls the API with "None" for that parameter. Or just assum it would act as expected and wire in debug logs in case it doesn't.

CSMastermind 3mo ago

Everything about LLMs is inefficient. They have their benefits but watching them reason over things that are painfully obvious, that they've literally investigated before (before a memory compaction), never take a step back aand be like 'this is going too slow let me look for a better way', etc. is painful.

ifwinterco 3mo ago
It’s got worse though right? Older models from before everyone went off the deep end with CoT don’t do this and just write the code with 1/10 the token usage. The downside is the code isn’t as good but it is produced a lot faster and more cheaply and often it’s actually fine. CoT has made LLMs better (say 50% improvement or something) but increases cost by an order of magnitude. That graph is going in the wrong direction and has been for a while now
- NuclearPM 3mo ago
  CoT?
  - NatKarmios 3mo ago
    Chain of Thought, according to a quick search

hoppp 3mo ago

I think I use it differently. I still mainly stick to web UI. I write a good prompt, paste the code then copy the output code and place it into my project. So in the end I hand assemble and I only give it what it needs to know so no extra context wasted. The human in the loop is of course the secret sauce but this way I am highly efficient, no vibecode and I work really fast too. Everything is audited.

jumpkick 3mo ago
This is how I worked with LLMs originally, and I much preferred it. This gave me a much better understanding of the code that I was adding. But, there's no way to keep up with my team like this anymore. It's just too slow when everyone else is working directly in Claude Code.
- hoppp 3mo ago
  If the entire team is vibe coding and there is no human audit then there is no way but to vibe code, for sure. I would also just vibe it if there is no responsibility, but if I do it that way I don't even care what happens with the project. I get so detached from it that I stop caring and if it has huge critical bugs..I just don't care anymore because it's not my responsibility or my code at all at that point. I'm just there to nudge things along. Just hook it up to Jira and let the managers add the features then pass it off to QA. Real engineering is fully automated at that point.
  - mancerayder 3mo ago
    Are the QA team bearing the brunt of the unexpected issues, bugs, performance etc or is it business-as-usual?
    - FridgeSeal 3mo ago
      Amusing that you think businesses still have QA teams.

zymhan 3mo ago

That's what the person you replied to is saying. You don't need this model.

hetspookjee 3mo ago

Next time it does that expensive scan, run order it to keep or update an index on the codebase. It really helps prevent these expansive scans if you have additional markdown files for LLM navigation.

deleted 3mo ago
[deleted]

byzantinegene 3mo ago

that's why i mostly use it for asynchronous work, the inefficiency is something i can bear with because the subscription costs are dirt cheap. if it's token-based, it wouldn't make financial sense.

adastra22 3mo ago
I need the full context window to get the work done though.