4 ms·
> This seems like it's fixing the symptom rather than the underlying issue? Exactly. LLM does not know how to use a debugger. LLM does not have runtime context
by vachina 1y ago
> This seems like it's fixing the symptom rather than the underlying issue?
Exactly. LLM does not know how to use a debugger. LLM does not have runtime contexts.
For all we know, the LLM could’ve fixed the issue simply by commenting out the assertions or sanity checks and everything seemed fine and dandy until every client’s device catches on fire.
- tossandthrow 1y agoThis was my latest experience of using agents. It created code with hard coded values from the tests.
- uludag 1y agoAnd if you were to attach a debugger to a SOTA LLM, give it a compute environment, have it constantly redo work when CI fails, I can easily imagine each of these PRs burning hundreds of dollars and still have a good chance at failing the task.