3 ms·

when this happens I do thew following 1) switch to a more expensive llm and ask it to debug: add debugging statements, reason about what's going on, try small

when this happens I do thew following 1) switch to a more expensive llm and ask it to debug: add debugging statements, reason about what's going on, try small tasks, etc 2) find issue 3) ask it to summarize what was wrong and what to do differently next time 4) copy and paste that recommendation to a small text document 5) revert to the original state and ask the llm to make the change with the recommendation as context

rurp 1y ago
This honestly sounds slower than just doing it myself, and with more potential for bugs or non-standard code. I've had the same experience as parent where LLMs are great for simple tasks but still fall down surprisingly quickly on anything complex and sometimes make simple problems complex. Just a few days ago I asked Claude how to do something with a library and rather than give me the simple answer it suggested I rewrite a large chunk of that library instead, in a way that I highly doubt was bug-free. Fortunately I figured there would be a much simpler answer but mistakes like that could easily slip through.
- qazxcvbnmlp 1y ago
  it is slower than doing it yourself in the following scenarios 1) working in a language you are familiar in but has speed advantages in other areas 1) working in languages and or frameworks you are not familiar in 2) by documenting where the llm went wrong, you can append that to its rules and avoid the issue next time
- ziml77 1y ago
  Yeah if it gets stuck and can't easily get itself unstuck, that's when I step in to do the work for it. Otherwise it will continue to make more and more of a mess as it iterates on its own code.
nico 1y ago
> 1) switch to a more expensive llm and ask it to debug You might not even need to switch A lot of times, just asking the model to debug an issue, instead of fixing it, helps to get the model unstuck (and also helps providing better context)

rurp 1y ago

This honestly sounds slower than just doing it myself, and with more potential for bugs or non-standard code. I've had the same experience as parent where LLMs are great for simple tasks but still fall down surprisingly quickly on anything complex and sometimes make simple problems complex. Just a few days ago I asked Claude how to do something with a library and rather than give me the simple answer it suggested I rewrite a large chunk of that library instead, in a way that I highly doubt was bug-free. Fortunately I figured there would be a much simpler answer but mistakes like that could easily slip through.

qazxcvbnmlp 1y ago
it is slower than doing it yourself in the following scenarios 1) working in a language you are familiar in but has speed advantages in other areas 1) working in languages and or frameworks you are not familiar in 2) by documenting where the llm went wrong, you can append that to its rules and avoid the issue next time
ziml77 1y ago
Yeah if it gets stuck and can't easily get itself unstuck, that's when I step in to do the work for it. Otherwise it will continue to make more and more of a mess as it iterates on its own code.

nico 1y ago

> 1) switch to a more expensive llm and ask it to debug You might not even need to switch A lot of times, just asking the model to debug an issue, instead of fixing it, helps to get the model unstuck (and also helps providing better context)