4 ms·
Codex has got me a few times lately, doing what I asked but certainly not what I intended: - Get rid of these warnings "...": captures and silences warnings in
by nxpnsv 1y ago
Codex has got me a few times lately, doing what I asked but certainly not what I intended:
- Get rid of these warnings "...": captures and silences warnings instead of fixing them
- Update this unit test to relfect the changes "...": changes the code so the outdated test works
- The argument passed is now wrong: catches the exception instead of fixing the argument
My advice is to prefer small changes and read everything it does before accepting anything, often this means using the agent actually is slower than just coding...
- d1sxeyes 1y agoYou also have to be a bit careful: “Fix the issues causing these warnings” Retrospectively fixing a test to be passing given the current code is a complex task, instead, you can ask it to write a test that tests the intended behaviour, without needing to infer it. “The argument passed is now wrong” - you’re asking the LLM to infer that there’s a problem somewhere else, and to find and fix it. When you’re asking an LLM to do something, you have to be very explicit about what you want it to do.
- nxpnsv 1y agoExactly, I think the takeaway is that being careful when formulating a task is essential with LLMs. They make errors that wouldn’t be expected when asking the same from a person.