3 ms·

When I generated a few tests with a LLM, there was one that were generated to test he code as written, not the code as I should have wrote it. Was great to hav

• RealityVoid 2y ago
  > there was one that were generated to test he code as written, not the code as I should have wrote it. So like 90% of human-written tests then?
• chfalck 2y ago
  This is where prompt engineering becomes more important. Next time consider pre-pending some kind of plain English set of expectations before pasting your code. Something like, “I want you to write tests for this code. Here are the expected behaviors <expected behaviors list>, and here are unexpected behaviors <unexpected behaviors list>. Tests should pass if they adhere to the expected behaviors and fail if they have unexpected behaviors. Here is the code: <code>”. Like most LLM generation though, it’s not a deterministic thing and like you mentioned originally it takes some verification of the output. I still think with the extra steps it saves time when applied to the right scenarios. The longer the input the higher the hallucinations count in my experience though, so I always keep the code provided in the smallest chunk possible which still has enough context.
• szundi 2y ago
  How can you expect this to find out what should have been the goal when this code was written without meticulous prompting?