4 ms·
Writing unit tests used to be the bane of my existence. I used to hate them. Often times, the LoC for unit tests was 3X the LoC of the actual code. But not any
by mlmonkey 4mo ago
Writing unit tests used to be the bane of my existence. I used to hate them. Often times, the LoC for unit tests was 3X the LoC of the actual code.
But not any more! Now I point the LLM to the code and order it to write unit tests, covering all edge cases, etc. I'd rather spend 3 hours arguing with the LLM than writing unit tests! :-D
- dkn 4mo agoI am curious in your experience how often the LLM must also update the tests. I find that if LLMs write tests after the implementation exists, they are either extremely brittle because they are coupled to the implementation, or they cover little of value because they mock everything to the point of testing nothing.
- dcastm 4mo agoSame for me. I actively ask the LLM to write as few tests as possible. Otherwise you end up redundant and low value ttests.
- dkn 4mo agoYep, and wasted token spend on an ongoing basis. I instruct the LLM to follow TDD practices in certain areas, but otherwise prioritize integration style tests at the edges.
- mplanchard 4mo agoI have found a decent trick to be to write a parameterized test with e.g. a `cases` array that tests a function how you want it tested. Then ask the LLM to fill out more cases. It’s not perfect, but results in much less brittleness since you’ve already defined the specifics of what gets tested and what doesn’t.
- spaceclay 4mo ago[dead]
- aplomb1026 4mo ago[flagged]
- zerr 4mo agoSome companies (e.g. Microsoft) used to have "Software Engineers in Test" who's job was writing such tests all day long, so that those developers who were developing features wouldn't waste time on it.
- kovek 4mo agoI heard people say this before. I'm wondering, how do you instruct the LLM to generate the tests? Do you tell it the scenarios that would be covered, or do you just tell it to write tests for the code?
- pydry 4mo agoIf your tests align with the spec and the tooling is good it isnt tedious. If you find writing tests tedious enough to make using an LLM to write them seem like a good idea you're probably churning out repetitive tests, unnecessary tests, tests which aren't great at catching bugs.