3 ms·
Instead of relying only on reviews, rely on tests. You can have an LLM generate tests first (yes, needs reviewing) and then have the LLM generate code until all
by ako 1y ago
Instead of relying only on reviews, rely on tests. You can have an LLM generate tests first (yes, needs reviewing) and then have the LLM generate code until all tests work. This will also help with non deterministic challenges, as it either works or it doesn’t.
- djoldman 1y agoThis. Tests are important and they're about to become overwhelmingly important. The ability to formalize and specify the desired functionality and output will become the essential job of the programmer.
- sn9 1y agoI'm shocked how many people haven't yet realized how important formal methods are about to be. If you can formally specify what you need and prove that an LLM has produced something that meets the spec, that's a much higher level of confidence than hoping you have complete test coverage (possibly from LLM generated tests).
- deleted 1y ago[deleted]
- vacuity 1y agoThere is a critical distinction between tests and formal specification/verification. It is not sufficient to make a bunch of tests if you want to ensure behavior. Formal methods have long been recognized. If a programmer is only now realizing their necessity due to LLM code synthesis, I do not trust the programmer with human-generated code, let alone LLM-generated code. I don't expect everyone to formally verify all code for many reasons, but the principles should always be present for any program more serious than a hobby project. Take a look at [0]. Cautious design is needed at all levels; if tests or formal methods are relied upon, they count. [0] https://news.ycombinator.com/item?id=43818169 https://news.ycombinator.com/item?id=43818169