3 ms·

Some people have the agent hallucinating its own meaningless tests

• CuriouslyC 2mo ago
  Hence the advice to read the tests? Interesting how people make the argument that AI code is garbage and can't be trusted (it's inelegant and poorly structured, sure, but that's a different matter), and how the only option isn't to just spot check the work, but to laboriously write it by hand.
  • unknownfuture 2mo ago
    > Hence the advice to read the tests? Reading tests and actually catching issues requires a heck of a lot of effort. IME code reviews of tests are often more laborious than reviewing the code itself since so much if it is reasoning about corner cases. I'm not saying don't do it, but the LOE is high. Done well I'd argue it runs close to the effort involved in just authoring those tests by hand. And that's ignoring that the "vibe all the things" crowd is explicitly telling people not to read the code. At all. (And if you think I'm exaggerating: an exec where I work decided to own rebuilding one of our services. They told me how recently the LLM generated multiple thousands of lines of code and they pushed it with minimal review, figuring we'll just fix the bugs as they happen...).
• abirch 2mo ago
  If people misuse or don't understand a tool, that doesn't invalidate the usefulness of the tool.