4 ms·

> "no no, it has full test coverage" i don't have enough fingers (and toes) to count how many times i've demonstrated that "100% coverage" is almost universall

• kevinsync 5mo ago
  Codex is freakin hot-to-trot to churn out test coverage for every single thing it implements, and some of it is very esoteric and highly prescriptive (regexes for days) BUT .. after a while, it dawned on me that LLM-driven test coverage is less about proving “code correctness” (you’re better off writing those tests yourself alongside them), and more about just trying to ensure that whatever gets bolted on stays bolted on. For better or worse, obviously, since if you bolt on trash, trash you shall have.
• throw310822 5mo ago
  Wholeheartedly agree, but in fairness, I trust the tests of the best AI models more than those of the average human developer. There's a lot of people around that combine high diligence with complete intellectual laziness, producing tons of useless tests. Actually no, cancel that. I realise now that I trust AIs more than the average developer, period. At this point they do produce better code than most people I've dealt with.
• GrumpyYoungMan 5mo ago
  There's a very old paper by Cem Kaner about the meaninglessness of "100% coverage" where he included an appendix where he enumerates 101 different possible types of code coverage: https://www.researchgate.net/publication/243782285_Software_Negligence_Testing_Coverage https://www.researchgate.net/publication/243782285_Software_...