4 ms·

So what does 90% accuracy mean here? Is this like you ran it through a linter or language server and 90% had errors? Or just through a quick glance you felt it

So what does 90% accuracy mean here? Is this like you ran it through a linter or language server and 90% had errors? Or just through a quick glance you felt it was that accurate? I've found incorrect type hints to be one of the biggest issues when trying to use Python type-safely. Mostly (entirely?) with packages that get their own hints wrong, meaning methods aren't shown as existing within the class or the returned instance isn't the class it said it would be.

bonzini 1y ago
90% accuracy was just my own hunch but it is true that, out of 2000 lines, roughly 200 lines were changed, and there were roughly 20 mypy errors left after the first fully automatic pass. Most of these were due to code that did use duck typing, but it turned out that this was dead code and therefore it's understandable that the LLM got confused. Once the dead code was removed, the remaining 5 or so issues were annotations that were too loose in either variable or argument declarations. It would have taken at least a couple hours to do it by hand.