4 ms·
I have a similar mandate and a similar take, but slightly different use cases. As to the search engine, my searches are often very narrow, like I want to recal
by bonzini 1y ago
I have a similar mandate and a similar take, but slightly different use cases.
As to the search engine, my searches are often very narrow, like I want to recall a specific message from a mailing list, so I don't use that too much. On the other hand, I found Google's NotebookLM to be really good at recalling concepts from both source code and manuals (e.g. processor manuals in my case).
Code generators are incredible refactoring machines. In one case (not so easy to reproduce in general, but it did work) Claude Code did a Python to decently idiomatic Rust conversion in a matter of minutes; it added mypy annotations to 2000 lines of Python code (with 90% accuracy) in half an hour and got the entire job done with my assistance in about an hour. For the actual writing and debugging where the logic matters they're still not there even for small code bases (again 2000 lines of code ballpark). They're relatively good at writing and debugging testcases but IMO that's also where there's a risk of copyright taint. Anyhow it's something I would use maybe 2-3 times a month.
In one case I used it for natural language translation, with pretty good results, but I knew both languages because I needed to check the result. Ask it first to develop a glossary and then to translate.
For studying they're interesting too, though for now I have mostly tried that outside work. At work, Google Deep Research worked well compared to the time it takes and it's able to find a variety of sources (including HackerNews comments in one case :)) which is useful for cross-checking.
- Neywiny 1y agoSo what does 90% accuracy mean here? Is this like you ran it through a linter or language server and 90% had errors? Or just through a quick glance you felt it was that accurate? I've found incorrect type hints to be one of the biggest issues when trying to use Python type-safely. Mostly (entirely?) with packages that get their own hints wrong, meaning methods aren't shown as existing within the class or the returned instance isn't the class it said it would be.
- bonzini 1y ago90% accuracy was just my own hunch but it is true that, out of 2000 lines, roughly 200 lines were changed, and there were roughly 20 mypy errors left after the first fully automatic pass. Most of these were due to code that did use duck typing, but it turned out that this was dead code and therefore it's understandable that the LLM got confused. Once the dead code was removed, the remaining 5 or so issues were annotations that were too loose in either variable or argument declarations. It would have taken at least a couple hours to do it by hand.