3 ms·

I like this approach. After some months of gamely trying to involve Claude in my coding process, I find it's much more enjoyable and efficient for me to just wr

I like this approach. After some months of gamely trying to involve Claude in my coding process, I find it's much more enjoyable and efficient for me to just write the code myself, rather than babysitting it and going over and over something it wrote finding errors and logical flaws. I was about to cancel my subscription. This month, however, I had to start upgrading a very large db from MySQL 8 to 9, on a codebase that involves a few hundred long and complex queries. I wrote this code, so I know what they do and I understand the schema, but I'm also aware that some of them may violate ONLY_FULL_GROUP_BY, and a few (but can't remember which) had historical oddities unless DERIVED_MERGE was turned off. I was girding myself to run tests on dozens of suspects, but as a lark I handed Claude the whole codebase and the schema and asked it to identify any queries that might break if those two default options were enabled. Astonishingly, it did. It identified 15 queries and explained what they violated. It was wrong about 7 of them... those seven were actually fine, and it acknowledged that it had farmed this out in some way and that whatever it used for evaluation had been overly conservative and had returned false positives. But the other 8 queries were fixable. It was also a bit off about how to rewrite them - it created a lot of slow and inefficient workarounds. It neglected the idea of lateral joins, it didn't write SQL optimized for the indexes, etc. But it saved me hours and hours of locating these queries myself. To me, this idea of letting the LLM render down a lot of code into potential pain points seems like a much more valuable use of it than asking it to write code itself. I kept my subscription for that reason.

fibonachos 1y ago
This has been saving me a lot of time as well in a decade old code base. I can paste a stack trace and provide additional relevant context, then ask the LLM to do a first pass debug. From that I usually get a list of file+lines to manually review, along with some initial leads to chase. Another use case is when fixing performance issues. I can feature flag my fix and ask the model to confirm the new code path will produce the same result for a given set of inputs. We also have test coverage for this kind of thing, but the LLM can do a once-over and point out some flaws before I ever run those tests. I haven’t gotten to the point where it writes much code for me beyond the auto-complete, which has been a modest boost in efficiency.
- noduerme 1y ago
  Yeah. As a debugging aid, I think it's fairly solid at surfacing things to look at and fix manually. And when you do that, you're actually hoping for more false positives than false negatives - which plays to the strengths of an LLM. When it comes to asking for rewrite suggestions for anything, I have to really go over its logic with a fine-tooth comb, because there are usually edge cases that can be spotted if you really think through it. I abhor its tendency to use try/catch. I've seen it write weird SQL joins that slow down queries by 30x. I'd never trust it to debug a race condition or to consider any side effects outside the 30 LoC it's currently looking at. I guess I wouldn't trust it to confirm that new code would give the same result, but it can't hurt to ask, since if it told me the code wouldn't, that would make me look more closely at it. I think as long as you look at it as part of a distillation process, and aim for false positives, and never actually trust it, it's good at helping to surface issues you may have missed.

fibonachos 1y ago

This has been saving me a lot of time as well in a decade old code base. I can paste a stack trace and provide additional relevant context, then ask the LLM to do a first pass debug. From that I usually get a list of file+lines to manually review, along with some initial leads to chase. Another use case is when fixing performance issues. I can feature flag my fix and ask the model to confirm the new code path will produce the same result for a given set of inputs. We also have test coverage for this kind of thing, but the LLM can do a once-over and point out some flaws before I ever run those tests. I haven’t gotten to the point where it writes much code for me beyond the auto-complete, which has been a modest boost in efficiency.

noduerme 1y ago
Yeah. As a debugging aid, I think it's fairly solid at surfacing things to look at and fix manually. And when you do that, you're actually hoping for more false positives than false negatives - which plays to the strengths of an LLM. When it comes to asking for rewrite suggestions for anything, I have to really go over its logic with a fine-tooth comb, because there are usually edge cases that can be spotted if you really think through it. I abhor its tendency to use try/catch. I've seen it write weird SQL joins that slow down queries by 30x. I'd never trust it to debug a race condition or to consider any side effects outside the 30 LoC it's currently looking at. I guess I wouldn't trust it to confirm that new code would give the same result, but it can't hurt to ask, since if it told me the code wouldn't, that would make me look more closely at it. I think as long as you look at it as part of a distillation process, and aim for false positives, and never actually trust it, it's good at helping to surface issues you may have missed.