4 ms·
You know there are other kinds of diffs, right? Its common to change git's diff to things like difftastic, so formatting slop doesn't trigger false diff lines.
by DiabloD3 9mo ago
You know there are other kinds of diffs, right?
Its common to change git's diff to things like difftastic, so formatting slop doesn't trigger false diff lines.
You're probably better off, FWIW, just avoiding LLMs. LLMs cannot produce working code, and they're the wrong tool for this. They're just predicting tokens around other tokens, they do not ascribe meaning to them, just statistical likelihood.
LLM weights themselves would be far more useful if we used them to indicate statistical likelihood (ie, perplexity) of the code that has been written; ie, strange looking code is likely to be buggy, but nobody has written this tool yet.
- nuky 9mo agoYeah difftastic and similar tools help a lot with formatting noise really. My question is slightly orthogonal though: even with a cleaner diff, I still find it hard to quickly tell whether public API or behavior changed, or whether logic just moved around. Not really about LLMs as reviewers — more about whether there are useful deterministic signals above line-level diff.
- veunes 9mo agoThe tools exist, they're just rarely used in web dev. Look into ApiDiff or tools using Tree-sitter to compare function signatures. In the Rust/Go ecosystem, there are tools that scream in CI if the public contract changes. We need to bring that rigor into everyday AI-assisted dev. A diff should say "Function X now accepts null", not "line 42 changed"
- nuky 9mo agoIt was precisely because this was going too far that I thought the consequences of the active adoption of LLM tools could be made visible. I'm not saying LLM is completely bad—after all, and not all tools, even non-LLM ones, are 100% deterministic. At the same time, reckless and uncontrolled use of LLM is increasingly gaining ground not only in coding but even in code analyze/review.