49 ms·
How far should a programming language aware diff go?
- joe_the_user 2y agoIt's a very interesting question. One idea I've toyed with over the years is a language specifically designed to facilitate effective diffs. Anyway, it seems the "Level 3: semantic diff" actually could be divided into different levels. But "Level 4: Mostly identical" seems quite problematic.
- jmull 2y agoI think the general answer is, it depends. Hopefully this tool gives a dev ready control of what kinds of differences to hide/show. I'm actually not convinced of the concept of semantic diff (not talking just about this tool specifically)... when we talk about code that is different but equivalent, I think we're talking about elements of style. It seems to me that it would pretty much always be better to normalize the elements of style considered insignificant, rather than hide them just in the diff tool. That covers diffing as well as viewing/reading the code. If you don't care about a particular element of style then either it shouldn't be coming up much or I think you'd be better off using some kind of enforcing/fixing linter.
- rty32 2y agoIn theory semantic diff is useful, but based on my code review experience, it hardly matters. For a language like Python or JavaScript, a developer fluent in these languages don't really pay much attention to these things anyway, just like you don't normally pay much attention to commas and periods in a sentence unless it causes confusion. Personally I wouldn't pay $5/month out of the pocket for this functionality.
- xg15 2y agoI think I'd appreciate some sort of "semantic grouping" of individual changes more than drawing someone random line and classifying all changes below it as "trivial". The problem is that even a lot of the changes that normally constitute clutter can become relevant in certain situations or even introduce bugs. One example would be ordering of Python imports: Changing the order of imports should have no effect on program behaviour if all your packages are well-behaved - and in 99.99% of cases it indeed hasn't. But the fact remains that imports are statements that are executed and can have side-effects. If a package does something nontrivial during load, changing the import order can have effects. Hiding such a change could mask introduction of a bug. Hiding changes can also lead to confusion if you are trying to understand a series of changes that are based on each other, or if all changes of a commit are hidden. I've had the latter situation with IntelliJ, where the working tree was shown as "unclean" but the diff was completely empty. Solution: The diff wasn't actually empty, IntelliJ was just set to hide the changes. I think a more interesting solution would be to build a sort of "tree of changes": At the bottom, you'd have the individual changes in the file; one level up, the changes would be grouped into higher-level operations, such as "change formatting", "rename identifier", "remove field", "move function", etc. If possible, those could be grouped into even higher-level changes, such as "implement new class" or "extract expression into function", etc.
- blackenedgem 2y agoI think the problem you'll eventually run into is figuring out intent from the diff. It seems like an easier version of reverse compiling. When it comes down to semantic diffs I'm more interested in something like the Semantic Patch Language by Coccinelle. Being able to represent mundane refactorings across an entire codebase in a few lines seems great. And it unifies intent with the diff.
- golergka 2y agoAnd just like that, another GPT-4 wrapper startup was born.
- tsimionescu 2y agoAgreed, I don't think the value of a semantic diff would be in hiding changes. Instead, the value should be in generating more useful diffs. Normal diff often gets "confused" compared to how you'd logically identify the code. For example, if you extract a piece of a larger function as a smaller function, instead of showing that a piece of code was moved, it will show that you changed a header, deleted some lines, added others below, etc. A semantic diff should be able to refine these diffs in a better way, but shouldn't hide them. Even for the whitespace changes, I'd like it to show the diff, but the overlay to explain that only whitespace is different, so I know I don't need to look at it carefully.
- whirlwin 2y agoAfter switching to difftastic for semantic diff, I have never looked back. (https://github.com/Wilfred/difftastic https://github.com/Wilfred/difftastic) How does semanticdiff compare to that? Anyone got experience?
- DarkPlayer 2y agoYou can find a comparison of the two tools here: https://semanticdiff.com/blog/semanticdiff-vs-difftastic/ https://semanticdiff.com/blog/semanticdiff-vs-difftastic/ As author of SemanticDiff, I am obviously a bit biased. But Wilfred, the author of difftastic, found the analysis to be "pretty even-handed" [1], so I think it should be somewhat fair. [1]: https://x.com/_wilfredh/status/1764424652611318146 https://x.com/_wilfredh/status/1764424652611318146
- zokier 2y agoI think this question has been already largely been answered by automatic style (etc) tools. Such tools generally should not make semantic changes to programs, so they (implicitly) define what are meaningful semantic changes and what are meaningless changes.
- emporas 2y agoThere is also diffsitter. I was testing it a month ago, it works fine. Not sure what language-aware diffing exactly means, but diffsitter uses tree-sitter and it is comparing ASTs and CSTs of the files. [1] https://github.com/afnanenayet/diffsitter https://github.com/afnanenayet/diffsitter
- sureglymop 2y agoSeems to be the best choice since treesitter is already generic and supports many languages. Will try this out first.
- emporas 2y agotree-sitter is very generic and supports so many languages, it is really great. The first use case of the article, "Level 1: Irrelevant Whitespace" is covered by diffsitter. I wanted at some point, to diff files and ignore comments for Rust source code. I wrote a small program, to remove the two different comment nodes the Rust grammar defines: line_comments and block_comments. Then i diffed the resulting uncommented code using diffsitter. From start to finish, writing the program and testing it to many different files it took 5 hours.
- MathMonkeyMan 2y agoI haven't actually checked the source, but I've heard that clang-format works by assigning "badness" weights to each choice of whitespace between tokens, and then runs Dijkstra's (or some other DP) to find the least bad set of choices. A recent Tom7 video said that Knuth did the same thing for text justification. How about we do a similar thing for ASTs? Like a peephole optimizer looking for runs of instructions that could be substituted for simpler alternatives, a tree diff could identify diff patterns that "might be trivial." You have a whole catalog of these patterns, and assign to each a weight. Then the displayed diff is the optimal set of choices "consider different or not?" You would need some additional ingredient, though; some boundary condition. Otherwise "everything is the same" would always minimize badness.
- amelius 2y agohttps://en.wikipedia.org/wiki/Edit_distance https://en.wikipedia.org/wiki/Edit_distance
- ckdot2 2y agoNot far. Just show all changes. Like the blog article already states, for many projects you already have code formatters, so changes in format usually don’t happen a lot - and if they do there might be a reason you don’t want to hide (like… you change your rules of code formatting). For all the other example I neither see the point why you would want to hide it. If you don’t want to see commas added in a list, make it a rule that the comma always has to be appended after the last element. Most languages allow that. Semantic equivalence? The JS example isn’t even equivalent because „this“ may have a different context. I’d prefer to have a „dumb“ diff that simply shows all the changes instead of adding these kind of complexities. Just keep your MRs small and there’s no real issue.
- HelloNurse 2y agoThe healthy workflow is: notice formatting discrepancies -> reformat -> reopen the diff, now containing only intentional, substantial changes. Of course the edited source files should have been reformatted automatically, on save or on build, before someone opens a diff: this should never happen except as a symptom of inadequate reformatting (e.g. I decide to adopt redundant commas at the end of comma-separated lists) or abnormal operations (e.g. non-reformatted code was accidentally committed to version control).
- philipwhiuk 2y agoI think a slider on the code review would be nice. That way I could start at the 'definitely a change' stuff and then slide down towards L2 until I decided it was fine.
- diffxx 2y ago- def foo(): int | None + def foo(): None | int Whether or not this makes a semantic difference is language implementation dependent. I think that is why this kind of tool is not especially appealing to me. I would have to have almost complete knowledge of the compiler and the diff tool to truly trust that there is no semantic difference. Moreover, I would like to know why changes to the text that are being made that have no semantic effect are being mixed with those that do. For me, text is king and that is the level at which I want to evaluate diffs 99% of the time, but I do recognize that others have different goals and preferences.
- shaftway 2y agoPersonally, I really don't care about these cases. What really grinds my gears is when a diff plucks out a weird line in the middle of a block of code that only has a closing curly brace and that's the line that it thinks is the same, and everything around it is a diff. If you're going to call yourself a semantic diff-ing company, fix that before you worry about the order of my imports.
- deleted 2y ago[deleted]
- fragmede 2y agoI'm not sure how they wrote up that whole article without bringing up looking at the AST representation of the code being diffed.
- bmitc 2y agoAs far as possible. Git's line-based diffing is ridiculously primitive and gets in the way of software development. I wonder how many bugs are introduced because of Git's diffing system.
- aaron695 2y ago[dead]
- firethief 2y agoInteresting idea. I've just tried it with a couple of languages: - TS with Vue: SFC are not really working (it's showing a style change as if the whole stylesheet were replaced with a mostly-identical stylesheet). - Rust: It doesn't seem semantic at all. It's showing a lot of character-level insertions and deletions that seem worse than how git-diff or GitHub would break down the changes. It doesn't seem ready yet for what I'd like to use it for.
- DarkPlayer 2y agoHi, author of SemanticDiff here. I'm sorry you didn't have a good experience testing the tool. If it doesn't work / makes things worse than a standard diff, that's definitely considered a bug. It is probably something specific to your code and not a general issue. It would therefore be great if you could open an issue [1] or support ticket [2], ideally with some sample code, so we can take a look. Thanks in advance! [1] https://github.com/Sysmagine/SemanticDiff/issues https://github.com/Sysmagine/SemanticDiff/issues [2] support@semanticdiff.com
- PullJosh 2y agoI was expecting this to refer to different ways to represent the same diff. (For example, you could represent a change from `console.log(“hello”)` as `console.log('hello')` as +'-“ … +’-“ or as +'hello'-“hello”) I don’t have a specific example in mind, but it seems reasonable that different languages could benefit from different ways of representing the same diff.
- kmoser 2y ago> - const foo = function(a, b) { ... } > + const foo = (a, b) => { ... } Assuming this is JS code, these differences should not be ignored, as an arrow function can behave differently than a traditional function.
- olliej 2y agoYeah I came to say that these are not semantically equivalent (I guess you _could_ verify equivalence if you ensured it did not use this or eval)
- kqr 2y agoYeah, that was a bit of an unfortunate example for a blogvertising post. Even I as a non-frontend developer knew to watch out for that one. A company working with semantic diffs should really know better and such mistakes do not inspire confidence for me!
- persnickety 2y agoSame goes for the loop example, where the i variable has a different scope. I think those are meant to be illustrative of an idea rather than exact examples.
- kqr 2y agoBut if they have worked in this field for a few months, shouldn't they have a crapton of exact examples to draw from? If not, isn't that worrying?
- persnickety 2y agoDepends on how technical the audience of this post should be. Putting an accurate example which is only understandable to someone with years of experience might make newbies think it's a made up concern.
- culi 2y agoMore specifically, the `function` keyword version of an anonymous function preserves the keyword `this` whilst the arrow syntax anonymous function does not. Arrow functions also cannot use the `yield` keyword nor be used as constructors https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions/Arrow_functions#call_apply_and_bind https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
- g4zj 2y agoI've never knowingly used a language-aware diff tool before, but I wouldn't mind the option. I think it would come in handy on occasion.
- Jabbles 2y agonit: the order of Go's imports makes no difference: https://go.dev/ref/spec#Program_initialization_and_execution:~:text=sorted%20by%20import%20path https://go.dev/ref/spec#Program_initialization_and_execution...
- twic 2y agoAccurately identifying whether any change is a semantic difference involves solving the Halting Problem, right?
- Smaug123 2y agoFortunately the article already says that hiding all semantically identical changes is "probably going too far", so they can just not try and solve the halting problem.
- rob74 2y agoI think I have heard of their product before, and reading the blog post intrigued me, so I wanted to try it, but... VS Code Integration? GitHub Integration? No standalone version which you could actually use as a diff tool for git locally? Ok, I guess only having a "cloud" version makes licensing easier, and you can call me old fashioned, but seeing an eminently "offline" task such as diff being turned into "online-only" seems a bit strange to me.
- DarkPlayer 2y agoThe VS Code extension works offline. The diff calculation is performed on the host where the VS Code GUI is running (makes a difference in case of SSH/Docker/WSL).