5 ms·

Indeed, but parsing has remained a hard (to implement properly) problem [1], and you likely don't want a language-agnostic tool like git to be tightly coupled w

Indeed, but parsing has remained a hard (to implement properly) problem [1], and you likely don't want a language-agnostic tool like git to be tightly coupled with a language parser. Maybe the popularity of language-servers can help bring this forward. [1] Even smart people have trouble with it: "Amazingly, surprisingly, counterintuitively, the indentation problem is almost totally orthogonal to parsing and syntax validation." http://steve-yegge.blogspot.com/2008/03/js2-mode-new-javascript-mode-for-emacs.html http://steve-yegge.blogspot.com/2008/03/js2-mode-new-javascr...

sdesol 5y ago
I was looking at https://github.com/github/semantic https://github.com/github/semantic for providing semantic diffs a while back and I still think it would be a good fit. By looking at the diff between trees, you can ignore a lot of the extra noise like indentations, spaces and other styling changes.
WorldMaker 5y ago
I found great results using syntax highlighter token streams for diffs. In my PoC I was using Pygments. It's a great compromise for "almost semantic" diffs. Syntax highlighting tokenizers have great language support, are blazing fast (we use them constantly in real time in IDEs), and work far better in "degenerate" cases that don't entirely parse/compile yet such as work-in-progress code (again because we use them all the time in text editors). https://github.com/WorldMaker/tokdiff https://github.com/WorldMaker/tokdiff
WalterGR 5y ago
> I ndeed, but parsing has remained a hard (to implement properly) problem Indeed. But there’s a perfect one (for some definition of perfect) built into every compiler. It’s a shame that more compilers don’t do things like make the AST available. .NET Compiler Platform (“Roslyn”) is a great counter-example.

5 ms·

Indeed, but parsing has remained a hard (to implement properly) problem [1], and you likely don't want a language-agnostic tool like git to be tightly coupled w

by AceJohnny2 5y ago

sdesol 5y ago
I was looking at https://github.com/github/semantic https://github.com/github/semantic for providing semantic diffs a while back and I still think it would be a good fit. By looking at the diff between trees, you can ignore a lot of the extra noise like indentations, spaces and other styling changes.
WorldMaker 5y ago
I found great results using syntax highlighter token streams for diffs. In my PoC I was using Pygments. It's a great compromise for "almost semantic" diffs. Syntax highlighting tokenizers have great language support, are blazing fast (we use them constantly in real time in IDEs), and work far better in "degenerate" cases that don't entirely parse/compile yet such as work-in-progress code (again because we use them all the time in text editors). https://github.com/WorldMaker/tokdiff https://github.com/WorldMaker/tokdiff
WalterGR 5y ago
> I ndeed, but parsing has remained a hard (to implement properly) problem Indeed. But there’s a perfect one (for some definition of perfect) built into every compiler. It’s a shame that more compilers don’t do things like make the AST available. .NET Compiler Platform (“Roslyn”) is a great counter-example.