4 ms·

Shameless plug: I've written difftastic[1], a tool that builds ASTs and then does a structural diff of them. You can use it with git too. It's an incredibly ha

Shameless plug: I've written difftastic[1], a tool that builds ASTs and then does a structural diff of them. You can use it with git too. It's an incredibly hard problem though, both from a computational complexity point of view, and trying to build a comprehensible UI once you've done the structural AST diff. [1]: https://github.com/wilfred/difftastic https://github.com/wilfred/difftastic

willow8349 3y ago
I think part of the problem is it seems everyone is trying to make a version control tool that is agnostic to all languages. Both computationally and UI wise. But C++ users expect to see different things than JavaScript users and so forth. I’m surprised at the lack of hyper-specific language version control tools. I thought about making a side project for one in Julia a while back but not quite sure how it would look. Some random thoughts: - info on type, name, constant changes - let me checkout older revisions of individual functions / objects / whatever - on unit test result changes for functions that have unit tests - when changes are simply a refactor and are functionally the same
- pastage 3y ago
  Every change is different in the same way every program is unique, the change of a couple of characters will alter the meaning. I think you have to try to write a diff UI to understand why it is hard. Difftastic, Meld, diff -u, Word and other tools are amazing because they are usefull in many scenarios. Getting the UI right has been a long process, beingable to grok the changes is still hard even with thw best tooling. It is also a question of tool adoption it takes a long time to understand how a tool works.
- nathan_compton 3y ago
  Can you imagine having to learn a bunch of different language specific version control tools? Sounds like a hassle to me.
  - stevage 3y ago
    No, because I only code in one language.
- mqus 3y ago
  Most repositories I work on don't have only one language. They have at the very least two, like the main language and maybe markdown for README files, then configuration like .ini or .toml, json stuff, yml, xml, etcpp. And then you might have bash scripts, Dockerfiles, other build tool languages, etcpp. And those are only text files. You probably will also have images, maybe zipped stuff, office documents and more, all not the "core" repository content, but stored nearby and versioned alongside. Building a hyper-focussed tool won't be very useful, expect to at least rudimentarily support other file types.
  - fanf2 3y ago
    Off topic, but “etcpp” is a new one for me. Wiktionary suggests it is German, which amuses me because I write “etc. usw.” to mean the same thing.
    - mqus 3y ago
      whoops :D thought it was more international than this, since its pretty much as latin as etc. Interesting!
      - jorams 3y ago
        I read it as a little joke referring to C++, for what it's worth.
      - maleldil 3y ago
        What's the difference between etcpp and etc?
        fanf2 3y ago
        The pp implies there are manny more things, as opposed to just some more things. https://en.m.wiktionary.org/wiki/etc._pp https://en.m.wiktionary.org/wiki/etc._pp.
        mqus 3y ago
        sibling is right, although for me its mostly a habit, I rarely use etc.
imiric 3y ago
Ah, yes, I knew I was forgetting one project. difftastic is very cool, thanks for writing it! How well do existing VCSs integrate with it? Did you feel restricted at any point by writing a diffing tool, instead of basing a new VCS around this concept? Do you think a deeper integration would allow supporting other functionality beyond diffing, like automatic merging, conflict resolution, etc.? I agree that it's a very difficult problem. But as an industry, we have more than enough smart people and resources to work on it, which if solved would greatly improve our collective QoL. I can't imagine the amount of time and effort we've wasted fighting with version control tools over the years, and a tool that solved these issues in a smarter way would make our lives much easier.
- eitland 3y ago
  For languages that have strong IDE refactoring support and userbases that use it a (future) solution would be for the ide to autocommit along the way with metadata to explain what happen "removed unused function based on suggestion", "extracted duplicate", "renamed public method taxed to isTaxed and updated usages across files x, y and z, developer comment: every other of these methods follow the pattern isSomething ". The last example also add a new feature, and option for a developer to add a comment on an automated refactor. Ordinary commits could exist on top of this as milestones. I wouldn't be totally surprised if sooner or later Jetbrains does this. They are creating their own, often better versions of everything I feel and version control could be an obvious next step. As someone who often prefers other solutions to theirs, I'd prefer if someone else does it first so I end up with something I can use across NetBeans, VS Code, eclipse etc and not something like Kotlin which forces me to use IntelliJ. (Don't get me wrong, IntelliJ is great, I just have NetBeans as my personal favorite.)
- Wilfred 3y ago
  > How well do existing VCSs integrate with it? Git supports external diffing tools really well with GIT_EXTERNAL_DIFF, which you can use with difftastic[1]. Other VCSs are less flexible. For example, I haven't found a nice way of getting a pager when using difftastic with mercurial. > Did you feel restricted at any point by writing a diffing tool, instead of basing a new VCS around this concept? Oh, that's an interesting question! Difftastic has been a really big project[2] despite its limited scope and I'm less interested in VCS implementation. I think text works well as the backing store for a VCS. There are a few systems that have structured backends (e.g. monticello for smalltalk), but they're more constrained. You can only store structured content (e.g. monticello requires smalltalk code) and it must be well-formed (your VCS must understand any future syntax you use). Unison[3] is a really interesting project in this space, it stores code by hash in a sqlite backend. This makes some code changes trivial, such as renames. From the perspective of a text diff, an AST diff is lossy. If you add an extra blank line between two unchanged functions, difftastic ignores it. That's great for understanding changes, but not for storage. [1]: https://difftastic.wilfred.me.uk/git.html https://difftastic.wilfred.me.uk/git.html [2]: https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fantastic-diff/ https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fan... [3] https://www.unison-lang.org/ https://www.unison-lang.org/
pravus 3y ago
I was hoping to see something like this. Thank-you!
selectnull 3y ago
I've been using difftastic as default diff for the past few months. Thank you. It works well, and the only time I switch back is when comparing long strings.

willow8349 3y ago

I think part of the problem is it seems everyone is trying to make a version control tool that is agnostic to all languages. Both computationally and UI wise. But C++ users expect to see different things than JavaScript users and so forth. I’m surprised at the lack of hyper-specific language version control tools. I thought about making a side project for one in Julia a while back but not quite sure how it would look. Some random thoughts: - info on type, name, constant changes - let me checkout older revisions of individual functions / objects / whatever - on unit test result changes for functions that have unit tests - when changes are simply a refactor and are functionally the same

pastage 3y ago
Every change is different in the same way every program is unique, the change of a couple of characters will alter the meaning. I think you have to try to write a diff UI to understand why it is hard. Difftastic, Meld, diff -u, Word and other tools are amazing because they are usefull in many scenarios. Getting the UI right has been a long process, beingable to grok the changes is still hard even with thw best tooling. It is also a question of tool adoption it takes a long time to understand how a tool works.
nathan_compton 3y ago
Can you imagine having to learn a bunch of different language specific version control tools? Sounds like a hassle to me.
- stevage 3y ago
  No, because I only code in one language.
mqus 3y ago
Most repositories I work on don't have only one language. They have at the very least two, like the main language and maybe markdown for README files, then configuration like .ini or .toml, json stuff, yml, xml, etcpp. And then you might have bash scripts, Dockerfiles, other build tool languages, etcpp. And those are only text files. You probably will also have images, maybe zipped stuff, office documents and more, all not the "core" repository content, but stored nearby and versioned alongside. Building a hyper-focussed tool won't be very useful, expect to at least rudimentarily support other file types.
- fanf2 3y ago
  Off topic, but “etcpp” is a new one for me. Wiktionary suggests it is German, which amuses me because I write “etc. usw.” to mean the same thing.
  - mqus 3y ago
    whoops :D thought it was more international than this, since its pretty much as latin as etc. Interesting!
    - jorams 3y ago
      I read it as a little joke referring to C++, for what it's worth.
    - maleldil 3y ago
      What's the difference between etcpp and etc?
      - fanf2 3y ago
        The pp implies there are manny more things, as opposed to just some more things. https://en.m.wiktionary.org/wiki/etc._pp https://en.m.wiktionary.org/wiki/etc._pp.
      - mqus 3y ago
        sibling is right, although for me its mostly a habit, I rarely use etc.

imiric 3y ago

Ah, yes, I knew I was forgetting one project. difftastic is very cool, thanks for writing it! How well do existing VCSs integrate with it? Did you feel restricted at any point by writing a diffing tool, instead of basing a new VCS around this concept? Do you think a deeper integration would allow supporting other functionality beyond diffing, like automatic merging, conflict resolution, etc.? I agree that it's a very difficult problem. But as an industry, we have more than enough smart people and resources to work on it, which if solved would greatly improve our collective QoL. I can't imagine the amount of time and effort we've wasted fighting with version control tools over the years, and a tool that solved these issues in a smarter way would make our lives much easier.

eitland 3y ago
For languages that have strong IDE refactoring support and userbases that use it a (future) solution would be for the ide to autocommit along the way with metadata to explain what happen "removed unused function based on suggestion", "extracted duplicate", "renamed public method taxed to isTaxed and updated usages across files x, y and z, developer comment: every other of these methods follow the pattern isSomething ". The last example also add a new feature, and option for a developer to add a comment on an automated refactor. Ordinary commits could exist on top of this as milestones. I wouldn't be totally surprised if sooner or later Jetbrains does this. They are creating their own, often better versions of everything I feel and version control could be an obvious next step. As someone who often prefers other solutions to theirs, I'd prefer if someone else does it first so I end up with something I can use across NetBeans, VS Code, eclipse etc and not something like Kotlin which forces me to use IntelliJ. (Don't get me wrong, IntelliJ is great, I just have NetBeans as my personal favorite.)
Wilfred 3y ago
> How well do existing VCSs integrate with it? Git supports external diffing tools really well with GIT_EXTERNAL_DIFF, which you can use with difftastic[1]. Other VCSs are less flexible. For example, I haven't found a nice way of getting a pager when using difftastic with mercurial. > Did you feel restricted at any point by writing a diffing tool, instead of basing a new VCS around this concept? Oh, that's an interesting question! Difftastic has been a really big project[2] despite its limited scope and I'm less interested in VCS implementation. I think text works well as the backing store for a VCS. There are a few systems that have structured backends (e.g. monticello for smalltalk), but they're more constrained. You can only store structured content (e.g. monticello requires smalltalk code) and it must be well-formed (your VCS must understand any future syntax you use). Unison[3] is a really interesting project in this space, it stores code by hash in a sqlite backend. This makes some code changes trivial, such as renames. From the perspective of a text diff, an AST diff is lossy. If you add an extra blank line between two unchanged functions, difftastic ignores it. That's great for understanding changes, but not for storage. [1]: https://difftastic.wilfred.me.uk/git.html https://difftastic.wilfred.me.uk/git.html [2]: https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fantastic-diff/ https://www.wilfred.me.uk/blog/2022/09/06/difftastic-the-fan... [3] https://www.unison-lang.org/ https://www.unison-lang.org/

pravus 3y ago

I was hoping to see something like this. Thank-you!

selectnull 3y ago

I've been using difftastic as default diff for the past few months. Thank you. It works well, and the only time I switch back is when comparing long strings.