7 ms·

Reading this article, I feel as though the author doesn't deeply understand git. git works on blobs of data, not files, and not lines of text. It doesn't just

by mabbo 5y ago

Reading this article, I feel as though the author doesn't deeply understand git. git works on blobs of data, not files, and not lines of text. It doesn't just happen to also work on binary files- that's all it works on. Now, if the author is suggesting that git-diff ought to have a language specific mode that parses changed files as ASTs to compare, now I'm interested. Let's do that. I'll help! But git does not need to change how it works for that to happen. Git does not even need git-diff to exist to serve it's main purpose.

dboreham 5y ago
Pretty sure OP does understand, and is proposing what you deduced. Incorporate some semantic understanding of the version controlled data into the VCS. Currently this work is subcontracted to humans.
- mabbo 5y ago
  Maybe I'm misunderstanding. It's just lines like this: > The text-orientated design of git reflects... > The current version of git is also able to find differences in binary files. > if we were storing information as ASTs, rather than lines of text These all, to me, show a gap in the authors understanding of how git works. And that's okay- git is often easier to use than is to understand. But if they had a better understanding, they could make their point far better. And without understanding, they won't be able to implement this idea.
mbauman 5y ago
You can already choose different `diff` programs to use for particular filetypes. E.g., nbdime for Jupiter notebooks: https://nbdime.readthedocs.io/en/latest/vcs.html#git-integration https://nbdime.readthedocs.io/en/latest/vcs.html#git-integra...
hardwaregeek 5y ago
I dunno I feel like you're focusing on a detail that's not particularly relevant. The author's main thrust is precisely what you described about parsing changed files as ASTs.
- nemetroid 5y ago
  It isn't relevant to the author's vision of content-aware diffing, but it is relevant to the author's complaints about how Git's (alleged) text-based-ness makes Git awkward to use with Jupyter notebooks. Has the author tried searching the web for "git diff jupyter"?
tux3 5y ago
Note that git does work with diffs a lot. Rebases and cherry-picks work by applying diffs, not by copying blobs. Auto-merging also needs to look at file content as text, you can't auto-merge a binary file with git. It's an often repeated fact that if you look inside Git, it doesn't work with diffs, it works with blobs. But if you look closer, it's often diffs again!
- arghwhat 5y ago
  With cherry-picks (and thus rebase), you ask git to turn a commit into a patch, so it does just that. I would mostly consider auto merges (which I guess are bolted on) as the main area where git itself uses diffs during resolution and even then only as a suggested resolution (you get warned and need to confirm it when validating the merge). So no, it's blobs all the way down. Darcs and Pijul are patch based though.
  - tux3 5y ago
    It's true that git is blob based, as opposed to patch based, but it's not the full picture! In practice, git stores a lot more diffs (or rather, deltas) than it stores loose blobs. (And you probably know this already, but I feel it's still worth making explicit) This is necessary, because when a repo accumulates commits, it becomes a lot more efficient to store most of the objects as deltas instead of separate blobs. If Git didn't do this, it would have a lot of copies, and they would take a lot of space. So the fundamental model of git is truly based on blobs in theory, but in practice many or most git commands will operate on packfiles, and if you look in your .git object store, most likely you will have a few big packfiles containing most objects, and then a much smaller collection of loose blobs. All those diffs are what the "resolving deltas" progress indicator that people see when they do a big clone, fetch, or checkout is about =)
    - ori_b 5y ago
      > In practice, git stores a lot more diffs (or rather, deltas) than it stores loose blobs. The diffs it stores are not the diffs you see in git diff. They're rolling checksum based chunks. The data that the delta is computed against is picked with a heuristic ("sort by name and date, try the top 10, and use the smallest result"). And, in practice, the heuristic diffs the older files against the newer ones, rather than diffing in chronological order, so that getting recent data doesn't involve a lot of delta application. The git deltification is better thought of as a compression method than as diffing.
screye 5y ago
The git extension on VSCode is already pretty good at doing diffs on jupyter notebooks. I distinctly remember this not being a core feature of stock git and needing Jupytext to enable version control on notebooks. So, I feel like this sort of language specific stuff is already happening, but not in any unified product.
munificent 5y ago
The author is likely using "git" to mean "the entire typical git user experience that git users spend time looking at". And, from that perspective, Git-the-UX definitely does work on line-oriented files.
munk-a 5y ago
There's also a historical angle here that's important to inspect - Git was designed to specifically be content agnostic. There are some predecessors in the SCM space (like VSS) that are specifically language aware and allow the checking out of line ranges (pinning them so that no one else will make a conflicting change specifically) and even entire functions - these systems can cause a lot of grief while failing to protect the logic they're specifically trying to protect. As the warts on SVN got more and more visible I think the general assumption was that the replacement SCM would come out of this code aware space - but it didn't and in retrospect we all dodged a huge bullet when that happened. I absolutely adore tooling around git that makes diffs more visible - one thing I absolutely gush over is anything that can detect and highlight function reordering... however, the core process of merging and rebasing and all that jazz - I don't think we're going to find anything automated that I'll ever trust when I'm not working on a ridiculously clean codebase - minor changes can have echo effects and when two people are coding in the same general area they need to be aware of what the other person is trying to do.
tenaciousDaniel 5y ago
I'm in the process of building a programming language for UI designers, and realized that diffing the AST (or some other kind of object notation) would be far more useful and understandable. I'll probably be digging into this exact problem within the next year or so.
ggleason 5y ago
My two cents: https://github.com/GavinMendelGleason/syntactic_versioning/blob/master/IS_IT_TEXT.md https://github.com/GavinMendelGleason/syntactic_versioning/b...

by mabbo 5y ago

dboreham 5y ago

Pretty sure OP does understand, and is proposing what you deduced. Incorporate some semantic understanding of the version controlled data into the VCS. Currently this work is subcontracted to humans.

mabbo 5y ago
Maybe I'm misunderstanding. It's just lines like this: > The text-orientated design of git reflects... > The current version of git is also able to find differences in binary files. > if we were storing information as ASTs, rather than lines of text These all, to me, show a gap in the authors understanding of how git works. And that's okay- git is often easier to use than is to understand. But if they had a better understanding, they could make their point far better. And without understanding, they won't be able to implement this idea.

mbauman 5y ago

You can already choose different `diff` programs to use for particular filetypes. E.g., nbdime for Jupiter notebooks: https://nbdime.readthedocs.io/en/latest/vcs.html#git-integration https://nbdime.readthedocs.io/en/latest/vcs.html#git-integra...

hardwaregeek 5y ago

I dunno I feel like you're focusing on a detail that's not particularly relevant. The author's main thrust is precisely what you described about parsing changed files as ASTs.

nemetroid 5y ago
It isn't relevant to the author's vision of content-aware diffing, but it is relevant to the author's complaints about how Git's (alleged) text-based-ness makes Git awkward to use with Jupyter notebooks. Has the author tried searching the web for "git diff jupyter"?

tux3 5y ago

Note that git does work with diffs a lot. Rebases and cherry-picks work by applying diffs, not by copying blobs. Auto-merging also needs to look at file content as text, you can't auto-merge a binary file with git. It's an often repeated fact that if you look inside Git, it doesn't work with diffs, it works with blobs. But if you look closer, it's often diffs again!

arghwhat 5y ago
With cherry-picks (and thus rebase), you ask git to turn a commit into a patch, so it does just that. I would mostly consider auto merges (which I guess are bolted on) as the main area where git itself uses diffs during resolution and even then only as a suggested resolution (you get warned and need to confirm it when validating the merge). So no, it's blobs all the way down. Darcs and Pijul are patch based though.
- tux3 5y ago
  It's true that git is blob based, as opposed to patch based, but it's not the full picture! In practice, git stores a lot more diffs (or rather, deltas) than it stores loose blobs. (And you probably know this already, but I feel it's still worth making explicit) This is necessary, because when a repo accumulates commits, it becomes a lot more efficient to store most of the objects as deltas instead of separate blobs. If Git didn't do this, it would have a lot of copies, and they would take a lot of space. So the fundamental model of git is truly based on blobs in theory, but in practice many or most git commands will operate on packfiles, and if you look in your .git object store, most likely you will have a few big packfiles containing most objects, and then a much smaller collection of loose blobs. All those diffs are what the "resolving deltas" progress indicator that people see when they do a big clone, fetch, or checkout is about =)
  - ori_b 5y ago
    > In practice, git stores a lot more diffs (or rather, deltas) than it stores loose blobs. The diffs it stores are not the diffs you see in git diff. They're rolling checksum based chunks. The data that the delta is computed against is picked with a heuristic ("sort by name and date, try the top 10, and use the smallest result"). And, in practice, the heuristic diffs the older files against the newer ones, rather than diffing in chronological order, so that getting recent data doesn't involve a lot of delta application. The git deltification is better thought of as a compression method than as diffing.

screye 5y ago

The git extension on VSCode is already pretty good at doing diffs on jupyter notebooks. I distinctly remember this not being a core feature of stock git and needing Jupytext to enable version control on notebooks. So, I feel like this sort of language specific stuff is already happening, but not in any unified product.

munificent 5y ago

The author is likely using "git" to mean "the entire typical git user experience that git users spend time looking at". And, from that perspective, Git-the-UX definitely does work on line-oriented files.

munk-a 5y ago

There's also a historical angle here that's important to inspect - Git was designed to specifically be content agnostic. There are some predecessors in the SCM space (like VSS) that are specifically language aware and allow the checking out of line ranges (pinning them so that no one else will make a conflicting change specifically) and even entire functions - these systems can cause a lot of grief while failing to protect the logic they're specifically trying to protect. As the warts on SVN got more and more visible I think the general assumption was that the replacement SCM would come out of this code aware space - but it didn't and in retrospect we all dodged a huge bullet when that happened. I absolutely adore tooling around git that makes diffs more visible - one thing I absolutely gush over is anything that can detect and highlight function reordering... however, the core process of merging and rebasing and all that jazz - I don't think we're going to find anything automated that I'll ever trust when I'm not working on a ridiculously clean codebase - minor changes can have echo effects and when two people are coding in the same general area they need to be aware of what the other person is trying to do.

tenaciousDaniel 5y ago

I'm in the process of building a programming language for UI designers, and realized that diffing the AST (or some other kind of object notation) would be far more useful and understandable. I'll probably be digging into this exact problem within the next year or so.

ggleason 5y ago

My two cents: https://github.com/GavinMendelGleason/syntactic_versioning/blob/master/IS_IT_TEXT.md https://github.com/GavinMendelGleason/syntactic_versioning/b...