7 ms·

This blog post is the most compelling argument I've yet seen for pijul. Git should work the way we think it does! It's confusing that snapshots are being conve

This blog post is the most compelling argument I've yet seen for pijul. Git should work the way we think it does! It's confusing that snapshots are being converted into a few different forms of change object, which can be reconciled with merges or rebases or applying patches. Pijul (and darcs before it) actually works on the basis of patches, pijul with a robust theory of patches. A cherry-pick just moves a patch from one history-of-patches (branch) to another history-of-patches. One can share just a patch, and applying it is guaranteed to be the same action everywhere if that's possible, which it often is. I'm patiently waiting for pijul to be mature enough that I can move everything over to using it, it's one of the more exciting projects in the last ten years.

jayd16 6y ago
Can I make a shallow clone in Pijul?
- pmeunier 6y ago
  I'm one of the authors. The concepts are different, we do have a concept called "tags" (not ready yet) which is more efficient than shallow clones, in the sense that you can make patches (commits) on top of a shallow clone without any down side (unlike in Git, where shallow repositories can become slower). You can also do partial clones in Pijul: since patches commute, the patches you produce on top of a partial clone can be pushed to the full repository in the exact same way.
cmeacham98 6y ago
No idea what Pijul is, but how does this not describe git? Unless your complaint is that a commit is really a set of diffs/patches?
- chriswarbo 6y ago
  Pijul (and Darcs) operate on sets of patches. As a simple example, git commits have at least one 'parent', which imposes an order, e.g. let's say I edit file X in commit x and file Y in commit y; if I want both of those changes, git forces me to apply them in a particular order, e.g. [x, y]. If someone else applied those same two commits in a different order, they'll get a different commit ID, which may cause problems e.g. when trying to merge their changes with ours. If we treat x and y as (sets of) patches instead, then the set {x, y} is the same as the set {y, x}; the order doesn't matter (we say those patches commute). The idea of commuting patches is really useful, since we can rearrange patches to a more convenient form. For example, if we commit something we shouldn't (like a password, or a huge binary), then later remove it, a system like git makes it hard to remove that file from the history. If we're dealing with sets of patches, we can simply swap them around until the 'add file' and 'remove file' patches are next to each other, then merge those two patches. Voila, the file no longer appears, the rest of the history remains intact, the branch's content is guaranteed to remain unchanged (since we only swapped commuting patches, which doesn't change anything; and merged two patches, which doesn't change anything).
  - cmeacham98 6y ago
    Patches are not commutative in general though, so surely Pijul must have some history/ordering mechanism?
    - zaphar 6y ago
      Pijul can detect when a patch commutes and when it does not. From there it can construct a dependency graph of commits and use that information in various interesting and useful ways. Back when I used Darcs I would somewhat frequently do things like pull these three commits and their decscendants as a sort of pseudo branch that would include everything necessary for a specific line of work but leave everything else behind. To the point of the article when commits are diffs you sort of intuitively think such things should be possible. But because in git commits are snapshots it's not as easy as you would expect it to be.
    - chriswarbo 6y ago
      Yes, patches aren't commutative in general. That is precisely the reason Pijul can help us in these situations, and guarantee the result will be the same: it will either work without issue, or it will abort due to non-commutative patches (it's similar to a type checker, which either guarantees that functions won't be called with the wrong type of argument, or aborts because the types don't match up) From my understanding, the 'history/ordering mechanism' in Pijul is composition of patches. In general, the patch 'patch1 ∘ patch2' can be different from the patch 'patch2 ∘ patch1'; when they just-so-happen to be the same, we say that patch1 and patch2 commute.
      - pmeunier 6y ago
        > In general, the patch 'patch1 ∘ patch2' can be different from the patch 'patch2 ∘ patch1'. That isn't true. In Pijul, either patch1 explicitly depends on patch2, or patch2 explicitly depends on patch1, or else these two things you said are equal.
- dan-robertson 6y ago
  People using git think that commits are patches. But that isn’t how git works. Git sometimes tries to let you treat a commit like the diff between it and it’s parent and lets you try to rewrite history but these are really making new commits with new ids and this confuses people. In pijul, the objects you interact with actually are diffs (aka patches) and then snapshots are well-formed sets of patches. Here, well-formed means that if a patch is in the set then so are it’s dependencies (these dependencies aren’t like parent commits in git, they’re more like you need to add line 3 before you can delete it). So removing or modifying a patch in a branch isn’t a horrific interactive rebase operation anymore. When you move a patch in pijul it doesn’t affect any of the patches written before or after it (unless they depend on it). When you “move a patch” in git you rewrite the history and create new commits, so if I was talking about a commit (id) before the move, I would be talking about some dangling commit after the move and would need to update my id to the corresponding new post-move commit.
diegocg 6y ago
I have visited the pijul site 2 or three times, every time I would start reading about a "sound mathematical theory", get bored, and close the tab. To this date I still don't know what pijul is trying to do and why I should be interested on it. They really should improve their documentation (hint, in case someone reads this: nobody except a few geeks give a shit about sound mathematical models. Show me how pijul makes my life easier compared to git, that's all I need)
- avodonosov 6y ago
  Have you ever rebased a long chain of git commits onto new branch, where one of the first of those commits have a conflict with the new base, and after resolving this conflict for that commit you have the same conflict over and over again for all the subsequent commits, even if they did not modify that place in the code, and you need to manually resolve it again and again? Pijul will, as I understand, save us from those unnecesary repeated "conflicts". See also the answer by @chriswarbo about removing unndesired changes from history
  - jayd16 6y ago
    Could one not add a new rebasing strategy to git by generating patches from the git history? Are the concepts non-translatable?
    - avodonosov 6y ago
      I think rebase alreaby works by generating patches, but for some reason the repeated conflicts happen...
  - zemo 6y ago
    you know about rerere right? https://www.git-scm.com/book/en/v2/Git-Tools-Rerere https://www.git-scm.com/book/en/v2/Git-Tools-Rerere
    - MattIPv4 6y ago
      I feel like this is missing something about the drawbacks? Or are there truly no drawbacks beyond disk usage for the cache, and folks should just enable it once they're aware it exists?
ausbin 6y ago
> Git should work the way we think it does! Hold on, who is "we"? Personally speaking, git works the way I think it does. Granted, I've written my own (simple) libgit2 frontend, so I understand the git internals fairly well, on a high level at least I haven't looked into pijul, but why is teaching people a new tool more helpful than teaching people how the tool they already use works? (Like the OP blog post does.) Am I blinded by the knowledge I gained from writing my little tool and learning about git internals? I get that a tool you need to learn the internals of to use is probably a bad tool, but is asking git users to understand the contents of the OP blog post really too much? Maybe I'm just a git fanboy...
- notdonspaulding 6y ago
  >> Git should work the way we think it does! > Hold on, who is "we"? I'm not the GP, but I agree that git should work the way "we" think it does, and I think a reasonable definition of "we" in the context of Git Users is probably SaaS/Startup/SMB software engineers. Git is popular enough to have many thousands of different use cases, but I would speculate that the distribution of use cases probably follows the distribution of public Github/Gitlab repos pretty closely. > Personally speaking, git works the way I think it does. Granted, I've written my own (simple) libgit2 frontend, ...snip... > Am I blinded by the knowledge I gained from writing my little tool and learning about git internals? Yes. > I get that a tool you need to learn the internals of to use is probably a bad tool, but is asking git users to understand the contents of the OP blog post really too much? Yes. Or rather, knowing git's internals is incredibly helpful if you've already decided to use git and now you're deciding what workflow to use to develop software, because you can match your mental model of how to use git to the way git naturally wants to represent your stored work. However, if you come to git with an existing mental model of software development, and that existing mental model includes the idea of "branches" or "diffs" or "immutable history", then you're going to quickly and repeatedly run into stumbling blocks as your mental model doesn't match git's internal model. Git can do branches and diffs and immutable history, of course, but they're a leaky abstraction on top of the concepts git really cares about. > Maybe I'm just a git fanboy... Sure, nothing wrong with that!
klodolph 6y ago
> Git should work the way we think it does! I think it works using snapshots... or are you saying that Git should work the way that you think it does, and not how I think it does? It's clear that Git is not the final evolution of version control systems, that we are just currently in the "Git era" and at some point we're going to be in the "post-Git era" of VCS. It's unclear what that looks like, but I am skeptical when I hear these claims about Pijul. > One can share just a patch, and applying it is guaranteed to be the same action everywhere if that's possible, which it often is. My understanding is that you need to define a very weak version of "same version everywhere" which is useless. With Git, you can merge and get no conflicts, but that is no guarantee that the patch applied successfully... it just means that the merge operation didn't run into any obstacles. It's not just the patch that needs to be vetted by humans, it's the state which must also be vetted, and that's one of the problems that Git solves well.
mdnahas 6y ago
I strongly disagree. Snapshots are a useful concept for programming. Each snapshot represent a compilable program with a certain set of features. So snapshot A has a certain set of features and B has another. Diffs are not a useful concept. Does the diff between A and B represent the new features in B? No. Because if it did, it would mean I could take any another compilable snapshot C and apply the diff of A and B to it, then I should end up with a snapshot D is compilable and has all the features of C with the new features in B. And that doesn't work with any programming language I know. It doesn't even work with the most trivial features. Diffs may be a useful concept when working with some data formats. But for programming languages, snapshots are the right concept.
- pmeunier 6y ago
  I suggest you read more about Pijul and Darcs, because you seem to be confusing Pijul patches with the output of `diff`. Patches are much easier to work with, more reliable and fully deterministic. For example, merge and rebase are the same operation in Pijul, you can remove an old patch without changing the identity of all the patches after it, and yet have strong version identifiers, with the exact benefits you describe for snapshots.
volta83 6y ago
> I'm patiently waiting for pijul to be mature enough that I can move everything over to using it Pijul is super slow. I've tried it a couple of times, and is too slow to be usable.
- pmeunier 6y ago
  It has made huge progress lately, which was always the plan: the new data representation introduced in November 2020 made it scale to huge repositories (this was impossible before, because of disk space and speed), and then we also changed the backend recently (https://pijul.org/posts/2021-02-06-rethinking-sanakirja/ https://pijul.org/posts/2021-02-06-rethinking-sanakirja/).
  - volta83 6y ago
    I'll check this again!
smichel17 6y ago
I don't view git as a series of diffs. I view it as a logical extension of my file system to include a time dimension (or in fewer words, as snapshots). It replaces file-v1, file-v2, file-v2-with-changes-from-Alex, etc, that you commonly find on the hard drives of people not familiar with version control. That it can generate meaningful diffs is a product of the type of data we're storing.

jayd16 6y ago

Can I make a shallow clone in Pijul?

pmeunier 6y ago
I'm one of the authors. The concepts are different, we do have a concept called "tags" (not ready yet) which is more efficient than shallow clones, in the sense that you can make patches (commits) on top of a shallow clone without any down side (unlike in Git, where shallow repositories can become slower). You can also do partial clones in Pijul: since patches commute, the patches you produce on top of a partial clone can be pushed to the full repository in the exact same way.

cmeacham98 6y ago

No idea what Pijul is, but how does this not describe git? Unless your complaint is that a commit is really a set of diffs/patches?

chriswarbo 6y ago
Pijul (and Darcs) operate on sets of patches. As a simple example, git commits have at least one 'parent', which imposes an order, e.g. let's say I edit file X in commit x and file Y in commit y; if I want both of those changes, git forces me to apply them in a particular order, e.g. [x, y]. If someone else applied those same two commits in a different order, they'll get a different commit ID, which may cause problems e.g. when trying to merge their changes with ours. If we treat x and y as (sets of) patches instead, then the set {x, y} is the same as the set {y, x}; the order doesn't matter (we say those patches commute). The idea of commuting patches is really useful, since we can rearrange patches to a more convenient form. For example, if we commit something we shouldn't (like a password, or a huge binary), then later remove it, a system like git makes it hard to remove that file from the history. If we're dealing with sets of patches, we can simply swap them around until the 'add file' and 'remove file' patches are next to each other, then merge those two patches. Voila, the file no longer appears, the rest of the history remains intact, the branch's content is guaranteed to remain unchanged (since we only swapped commuting patches, which doesn't change anything; and merged two patches, which doesn't change anything).
- cmeacham98 6y ago
  Patches are not commutative in general though, so surely Pijul must have some history/ordering mechanism?
  - zaphar 6y ago
    Pijul can detect when a patch commutes and when it does not. From there it can construct a dependency graph of commits and use that information in various interesting and useful ways. Back when I used Darcs I would somewhat frequently do things like pull these three commits and their decscendants as a sort of pseudo branch that would include everything necessary for a specific line of work but leave everything else behind. To the point of the article when commits are diffs you sort of intuitively think such things should be possible. But because in git commits are snapshots it's not as easy as you would expect it to be.
  - chriswarbo 6y ago
    Yes, patches aren't commutative in general. That is precisely the reason Pijul can help us in these situations, and guarantee the result will be the same: it will either work without issue, or it will abort due to non-commutative patches (it's similar to a type checker, which either guarantees that functions won't be called with the wrong type of argument, or aborts because the types don't match up) From my understanding, the 'history/ordering mechanism' in Pijul is composition of patches. In general, the patch 'patch1 ∘ patch2' can be different from the patch 'patch2 ∘ patch1'; when they just-so-happen to be the same, we say that patch1 and patch2 commute.
    - pmeunier 6y ago
      > In general, the patch 'patch1 ∘ patch2' can be different from the patch 'patch2 ∘ patch1'. That isn't true. In Pijul, either patch1 explicitly depends on patch2, or patch2 explicitly depends on patch1, or else these two things you said are equal.
dan-robertson 6y ago
People using git think that commits are patches. But that isn’t how git works. Git sometimes tries to let you treat a commit like the diff between it and it’s parent and lets you try to rewrite history but these are really making new commits with new ids and this confuses people. In pijul, the objects you interact with actually are diffs (aka patches) and then snapshots are well-formed sets of patches. Here, well-formed means that if a patch is in the set then so are it’s dependencies (these dependencies aren’t like parent commits in git, they’re more like you need to add line 3 before you can delete it). So removing or modifying a patch in a branch isn’t a horrific interactive rebase operation anymore. When you move a patch in pijul it doesn’t affect any of the patches written before or after it (unless they depend on it). When you “move a patch” in git you rewrite the history and create new commits, so if I was talking about a commit (id) before the move, I would be talking about some dangling commit after the move and would need to update my id to the corresponding new post-move commit.

diegocg 6y ago

I have visited the pijul site 2 or three times, every time I would start reading about a "sound mathematical theory", get bored, and close the tab. To this date I still don't know what pijul is trying to do and why I should be interested on it. They really should improve their documentation (hint, in case someone reads this: nobody except a few geeks give a shit about sound mathematical models. Show me how pijul makes my life easier compared to git, that's all I need)

avodonosov 6y ago
Have you ever rebased a long chain of git commits onto new branch, where one of the first of those commits have a conflict with the new base, and after resolving this conflict for that commit you have the same conflict over and over again for all the subsequent commits, even if they did not modify that place in the code, and you need to manually resolve it again and again? Pijul will, as I understand, save us from those unnecesary repeated "conflicts". See also the answer by @chriswarbo about removing unndesired changes from history
- jayd16 6y ago
  Could one not add a new rebasing strategy to git by generating patches from the git history? Are the concepts non-translatable?
  - avodonosov 6y ago
    I think rebase alreaby works by generating patches, but for some reason the repeated conflicts happen...
- zemo 6y ago
  you know about rerere right? https://www.git-scm.com/book/en/v2/Git-Tools-Rerere https://www.git-scm.com/book/en/v2/Git-Tools-Rerere
  - MattIPv4 6y ago
    I feel like this is missing something about the drawbacks? Or are there truly no drawbacks beyond disk usage for the cache, and folks should just enable it once they're aware it exists?

ausbin 6y ago

> Git should work the way we think it does! Hold on, who is "we"? Personally speaking, git works the way I think it does. Granted, I've written my own (simple) libgit2 frontend, so I understand the git internals fairly well, on a high level at least I haven't looked into pijul, but why is teaching people a new tool more helpful than teaching people how the tool they already use works? (Like the OP blog post does.) Am I blinded by the knowledge I gained from writing my little tool and learning about git internals? I get that a tool you need to learn the internals of to use is probably a bad tool, but is asking git users to understand the contents of the OP blog post really too much? Maybe I'm just a git fanboy...

notdonspaulding 6y ago
>> Git should work the way we think it does! > Hold on, who is "we"? I'm not the GP, but I agree that git should work the way "we" think it does, and I think a reasonable definition of "we" in the context of Git Users is probably SaaS/Startup/SMB software engineers. Git is popular enough to have many thousands of different use cases, but I would speculate that the distribution of use cases probably follows the distribution of public Github/Gitlab repos pretty closely. > Personally speaking, git works the way I think it does. Granted, I've written my own (simple) libgit2 frontend, ...snip... > Am I blinded by the knowledge I gained from writing my little tool and learning about git internals? Yes. > I get that a tool you need to learn the internals of to use is probably a bad tool, but is asking git users to understand the contents of the OP blog post really too much? Yes. Or rather, knowing git's internals is incredibly helpful if you've already decided to use git and now you're deciding what workflow to use to develop software, because you can match your mental model of how to use git to the way git naturally wants to represent your stored work. However, if you come to git with an existing mental model of software development, and that existing mental model includes the idea of "branches" or "diffs" or "immutable history", then you're going to quickly and repeatedly run into stumbling blocks as your mental model doesn't match git's internal model. Git can do branches and diffs and immutable history, of course, but they're a leaky abstraction on top of the concepts git really cares about. > Maybe I'm just a git fanboy... Sure, nothing wrong with that!

klodolph 6y ago

> Git should work the way we think it does! I think it works using snapshots... or are you saying that Git should work the way that you think it does, and not how I think it does? It's clear that Git is not the final evolution of version control systems, that we are just currently in the "Git era" and at some point we're going to be in the "post-Git era" of VCS. It's unclear what that looks like, but I am skeptical when I hear these claims about Pijul. > One can share just a patch, and applying it is guaranteed to be the same action everywhere if that's possible, which it often is. My understanding is that you need to define a very weak version of "same version everywhere" which is useless. With Git, you can merge and get no conflicts, but that is no guarantee that the patch applied successfully... it just means that the merge operation didn't run into any obstacles. It's not just the patch that needs to be vetted by humans, it's the state which must also be vetted, and that's one of the problems that Git solves well.

mdnahas 6y ago

I strongly disagree. Snapshots are a useful concept for programming. Each snapshot represent a compilable program with a certain set of features. So snapshot A has a certain set of features and B has another. Diffs are not a useful concept. Does the diff between A and B represent the new features in B? No. Because if it did, it would mean I could take any another compilable snapshot C and apply the diff of A and B to it, then I should end up with a snapshot D is compilable and has all the features of C with the new features in B. And that doesn't work with any programming language I know. It doesn't even work with the most trivial features. Diffs may be a useful concept when working with some data formats. But for programming languages, snapshots are the right concept.

pmeunier 6y ago
I suggest you read more about Pijul and Darcs, because you seem to be confusing Pijul patches with the output of `diff`. Patches are much easier to work with, more reliable and fully deterministic. For example, merge and rebase are the same operation in Pijul, you can remove an old patch without changing the identity of all the patches after it, and yet have strong version identifiers, with the exact benefits you describe for snapshots.

volta83 6y ago

> I'm patiently waiting for pijul to be mature enough that I can move everything over to using it Pijul is super slow. I've tried it a couple of times, and is too slow to be usable.

pmeunier 6y ago
It has made huge progress lately, which was always the plan: the new data representation introduced in November 2020 made it scale to huge repositories (this was impossible before, because of disk space and speed), and then we also changed the backend recently (https://pijul.org/posts/2021-02-06-rethinking-sanakirja/ https://pijul.org/posts/2021-02-06-rethinking-sanakirja/).
- volta83 6y ago
  I'll check this again!

smichel17 6y ago

I don't view git as a series of diffs. I view it as a logical extension of my file system to include a time dimension (or in fewer words, as snapshots). It replaces file-v1, file-v2, file-v2-with-changes-from-Alex, etc, that you commonly find on the hard drives of people not familiar with version control. That it can generate meaningful diffs is a product of the type of data we're storing.