12 ms·

Pijul for Git users

105 points by Volundr 8y ago

chrismatheson 8y ago
Git and Subversion had a baby?
- unhammer 8y ago
  See https://pijul.org/ https://pijul.org/ (and its inspiration, http://darcs.net/ http://darcs.net/ ). It's actually something quite different from both.
pmontra 8y ago
Not the most important matter but from the FAQs > Where does the name come from? > Pijul is the mexican name of Crotophaga sulcirostris, a bird known to do collaborative nest building. About technical stuff: the mathematical theory of patches, commutability, etc are nice properties but any versioning system has to fight a very uphill battle nowadays. Incumbents (mostly git) are very entrenched. I'd like to hear which reasonable expectations the authors of pijul have. By the way, do you ever mistype it? How about a two or three letters official short name?
- michaelmrose 8y ago
  How about pi
- pmeunier 8y ago
  I'm one of the authors. We are actually quite optimistic about Pijul's possibilities. In particular, there are a few niches where people either: (1) do not understand Git, can't learn it, still need a VCS. I have a few industrial applications in mind, not necessarily related to code. (2) understand Git very well, and have spent too many hours debugging their failed merges, non-associativity issues, complex rebases. These users would know enough about "git rerere", for instance, to find the model of conflicts slightly absurd. For them, Git has become a source of cost in the project, and costs need to be kept under control. Again, I have at least one open source project in mind, and a handful of industrial contacts related to these use cases. (3) many people in academia, even in computer science, collaborate on their LaTeX papers using various combinations of Dropbox and emails. This is because they are "principles people" and like to understand the principles behind what they're using. If a tool seems too complicated, it looks suspicious. But anyway, I have written papers with Git before, the "fun" of using Git often fades away as the deadline approaches. Given the very young age of Pijul, I see this picture, and the interest many people have shown in the project, as quite promising. However, unlike what your post may suggest, I don't see Pijul as fighting against Git. Ultimately, when Pijul is ready, the two tools could become quite complementary: - use Git (with branches, but without merges) when you need a linear history and/or a Merkel tree of your content. Git is extremely efficient in disk usage and in speed of retrieving contents. Projects like "Software Heritage" are a good example of the perfect Git-but-not-Pijul user, storage of experimental data is another one ("I've done experiment X on day Y, the results were Z"). - use Pijul when you want to collaborate efficiently on text documents, and manipulate your repository in an extremely flexible way, handle conflicts sanely, while minimising your planning and maintenance costs. Pijul will probably never be as efficient as Git in terms of disk space, but could become quite competitive in terms of merge speed. Agile teams, academic papers, are good examples.
  - xearl 8y ago
    Regarding your niche (2): what open source project do you have in mind?
    - pmeunier 8y ago
      One of the largest open source repositories ever: https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs
  - pas 8y ago
    > [agile] teams yes please. git makes merges hard, and rebases are just too hard when you want to maintain a few different releases. without losing too many QALYs. (with git we [would] need to maintain a lot of branches, just to be able to have clean merge requests at the end for each big feature coming up for a release)
awestroke 8y ago
Pijul keeps touting "patches" instead of commits, and they show how to record and unrecord these patches. But how do you actually work with such a repository? How do you handle merge conflicts? The documentation does not even describe how to resolve conflicts, just that you can procrastinate and save the conflicts "for later"[1]. There is probably technical merit in Pijul, but the documentation and marketing sucks. [1]: https://pijul.org/manual/conflicts.html https://pijul.org/manual/conflicts.html
- emmelaich 8y ago
  I'm not convinced either. What's the performance of getting the the state 10 versions before? What is exactly the size difference between a git repo and a pijul repo of the same thing? As for patch vs snapshot, that's arguing at the implementation level. It doesn't affect my use of a tool except as it relates to performance and merging/branching... etc. Preferring a patch approach rather than a snapshot is like saying map(+)[1,1,1,1,1] != last([1,2,3,4,5]). I don't care.
  - adrianmsmith 8y ago
    You’re right about not caring about the implementation level. But the difference isn’t just at the implementation level, it also relates to merging etc. See here for an example of how merging works better, by taking into account intermediate commits: https://tahoe-lafs.org/~zooko/badmerge/concrete-good-semantics.html https://tahoe-lafs.org/~zooko/badmerge/concrete-good-semanti... That is to say, start with version A, on one branch commit versions B1 and then B2, on another branch commit version C1. To merge the two branches (B2 and C1), Git/Subversion etc. will do a three-way merge of “A to B2” and “A to C1”. It won’t take into account B1 which provides vital information to do a better merge.
    - emmelaich 8y ago
      OK, but if it's needed you could derive patches from snapshots and perform the same steps with a snapshot based VC. I may be missing something here! PS less I seem too harsh here, I love looking into version control systems. I have worked with sccs, rcs, arch, larch, bazaar, bzr, bitkeeper, svn and of course git. Looked at darcs very briefly. Never used cvs seriously. Managed to avoid clearcase thank god.
runyor 8y ago
This is unreasonable. First of all, often you don't need patches, so it's premature optimization. Second, git decided specifically for the actual-state version to have this key-object store, which means you throw away not just the disadvantages but the advantages as well. Last but not least, git ALSO STORES DIFFS when it packs stuff up, which is what happens if big amounts objects need to be transferred or big amounts of old data needs to be stored. That doesn't mean pijul is a bad tool. I didn't check it out. And I really think for everyday coders who don't want to become git-gurus it would be nice to have a simpler git-like VCS, for instance. But the marketing must be updated according to the facts. Try to find features that people really care about, e.g. ease of use, e.g. integration with build tooling and docker, e.g. better federation through more automation which makes centralized servers like github go away.
- pmeunier 8y ago
  I'd say maybe check it out first, especially the parts about patch commutation and the problems with associativity in git merges and rebases.
- feanaro 8y ago
  The primary novelty of Pijul is its sound patch theory, not any kind of technical achievement like smaller repository size. I do think it's premature calling it unreasonable without understanding this aspect.
  - runyor 8y ago
    Why not summarize it a little if you feel that info got lost. Technically if you need a patch you can generate it on the fly by comparing both objects (you can even do that in a bash script with `diff`, if you are willing to type in the logic to look up commit->tree->file->object-name first). So there shouldn't be anything lost. The only parameters I can see with storing diffs vs immutable objects is disk space vs processing time, which is also what git proofs by not storing old immutables and instead store diffs for old stuff (reducing space by increasing processing time).
    - 8y ago
gumby 8y ago
This is basically how CVS worked, as inherited iirc from RCS. Presumably the authors know this, so they must be doing something different, though I can’t tell what from the description or pijul’s FAQ. Perhaps they figured RCS, SCCS et al are so old that mentioning them would add confusion for most readers. Might be reasonable, but for me I am unclear as to what’s up with Pijul. (It’s sweet that their host is named ‘nest’ given the name of the project)
- pmeunier 8y ago
  I'm one of the authors, and we've done a lot of bibliography before engaging in such a big project. I'll talk only about the core principles behind version control, not about how things are actually implemented, or which options are the defaults in which tool. Pijul is absolutely not how RCS or CVS worked. Instead, RCS and CVS worked more or less like SVN (remember the "CVS done right"?), and SVN itself is not too different from "Git without branches", or "Centralised Git", in the sense that these tools are snapshot-based (even though the actual implementations may store patches sometimes). This means that they see history as sequential, and use algorithms to merge when necessary. One could actually very well simulate a large part of Git branches and merges using multiple SVN servers, even though that would obviously be quite cumbersome to implement and use in any concrete situation. The only version control system one can clearly relate Pijul to is Darcs, using the idea that patch commutation is the main thing people care about. Unfortunately, Darcs has important complexity problems when dealing with conflicts, and Pijul solves these problems. That said, short-lived branches ("feature branches") in Git mostly try to simulate commutation, i.e. independent development mergeable later, hopefully in any order (since they were written independently).
  - vnorilo 8y ago
    Kudos on your work. I'm looking forward to trying pijul - I used darcs in the past and I miss some of its capabilities in the current git monoculture. Please ignore the negativity if you can - it's not like anyone was taking git away from people by suggesting that they might try a new thing.
  - gumby 8y ago
vmandela 8y ago
> Git stores the repository changes as a series of snapshots. That means that every time we commit changes it keeps a new version of every file that changed so the size of the repo could grow quickly. The very first statement in the gtihub page seems to ignore Git packfiles completely for the sake of argument. https://git-scm.com/book/en/v2/Git-Internals-Packfiles https://git-scm.com/book/en/v2/Git-Internals-Packfiles
- georgyo 8y ago
  Their wording is indeed bad, and does not convey at all what they wanted to convey. Git stores patches to reduce disk size, and does so very effectively. However the same exact patch, applied to a different head has a different commit id. That commit id is a snapshot, even if it is stored as a patch. Pijul, that same patch applied to different heads, will always have the same hash. This has some advantages that is deeper in their documentation. Basically, you can cherry-pick changes (patches) from someone else, and then later merge their entire branch, but not conflict on the patches you already imported. Having dealt with this with git, I can see the appeal.
  - pmeunier 8y ago
    > That commit id is a snapshot, even if it is stored as a patch. I agree with your argument, but Git does store "blobs" only (plus "trees"), and never patches. When you can ask it to show you a diff between two snapshots, Git computes the diff on the fly. More almost-accurate info here: https://git-man-page-generator.lokaltog.net/ https://git-man-page-generator.lokaltog.net/
    - georgyo 8y ago
      Git does store deltas between files to reduce space. Every so often it stores a full copy of the file to improve performance. Perhapps patch was the wrong word, but a delta and a patch are the same in my mind. I am sure everything, including pijul, does diff generation on the fly of the full files at different points. It would be faster than figuring out the difference of patches alone.
gamesbrainiac 8y ago
How is this any different from mercurial?
- pmeunier 8y ago
  By having a mathematical theory of patches instead of a bunch of (user-friendly and often working) hacks. It's actually explained in the manual: https://pijul.org/manual/why_pijul.html https://pijul.org/manual/why_pijul.html
sametmax 8y ago
I read the article. Then I read the home page. It still don't know why I should give it a try. The practical added value, espacially compared to the cost of switching, is not clear to me. "faster, easier, mathematically proved" is a bit fuzzy as a sale pitch.
norswap 8y ago
For those who wonder what the heck Pijul is about, I'd like to emphasize these two articles linked at the bottom of the page: https://jneem.github.io/merging/ https://jneem.github.io/merging/ https://jneem.github.io/pijul/ https://jneem.github.io/pijul/
- gbrown_ 8y ago
  I read that merging blog post a while back but didn't completely grok it, I've yet to read the paper though. Do you know of any other write-ups on patch commutation?
  - norswap 8y ago
    Unfortunately, no. It's pretty much my only exposure to the idea.
    - gbrown_ 8y ago
      A shame, thanks for the reply though. I'll set aside some time this weekend and read through it again.
- carterschonwald 8y ago
  I find the stuff on jneem’s github wayyy more intelligible than the actual current pijul code. Granted I think it should be largely equivalent! Full disclosure : totally planning to messing around with a Haskell port of some of these ideas sometime soon.
ojosilva 8y ago
Since pijul's key advertised strength is the use of patches for resolving merges, couldn't this just be a new merge strategy option for `git merge -s <strategy>`? In this case, the algorithm would just produce temporary patches out of git snapshots before running its own algo. Even if it comes out being slower than natively storing patches it would be worth the cost if it made complicated merges easier.
- chrismorgan 8y ago
  https://pijul.com/faq/#why-not-a-git-merge-algorithm-instead-of-a-whole-vcs https://pijul.com/faq/#why-not-a-git-merge-algorithm-instead...
arghwhat 8y ago
I don't understand what justifies Pijul's existence. While I agree that Git is a complicated monster that takes ages to learn how to use for most people, Pijul does not seem to solve any problems with Git. Instead, it has many problems that Git solves. From a user-perspective, it doesn't even seem like there's much of a difference, despite the (annoying) choice of alternate command verbs (i.e. "record" rather than "add/commit"). It doesn't really seem like the authors fully understand Git, and what problems it solves. > [...] in Git each commit is related to a parent [...] But in Pijul there's no timeline and branches are just sets of patches. To me, that makes Pijul much, much less capable as a version control system. In Git, knowing the commit ID means that you know the full source tree state with the certainty being equal to the risk of a commit ID hash collision. With just individual, unrelated patches applied in sequence, you have no such guarantee, and no knowledge of the current state of the repository. Of course, you could add a seperate state tracking system allowing ID's to point to an immutable and unique set of patches in a certain sequence, but doing so just means that you have now implemented features identical to Git, with just a different internal representation (i.e. irrelevant differences for the user). > [...] And push those changes in whatever order we want You can do the same in Git (see interactive rebase, cherry-picking, similar). It rewrites the commit IDs in order to maintain the previously mentioned property of representing the state of the repository, but that's an implementation detail.
- tomp 8y ago
  Pijul is based on a different (and much more advanced) algorithm than Git. That doesn't mean it's the perfect tool - just as Git is better than SVN (even though they're based on the same underlying algorithms), Pijul might be the "SVN-like" tool waiting for its own Git... Edit: having said that, Git isn't perfect either, I'm keeping my fingers crossed for an even better tool!
  - DaiPlusPlus 8y ago
    > just as Git is better than SVN (even though they're based on the same underlying algorithms) Please explain this. Git is a content-addressable object-store. SVN is not. SVN’s architecture uses a diff as the basis for versioning which ties you to a particular diffing system, git uses snapshots and diffs can be generated using any algorithm because they aren’t part of the object-store; so what common “underlying algorithms” are you referring to?
samatman 8y ago
Why pijul? Cherry-picking. In git, cherry picking is broken. In pijul, it isn’t. That’s the shortest explanation possible of why a patch-based model is better than a snapshot one. Hope it helps.
Phenix88be 8y ago
People who have issue with Git are just doing Git wrong. They are the same that build a feature on one single big commit impacting lots of files. They forget to merge their hotfix back in dev branch, etc. And Git punish those peoples, but your tool don't, how can i trust the code base ?
- perfmode 8y ago
  Git doesn’t scale.
- johnisgood 8y ago
  > They are the same that build a feature on one single big commit You can easily tell with pijul as well if they are doing that or not.
lixtra 8y ago
> Pijul is the only version control system based on a complete mathematical theory of patches. [1] Can someone please provide a link to that theory? [1] https://nest.pijul.com/ https://nest.pijul.com/ EDIT: This? https://arxiv.org/abs/1311.3903 https://arxiv.org/abs/1311.3903
- cmiles74 8y ago
  I believe Pijul branched from the Darcs codebase. https://en.m.wikipedia.org/wiki/Darcs https://en.m.wikipedia.org/wiki/Darcs
  - Sean1708 8y ago
    > Contrarily to what we hear sometimes, Pijul is not a rewrite of darcs in Rust. The theory is different, and the algorithms have a different complexity. Then of course, Rust was chosen to gain an extra performance factor, but rewriting darcs in Rust would be a lot of work for really minor improvements (darcs is already really good, and Haskell is a great language). From [0]. It's possible that they did branch from darcs, but it seems unlikely given that quote. [0]: https://pijul.org/model/ https://pijul.org/model/
    - cmiles74 8y ago
      Fair enough, after looking around the big quote seems to be this one: "Pijul started as an attempt to fix the performance of darcs, and ended up among the fastest distributed version control systems." https://pijul.org/ https://pijul.org/ It sounds more like they wanted a better Darcs and started from there, rather than actually using any of the Darcs code.
      - pmeunier 8y ago
        The theory is actually quite different, even though Pijul and Darcs behave in a similar way on many examples.
- japanuspus 8y ago
IshKebab 8y ago
This just needs to ditch all of the command line stuff and explain the patch concept up-front. This was the most useful line: > if we unrecord a patch with dependencies all its dependencies are unrecorded as well. Up until that point I had been thinking "but what happens with patches that depend on each other", and I don't think they really explain that ever on the Pijul website. But it doesn't go far enough - how is the dependency automatically tracked for example?
- WorldMaker 8y ago
  Automatic dependency tracking is deceptively simple: If you think of a patch as something as simple as "Modify line 10 of file A from `dog\n` to `cat\n`", the dependency is obvious: whichever previous patch left `dog\n` in line 10 of file A. In most cases that's good enough to get stuff done. Of course, it's not automatic semantic dependency tracking. It can't always entirely tell that Patch B depends on a "feature"/"flag"/"magic bean" in Patch A or it won't CI build, without additional information (such as a CI build process in a bisect, or a manual dependency flag from the author of Patch B or a downstream consumer).
sys_64738 8y ago
How is this better than git format-patch?
jancsika 8y ago
Just a quick sanity check: 1. machine going bad causes data to get corrupted. oh no! 2. can still check the fiddly number with `git log`. whew! 3. grab a clone from $untrustable_cloud and put it on another machine 4. check the fiddly number of the clone with `git log` or whatever. It's the same as the other fiddly number. Hurrah! 5. run `git fsck`. no problems. 6. Keep working as if nothing had happened Same/similar process for Pijul?
jml7c5 8y ago
I'm very interested in Pijul! Patch-based version control always seemed like a much more intuitive paradigm to me. That said: From reading these comments, I would say that perhaps the number one priority for the project is an education/PR push. The best way to draw in more users (and thus more developers!) would be to have your very own Steve Klabnik. Someone to produce not just reference documentation or a manual, but a good, long(!) and detailed introduction with the tone of a blog post. The "why Pijul" section of the manual is a very good start, and the blog posts I've seen (particularly Joe Neeman's) are very fantastic, but it needs to all be tied together in a single section on the pijul.com site and expanded upon. Those docs should include a good description of what Pijul solves that git doesn't. That is to say, basically anything along the lines of "git has a hard time with _______. Will pijul be a silver bullet for that?". (Examples for _______: monorepos, rewriting history, completely purging some file or commit from history). It's unfortunate, but a lot of developers seem to look for what's "best". You see this with new programmers a lot. Any programming forum will have fresh faces asking "I want to learn how to program please tell me what's the best programming language". I've found the JavaScript community in particular is quite keen to chase the new shiny that everyone likes. It's part of why Rust obtained huge momentum so quickly. If there's a really good description of where Pijul beats the competition, it helps with these sorts of people. And finally, those docs (even the bits about Pijul vs. git) need to be written with the assumption that the reader has little or no understanding of version control or git. The docs may have to link to some 'git in 30 minutes' blog post elsewhere on the internet, if necessary. Writing in that style will not only draw in the new users who will help show where Pijul and it's documentation are confusing, but it will ensure that even those with lots of experience with git but little understanding of it will be able to appreciate Pijul's design. This is by no means meant to be a put-down on the project or the way it's been handled so far. There's a lot of negativity in the comments here, but I'm certainly not one to be negative about the project. :)
- adrianmsmith 8y ago
  I would have two pieces of advice: (1) Starting by explaining that Pijul works with patches probably isn't the best place to start. It might be true, but most people are going to think "but if I do 'git diff' I can get a patch, so it's not really any different." I would start with explaining the consequences of this e.g. better merging, and so on. (2) Basically take all the questions asked on this discussion, and put them into an FAQ together with their answers. These are the same questions everyone else is going to be asking too. Even if these questions are misguided, that still doesn't change the fact that's what everyone's going to be asking.
m0zg 8y ago
Rule number 1 for project naming: choose a name which anyone can easily figure out how to pronounce.

Pijul for Git users

105 points by Volundr 8y ago

chrismatheson 8y ago

Git and Subversion had a baby?

unhammer 8y ago
See https://pijul.org/ https://pijul.org/ (and its inspiration, http://darcs.net/ http://darcs.net/ ). It's actually something quite different from both.

pmontra 8y ago

Not the most important matter but from the FAQs > Where does the name come from? > Pijul is the mexican name of Crotophaga sulcirostris, a bird known to do collaborative nest building. About technical stuff: the mathematical theory of patches, commutability, etc are nice properties but any versioning system has to fight a very uphill battle nowadays. Incumbents (mostly git) are very entrenched. I'd like to hear which reasonable expectations the authors of pijul have. By the way, do you ever mistype it? How about a two or three letters official short name?

michaelmrose 8y ago
How about pi
pmeunier 8y ago
I'm one of the authors. We are actually quite optimistic about Pijul's possibilities. In particular, there are a few niches where people either: (1) do not understand Git, can't learn it, still need a VCS. I have a few industrial applications in mind, not necessarily related to code. (2) understand Git very well, and have spent too many hours debugging their failed merges, non-associativity issues, complex rebases. These users would know enough about "git rerere", for instance, to find the model of conflicts slightly absurd. For them, Git has become a source of cost in the project, and costs need to be kept under control. Again, I have at least one open source project in mind, and a handful of industrial contacts related to these use cases. (3) many people in academia, even in computer science, collaborate on their LaTeX papers using various combinations of Dropbox and emails. This is because they are "principles people" and like to understand the principles behind what they're using. If a tool seems too complicated, it looks suspicious. But anyway, I have written papers with Git before, the "fun" of using Git often fades away as the deadline approaches. Given the very young age of Pijul, I see this picture, and the interest many people have shown in the project, as quite promising. However, unlike what your post may suggest, I don't see Pijul as fighting against Git. Ultimately, when Pijul is ready, the two tools could become quite complementary: - use Git (with branches, but without merges) when you need a linear history and/or a Merkel tree of your content. Git is extremely efficient in disk usage and in speed of retrieving contents. Projects like "Software Heritage" are a good example of the perfect Git-but-not-Pijul user, storage of experimental data is another one ("I've done experiment X on day Y, the results were Z"). - use Pijul when you want to collaborate efficiently on text documents, and manipulate your repository in an extremely flexible way, handle conflicts sanely, while minimising your planning and maintenance costs. Pijul will probably never be as efficient as Git in terms of disk space, but could become quite competitive in terms of merge speed. Agile teams, academic papers, are good examples.
- xearl 8y ago
  Regarding your niche (2): what open source project do you have in mind?
  - pmeunier 8y ago
    One of the largest open source repositories ever: https://github.com/NixOS/nixpkgs https://github.com/NixOS/nixpkgs
- pas 8y ago
  > [agile] teams yes please. git makes merges hard, and rebases are just too hard when you want to maintain a few different releases. without losing too many QALYs. (with git we [would] need to maintain a lot of branches, just to be able to have clean merge requests at the end for each big feature coming up for a release)

awestroke 8y ago

Pijul keeps touting "patches" instead of commits, and they show how to record and unrecord these patches. But how do you actually work with such a repository? How do you handle merge conflicts? The documentation does not even describe how to resolve conflicts, just that you can procrastinate and save the conflicts "for later"[1]. There is probably technical merit in Pijul, but the documentation and marketing sucks. [1]: https://pijul.org/manual/conflicts.html https://pijul.org/manual/conflicts.html

emmelaich 8y ago
I'm not convinced either. What's the performance of getting the the state 10 versions before? What is exactly the size difference between a git repo and a pijul repo of the same thing? As for patch vs snapshot, that's arguing at the implementation level. It doesn't affect my use of a tool except as it relates to performance and merging/branching... etc. Preferring a patch approach rather than a snapshot is like saying map(+)[1,1,1,1,1] != last([1,2,3,4,5]). I don't care.
- adrianmsmith 8y ago
  You’re right about not caring about the implementation level. But the difference isn’t just at the implementation level, it also relates to merging etc. See here for an example of how merging works better, by taking into account intermediate commits: https://tahoe-lafs.org/~zooko/badmerge/concrete-good-semantics.html https://tahoe-lafs.org/~zooko/badmerge/concrete-good-semanti... That is to say, start with version A, on one branch commit versions B1 and then B2, on another branch commit version C1. To merge the two branches (B2 and C1), Git/Subversion etc. will do a three-way merge of “A to B2” and “A to C1”. It won’t take into account B1 which provides vital information to do a better merge.
  - emmelaich 8y ago
    OK, but if it's needed you could derive patches from snapshots and perform the same steps with a snapshot based VC. I may be missing something here! PS less I seem too harsh here, I love looking into version control systems. I have worked with sccs, rcs, arch, larch, bazaar, bzr, bitkeeper, svn and of course git. Looked at darcs very briefly. Never used cvs seriously. Managed to avoid clearcase thank god.

runyor 8y ago

This is unreasonable. First of all, often you don't need patches, so it's premature optimization. Second, git decided specifically for the actual-state version to have this key-object store, which means you throw away not just the disadvantages but the advantages as well. Last but not least, git ALSO STORES DIFFS when it packs stuff up, which is what happens if big amounts objects need to be transferred or big amounts of old data needs to be stored. That doesn't mean pijul is a bad tool. I didn't check it out. And I really think for everyday coders who don't want to become git-gurus it would be nice to have a simpler git-like VCS, for instance. But the marketing must be updated according to the facts. Try to find features that people really care about, e.g. ease of use, e.g. integration with build tooling and docker, e.g. better federation through more automation which makes centralized servers like github go away.

pmeunier 8y ago
I'd say maybe check it out first, especially the parts about patch commutation and the problems with associativity in git merges and rebases.
feanaro 8y ago
The primary novelty of Pijul is its sound patch theory, not any kind of technical achievement like smaller repository size. I do think it's premature calling it unreasonable without understanding this aspect.
- runyor 8y ago
  Why not summarize it a little if you feel that info got lost. Technically if you need a patch you can generate it on the fly by comparing both objects (you can even do that in a bash script with `diff`, if you are willing to type in the logic to look up commit->tree->file->object-name first). So there shouldn't be anything lost. The only parameters I can see with storing diffs vs immutable objects is disk space vs processing time, which is also what git proofs by not storing old immutables and instead store diffs for old stuff (reducing space by increasing processing time).
  - 8y ago

gumby 8y ago

This is basically how CVS worked, as inherited iirc from RCS. Presumably the authors know this, so they must be doing something different, though I can’t tell what from the description or pijul’s FAQ. Perhaps they figured RCS, SCCS et al are so old that mentioning them would add confusion for most readers. Might be reasonable, but for me I am unclear as to what’s up with Pijul. (It’s sweet that their host is named ‘nest’ given the name of the project)

pmeunier 8y ago
I'm one of the authors, and we've done a lot of bibliography before engaging in such a big project. I'll talk only about the core principles behind version control, not about how things are actually implemented, or which options are the defaults in which tool. Pijul is absolutely not how RCS or CVS worked. Instead, RCS and CVS worked more or less like SVN (remember the "CVS done right"?), and SVN itself is not too different from "Git without branches", or "Centralised Git", in the sense that these tools are snapshot-based (even though the actual implementations may store patches sometimes). This means that they see history as sequential, and use algorithms to merge when necessary. One could actually very well simulate a large part of Git branches and merges using multiple SVN servers, even though that would obviously be quite cumbersome to implement and use in any concrete situation. The only version control system one can clearly relate Pijul to is Darcs, using the idea that patch commutation is the main thing people care about. Unfortunately, Darcs has important complexity problems when dealing with conflicts, and Pijul solves these problems. That said, short-lived branches ("feature branches") in Git mostly try to simulate commutation, i.e. independent development mergeable later, hopefully in any order (since they were written independently).
- vnorilo 8y ago
  Kudos on your work. I'm looking forward to trying pijul - I used darcs in the past and I miss some of its capabilities in the current git monoculture. Please ignore the negativity if you can - it's not like anyone was taking git away from people by suggesting that they might try a new thing.
- gumby 8y ago

vmandela 8y ago

> Git stores the repository changes as a series of snapshots. That means that every time we commit changes it keeps a new version of every file that changed so the size of the repo could grow quickly. The very first statement in the gtihub page seems to ignore Git packfiles completely for the sake of argument. https://git-scm.com/book/en/v2/Git-Internals-Packfiles https://git-scm.com/book/en/v2/Git-Internals-Packfiles

georgyo 8y ago
Their wording is indeed bad, and does not convey at all what they wanted to convey. Git stores patches to reduce disk size, and does so very effectively. However the same exact patch, applied to a different head has a different commit id. That commit id is a snapshot, even if it is stored as a patch. Pijul, that same patch applied to different heads, will always have the same hash. This has some advantages that is deeper in their documentation. Basically, you can cherry-pick changes (patches) from someone else, and then later merge their entire branch, but not conflict on the patches you already imported. Having dealt with this with git, I can see the appeal.
- pmeunier 8y ago
  > That commit id is a snapshot, even if it is stored as a patch. I agree with your argument, but Git does store "blobs" only (plus "trees"), and never patches. When you can ask it to show you a diff between two snapshots, Git computes the diff on the fly. More almost-accurate info here: https://git-man-page-generator.lokaltog.net/ https://git-man-page-generator.lokaltog.net/
  - georgyo 8y ago
    Git does store deltas between files to reduce space. Every so often it stores a full copy of the file to improve performance. Perhapps patch was the wrong word, but a delta and a patch are the same in my mind. I am sure everything, including pijul, does diff generation on the fly of the full files at different points. It would be faster than figuring out the difference of patches alone.

gamesbrainiac 8y ago

How is this any different from mercurial?

pmeunier 8y ago
By having a mathematical theory of patches instead of a bunch of (user-friendly and often working) hacks. It's actually explained in the manual: https://pijul.org/manual/why_pijul.html https://pijul.org/manual/why_pijul.html

sametmax 8y ago

I read the article. Then I read the home page. It still don't know why I should give it a try. The practical added value, espacially compared to the cost of switching, is not clear to me. "faster, easier, mathematically proved" is a bit fuzzy as a sale pitch.

norswap 8y ago

For those who wonder what the heck Pijul is about, I'd like to emphasize these two articles linked at the bottom of the page: https://jneem.github.io/merging/ https://jneem.github.io/merging/ https://jneem.github.io/pijul/ https://jneem.github.io/pijul/

gbrown_ 8y ago
I read that merging blog post a while back but didn't completely grok it, I've yet to read the paper though. Do you know of any other write-ups on patch commutation?
- norswap 8y ago
  Unfortunately, no. It's pretty much my only exposure to the idea.
  - gbrown_ 8y ago
    A shame, thanks for the reply though. I'll set aside some time this weekend and read through it again.
carterschonwald 8y ago
I find the stuff on jneem’s github wayyy more intelligible than the actual current pijul code. Granted I think it should be largely equivalent! Full disclosure : totally planning to messing around with a Haskell port of some of these ideas sometime soon.

ojosilva 8y ago

Since pijul's key advertised strength is the use of patches for resolving merges, couldn't this just be a new merge strategy option for `git merge -s <strategy>`? In this case, the algorithm would just produce temporary patches out of git snapshots before running its own algo. Even if it comes out being slower than natively storing patches it would be worth the cost if it made complicated merges easier.

chrismorgan 8y ago
https://pijul.com/faq/#why-not-a-git-merge-algorithm-instead-of-a-whole-vcs https://pijul.com/faq/#why-not-a-git-merge-algorithm-instead...

arghwhat 8y ago

I don't understand what justifies Pijul's existence. While I agree that Git is a complicated monster that takes ages to learn how to use for most people, Pijul does not seem to solve any problems with Git. Instead, it has many problems that Git solves. From a user-perspective, it doesn't even seem like there's much of a difference, despite the (annoying) choice of alternate command verbs (i.e. "record" rather than "add/commit"). It doesn't really seem like the authors fully understand Git, and what problems it solves. > [...] in Git each commit is related to a parent [...] But in Pijul there's no timeline and branches are just sets of patches. To me, that makes Pijul much, much less capable as a version control system. In Git, knowing the commit ID means that you know the full source tree state with the certainty being equal to the risk of a commit ID hash collision. With just individual, unrelated patches applied in sequence, you have no such guarantee, and no knowledge of the current state of the repository. Of course, you could add a seperate state tracking system allowing ID's to point to an immutable and unique set of patches in a certain sequence, but doing so just means that you have now implemented features identical to Git, with just a different internal representation (i.e. irrelevant differences for the user). > [...] And push those changes in whatever order we want You can do the same in Git (see interactive rebase, cherry-picking, similar). It rewrites the commit IDs in order to maintain the previously mentioned property of representing the state of the repository, but that's an implementation detail.

tomp 8y ago
Pijul is based on a different (and much more advanced) algorithm than Git. That doesn't mean it's the perfect tool - just as Git is better than SVN (even though they're based on the same underlying algorithms), Pijul might be the "SVN-like" tool waiting for its own Git... Edit: having said that, Git isn't perfect either, I'm keeping my fingers crossed for an even better tool!
- DaiPlusPlus 8y ago
  > just as Git is better than SVN (even though they're based on the same underlying algorithms) Please explain this. Git is a content-addressable object-store. SVN is not. SVN’s architecture uses a diff as the basis for versioning which ties you to a particular diffing system, git uses snapshots and diffs can be generated using any algorithm because they aren’t part of the object-store; so what common “underlying algorithms” are you referring to?

samatman 8y ago

Why pijul? Cherry-picking. In git, cherry picking is broken. In pijul, it isn’t. That’s the shortest explanation possible of why a patch-based model is better than a snapshot one. Hope it helps.

Phenix88be 8y ago

People who have issue with Git are just doing Git wrong. They are the same that build a feature on one single big commit impacting lots of files. They forget to merge their hotfix back in dev branch, etc. And Git punish those peoples, but your tool don't, how can i trust the code base ?

perfmode 8y ago
Git doesn’t scale.
johnisgood 8y ago
> They are the same that build a feature on one single big commit You can easily tell with pijul as well if they are doing that or not.

lixtra 8y ago

> Pijul is the only version control system based on a complete mathematical theory of patches. [1] Can someone please provide a link to that theory? [1] https://nest.pijul.com/ https://nest.pijul.com/ EDIT: This? https://arxiv.org/abs/1311.3903 https://arxiv.org/abs/1311.3903

cmiles74 8y ago
I believe Pijul branched from the Darcs codebase. https://en.m.wikipedia.org/wiki/Darcs https://en.m.wikipedia.org/wiki/Darcs
- Sean1708 8y ago
  > Contrarily to what we hear sometimes, Pijul is not a rewrite of darcs in Rust. The theory is different, and the algorithms have a different complexity. Then of course, Rust was chosen to gain an extra performance factor, but rewriting darcs in Rust would be a lot of work for really minor improvements (darcs is already really good, and Haskell is a great language). From [0]. It's possible that they did branch from darcs, but it seems unlikely given that quote. [0]: https://pijul.org/model/ https://pijul.org/model/
  - cmiles74 8y ago
    Fair enough, after looking around the big quote seems to be this one: "Pijul started as an attempt to fix the performance of darcs, and ended up among the fastest distributed version control systems." https://pijul.org/ https://pijul.org/ It sounds more like they wanted a better Darcs and started from there, rather than actually using any of the Darcs code.
    - pmeunier 8y ago
      The theory is actually quite different, even though Pijul and Darcs behave in a similar way on many examples.
japanuspus 8y ago

IshKebab 8y ago

This just needs to ditch all of the command line stuff and explain the patch concept up-front. This was the most useful line: > if we unrecord a patch with dependencies all its dependencies are unrecorded as well. Up until that point I had been thinking "but what happens with patches that depend on each other", and I don't think they really explain that ever on the Pijul website. But it doesn't go far enough - how is the dependency automatically tracked for example?

WorldMaker 8y ago
Automatic dependency tracking is deceptively simple: If you think of a patch as something as simple as "Modify line 10 of file A from `dog\n` to `cat\n`", the dependency is obvious: whichever previous patch left `dog\n` in line 10 of file A. In most cases that's good enough to get stuff done. Of course, it's not automatic semantic dependency tracking. It can't always entirely tell that Patch B depends on a "feature"/"flag"/"magic bean" in Patch A or it won't CI build, without additional information (such as a CI build process in a bisect, or a manual dependency flag from the author of Patch B or a downstream consumer).

sys_64738 8y ago

How is this better than git format-patch?

jancsika 8y ago

Just a quick sanity check: 1. machine going bad causes data to get corrupted. oh no! 2. can still check the fiddly number with `git log`. whew! 3. grab a clone from $untrustable_cloud and put it on another machine 4. check the fiddly number of the clone with `git log` or whatever. It's the same as the other fiddly number. Hurrah! 5. run `git fsck`. no problems. 6. Keep working as if nothing had happened Same/similar process for Pijul?

jml7c5 8y ago

I'm very interested in Pijul! Patch-based version control always seemed like a much more intuitive paradigm to me. That said: From reading these comments, I would say that perhaps the number one priority for the project is an education/PR push. The best way to draw in more users (and thus more developers!) would be to have your very own Steve Klabnik. Someone to produce not just reference documentation or a manual, but a good, long(!) and detailed introduction with the tone of a blog post. The "why Pijul" section of the manual is a very good start, and the blog posts I've seen (particularly Joe Neeman's) are very fantastic, but it needs to all be tied together in a single section on the pijul.com site and expanded upon. Those docs should include a good description of what Pijul solves that git doesn't. That is to say, basically anything along the lines of "git has a hard time with _______. Will pijul be a silver bullet for that?". (Examples for _______: monorepos, rewriting history, completely purging some file or commit from history). It's unfortunate, but a lot of developers seem to look for what's "best". You see this with new programmers a lot. Any programming forum will have fresh faces asking "I want to learn how to program please tell me what's the best programming language". I've found the JavaScript community in particular is quite keen to chase the new shiny that everyone likes. It's part of why Rust obtained huge momentum so quickly. If there's a really good description of where Pijul beats the competition, it helps with these sorts of people. And finally, those docs (even the bits about Pijul vs. git) need to be written with the assumption that the reader has little or no understanding of version control or git. The docs may have to link to some 'git in 30 minutes' blog post elsewhere on the internet, if necessary. Writing in that style will not only draw in the new users who will help show where Pijul and it's documentation are confusing, but it will ensure that even those with lots of experience with git but little understanding of it will be able to appreciate Pijul's design. This is by no means meant to be a put-down on the project or the way it's been handled so far. There's a lot of negativity in the comments here, but I'm certainly not one to be negative about the project. :)

adrianmsmith 8y ago
I would have two pieces of advice: (1) Starting by explaining that Pijul works with patches probably isn't the best place to start. It might be true, but most people are going to think "but if I do 'git diff' I can get a patch, so it's not really any different." I would start with explaining the consequences of this e.g. better merging, and so on. (2) Basically take all the questions asked on this discussion, and put them into an FAQ together with their answers. These are the same questions everyone else is going to be asking too. Even if these questions are misguided, that still doesn't change the fact that's what everyone's going to be asking.

m0zg 8y ago

Rule number 1 for project naming: choose a name which anyone can easily figure out how to pronounce.