12 ms·
Pijul: Version-Control Post-Git [video]
- tiberriver256 3y agoThis is supposed to be easier to use than git but the getting started guide is mostly mathematic algorithms. I think maybe they need someone to help write some user friendly documentation.
- dharmab 3y agoWhich page are you looking at? https://pijul.org/manual/getting_started.html https://pijul.org/manual/getting_started.html is mostly some shell commands, similar to a Git beginner's guide.
- ComputerGuru 3y agoNeeds [video] in the title!
- cryptonector 3y agoIt says "youtube.com" on the side of the title. The main content type at YouTube is video content. It's clear enough.
- ComputerGuru 3y agoThe site has outstanding conventions and policies. This is one of them.
- crickey 3y agoNothing against the project or talk itself. But kinda funny when a talk starts with “i bet many of you thought X was a solved problem, well im here to tell u it isnt.” Git was populare not because its somehow revolutionary, its popular because the previous options where so increadibly shit. Any alternative to git is gonna have a hard time without that advantage
- tiffanyg 3y agoHaha - in enough ways, git is shit. I'm not disagreeing, though. I think you're right (for the "foreseeable future"). CVS, SVN, etc., really became serious obstacles once you moved to larger projects and more distributed teams etc. Hell, they caused problems even with small colocated teams. I think it's clear that git (and its forerunners, esp. "BitKeeper") meets a solid "good enough" standard. Potential competitors from that era, including, in particular, "Mercurial", largely fall into categories of "trade-offs". But, I am, personally, very happy to see the work that's been done with "Pijul", including completing a "theory of merging". I don't think Pijul has much of a chance in even 10+ years of replacing git (and "GitHub", part of the success story of git). But, I do think it'll see some use, become a solid foundation for some projects, and, there's a reasonable chance it will influence or become the foundation of some "next git" and/or future versions of git. With the caveat, of course, that forecasting anything on those kinds of timeframes is even more of a fools errand now than it was even ~15+ years ago (about when git was first developed).
- deleted 3y ago[deleted]
- imiric 3y ago> Potential competitors from that era, including, in particular, "Mercurial", largely fall into categories of "trade-offs". Which "trade-offs" are you referring to? Mercurial was, and still is, a solid DVCS. It's not often used today because it lost the popularity contest, due to several reasons[0], but technically it's as good as Git, and functionally it's even better. It has a much saner and friendlier UI, which is the main complaint about Git. [0]: https://news.ycombinator.com/item?id=37095925 https://news.ycombinator.com/item?id=37095925
- imiric 3y agoGit itself might not have been revolutionary, but the concept of distributed version control certainly was. Git wasn't the only tool from that era to adopt this model, but it's fair to say that it has won the popularity contest, and is the modern standard in most projects. Mercurial, Darcs and Fossil are also interesting, and in some ways better than Git, but Git won because it had the persona of Linus behind it, one of the most popular and influential OSS projects using it as proving ground, and a successful and user friendly commercial service built directly around it, that included it even in its name. All of this was enough for Git to gain traction and pull ahead of other DVCSs, even though in the early days Mercurial and Bitbucket were also solid and popular choices. I used and preferred Mercurial for a long time, but ultimately Git was more prevalent, and it felt like swimming against the current. I feel like that also happened with Docker (Swarm) and Kubernetes, where k8s is now the de facto container orchestration standard, much to my own chagrin.
- MarkSweep 3y agoI don’t know much about using Pijul, but one nice thing about it compared to Git is its implementation is mostly defined in a library while the command line executable is separate: https://nest.pijul.com/pijul/pijul https://nest.pijul.com/pijul/pijul Git can be annoying to integrate into a larger system without resorting to shelling out to the Git executable. There are alternatives like libgit2 and jgit, but they only have a subset of functionality.
- jiggawatts 3y agoThis causes a lot of grief for IDEs like Visual Studio, where updating Git breaks the IDE because there is no "API contract". The text output changes, VS fails to parse it, and just crashes out with inscrutable errors. I've become very opinionated in my old age, and I now firmly believe that: 1. All command-line tools should be "library first, cli second". 2. All text-based formats should include a parser and formatter as a function. Never specify a text format you can't round-trip. In other words, always include an "escape" function and an "unescape" function, or better yet, a parser and serializer. Random config files in Linux are notorious for not doing this. I want to be able to parse them, modify the object in memory, and then write them back out without having to worry about how strings are quoted or dates are formatted. 3. Protocols should always come with a non-executable and machine-readable spec. Think ANTLR grammar file, Open API spec, or something. Never use English only to describe a protocol. Make sure client code can be 100% automatically generated by a tool, in multiple languages.
- pydave 3y agoIs that because they're parsing porcelain output? Or is git's plumbing machine-readable but not well specified? But git users are more familiar with porcelain so I wouldn't be surprised if they parsed that for an initial implementation. It sounds like plumbing shouldn't break as often as you imply: > The interface (input, output, set of options and the semantics) to these low-level commands are meant to be a lot more stable than Porcelain level commands, because these commands are primarily for scripted use. https://schacon.github.io/git/git.html#_low_level_commands_plumbing https://schacon.github.io/git/git.html#_low_level_commands_p... However, doesn't seem like they're nearly as rigorous as you hope.
- est31 3y agoSee also: https://news.ycombinator.com/item?id=25032956 https://news.ycombinator.com/item?id=25032956 Pijul is definitely a nice research project, and maybe some of the technology behind it could be integrated into better git tooling. But I doubt that exposing users to the raw "everything is just a bucket of patches, you have no history of repo state" idea is good. People want to be able to bisect, or check out some older version at a given time point. There are benefits of pijul's model, but I don't think they are worth as much as losing that.
- deleted 3y ago[deleted]
- pkulak 3y agoWait, you can't see the state of the repository at an arbitrary point in time?
- h0l0cube 3y ago"bucket of patches" doesn't imply the patches couldn't be replayed to a point in time. Looking at the docs, you could probably use `log` and `unrecord` to go back to an earlier state. Or use `change` to view a particular change. https://pijul.org/manual/reference.html https://pijul.org/manual/reference.html
- est31 3y agoThe issue is that `log` orders the patches by when they were made, not by when they were merged (in reality it might be a bit more detailed, probably a patch will always be ordered after the patches it depends on). git log also has parameters to configure it in that fashion, and it's also what github uses. However, importantly, git bisect does not use it. It uses the DAG of commits that git maintains. Yes, also git also only has a weak notion of a history of a branch: it is somewhat inside the titles of merge commits, and merge commits also have an order of their parents, so often you can just walk the first parent and always get to commits that used to be state of the master branch at a specific point in time. Sometimes it's even easier when merged PRs get squashed. But yeah, it's a bit implicit and if you e.g. rebase a PR then some of the commits in it have never been the tip of the master branch. But very importantly, every commit in git depends on every single commit that has contributed to that commit's tree. In pijul, they break up that property, and patches only depend on those patches which have directly modified the pieces of code that the change is editing. IIRC you can also manually introduce dependencies but those are rare. There isn't even a notion of a "patch's tree". You could construct a branch with only the patch and its dependencies but probably that wouldn't compile, and it will definitely not reflect the state of the branch at that point in time. You can make new branches in pijul that you don't modify and use like tags, but it is an explicitly requested thing. I mean, I see the elegance in pijul. Reverting a patch that has no conflicts and un-reverting it is trivial. But on the other hand, how often do you do that? Some workflows need patch based thinking, others need snapshot based thinking, and VCS tools need to master both, whether they start out with a snapshot history (like git) and then build diff based workflows on it (diff, rebase, etc), or whether they start out with a patch tree, and then build snapshot based workflows on it. pijul has weak snapshot based workflows, this part is still missing.
- PaulDavisThe1st 3y agoWithin the first 5 minutes, the presentation claims that "the tools are unusable without a global central server". I really don't know what that is supposed to mean. Plemty of people do in fact use github, but I don't know why anyone would say that git in particular is "unusable" without a global central server. Github (etc.) are conveniences for people who don't want to self-host. They provide some additional tools that can be useful in some situations. But nothing more.
- reilly3000 3y agoThere are vast swaths of devs who are not aware people still use mailing lists, nor that git has first class support for applying patches from emails.
- crdrost 3y agoAnd that this is not just a minor niche use case, "the largest and oldest Git project that is, Linux, uses a mailing list and patch files to develop."
- bananapub 3y agoit is definitely not the oldest
- JoshTriplett 3y agoIn the sense that it was the first project to use git (has been using git the longest), not in the sense that it's the oldest project that switched to git.
- smohare 3y ago[dead]
- cesarb 3y ago> it is definitely not the oldest IIRC, the oldest was git itself (it self-hosted early on its development history); git, linux, and sparse are IIRC the three projects that have been using git the longest.
- wyldfire 3y agoJujutsu was discussed recently - https://news.ycombinator.com/item?id=36952796 https://news.ycombinator.com/item?id=36952796 IIRC it's inspired by Pijul and mercurial and uses git database underneath? or it's git-compatible at least.
- blurbleblurble 3y agoIt doesn't support patch commutation, which in my mind is the whole point of using Pijul. https://news.ycombinator.com/item?id=29792092 https://news.ycombinator.com/item?id=29792092 Imagine being able to "toggle" features or changes quickly and easily without worrying about which order they were implemented in. Patch commutation makes that easier.
- wyldfire 3y agoIf we can get rid of the "staging area" and resolve the ambiguities of "checkout" it would be a great step in the right direction.
- Izkata 3y agoHaving come from subversion, the staging area is by far my favorite part of git in day-to-day use.
- forrestthewoods 3y agoWhy is that? I work regularly with Mercurial and Git. The stage isn’t a huge problem or anything, but it’s super annoying and gives me zero value. TBH I think the stage is kinda dirty. Because it means you’re making a commit that has never actually existed on your machine and thus can’t have possibly been tested.
- Izkata 3y agoTwo things: To review the changes piece by piece before commit (git add -p) in case there's something temporary in there I forgot, and to easily exclude any such thing I never intended to commit (usually todo-style comments, sometimes hardcoding a value to trigger a bug or a popup, etc) I remember a decade ago trying mercurial and git at the same time when subversion was all I knew, and that there was something about mercurial that I found really confusing. It's long enough ago I don't remember what it was, and it may be better now, but git clicked immediately. I wouldn't be surprised if it was the way they handle commits instead of having a staging area. Also compared to subversion, the staging area is an explicit form of something that half-exists in how it tracks files for committing.
- MattyRad 3y agoThat was very in depth! As far as Pijul as a tool goes, I'm not seeing a git compatibility layer? So I think it's a neat project, but I probably won't try it because nearly all code is rooted squarely in git. Even if Pijul is perfect, you'd need to convince everyone else to use it. Nevertheless, the increased interest in moving to patch based workflows from branch based ones is great. There's a lot of similar tools here (https://github.com/gitext-rs/git-stack/blob/main/docs/comparison.md https://github.com/gitext-rs/git-stack/blob/main/docs/compar...) which I refer to infrequently. Personally my favorite tool for living-with-the-reality-that-is-branches is git-machete (https://github.com/VirtusLab/git-machete https://github.com/VirtusLab/git-machete).
- aozgaa 3y agoSee “Import a Git Repository” at https://nest.pijul.com/pijul/pijul https://nest.pijul.com/pijul/pijul
- Tempest1981 3y agoLooks cool -- What does machete do when it hits a merge conflict that requires manual resolution?
- _ikke_ 3y agoIt stops and asks you to resolve it and then you'd use the regular git commands to continue that operation (like git rebase --continue)
- raggi 3y agoI don't buy (as sold) the argument that we need associative and commutative merges, and I believe that this constraint when applied by the software makes resolving merge requirements harder on the user, rather than easier. It's punting the slightly more solved part of the problem back on the user, when it's one of the least favorite parts of the problem for the user. The solution that he writes up is basically "git merge" and part of the reason that we have such a bad time with git is that the industry largely standardized on "git rebase", which destroys many of the properties listed: changes are partially ordered: if you merge, this is true, if you rebase, this is not, your change is strictly reodered atop the merge base, and you dear user are responsible for the hell of fixing the constant conflict this produces, even if an alternative merge order would implicitly fix this. when pijul says "partial ordering" here, it means (in slightly abstract and minutely different terms) it's doing what git merge does, because it's better. As it happens one of the various reasons rebase became popular was that `git log` order shifts around for efficiency and correctness reasons, when people mostly want the output of `git log --topo-order` by default, which for the common case of "merge right onto left" presents "right first, then left" in linearized history, unless there's a content defined order. Pijul dodges the presentation problem by just not presenting this information at all. Git's mistake was in many ways not hiding enough from users - but it came from kernel engineers, who don't want things being hidden. No git rerere: rerere is a hack to try to make rebase pretend that it doesn't drop usable merge information on the floor, if you stop imposing rebase on yourself you don't need the rerere hack. Partial clones & large files: these are storage structure optimizations, and yes, a problem for git, though for example git lfs approach is similar in solution to the described pijul solution: "we only need a description of operations" - that's how you would describe lfs changes. As it happens, that's also how you would describe subproject updates, and yes, sure subproject UX isn't great, but now we're back to a UX problem, not a storage problem. Working with large graphs: cache snapshots and index deltas, um, yeah. You don't have to squint very hard to draw equivalence here either. I'm not apologizing for git, but there's a lot of "here's some non-ascii literals on a slide, it's science" presented, but not actually a very substantial shift in trade-offs when applied in practice. There are some pathologies improved, and some worsened, and there are some more onus put on the user for some cases, and some removed for other cases. It's a very different (at the minute level) structure, sure, but I've not been sold that it makes a set of trade-offs that substantially change the game, it's just the other side of the field, not all that dissimilar from choosing between two different database engines. An example of it not being that different in practice: can you conflict with yourself in pijul? Absolutely, and you end up with the pijul version of merge commits, e.g. https://nest.pijul.com/pijul/thrussh/changes/7S7FHFDVSSRB4DCBZINHNWM3QJS6KYAR2L2HADDBDN4IHKN6KGLAC https://nest.pijul.com/pijul/thrussh/changes/7S7FHFDVSSRB4DC..., more than that you can conflict with yourself multiple times (which is basically the rerere problem but actually distributed, which was somewhat erroneously claimed to be eradicated). It hasn't substantially moved the needle on merging history quality _in practice_. The manual is very incomplete on this, i.e. https://pijul.org/manual/conflicts.html https://pijul.org/manual/conflicts.html and there are some nascent but very slow tools to unhide the totality of conflicts in a repo https://nest.pijul.com/laumann/pijul-conflicts https://nest.pijul.com/laumann/pijul-conflicts which would bring the rerereal problem (couldn't resist, sorry) into the light. What's the largest team concurrently working on one project full time using pijul today? I ask because small numbers of users don't create many interesting conflicts often, and don't provide a good insight for the properties of a VCS at scale/in practice. You can use CVS with zero problems when you're on your own. There's an embedded theory which is that you can do very lazy conflict resolution requirement and that it'll pay off. I'd like to see that theory tested at scale.
- nemetroid 3y agoI really like the idea of a version control system based on commutative patches but I'm just not seeing how it's going to translate to a system that's intuitive and easy to use not just for programmers, but also for lawyers and artists. Consider a patch like this one, titled "Solving conflicts": https://nest.pijul.com/pijul/pijul/changes/2MLOE3FPVT4ZXC4L74VC7GV2IXATZQNFLJECDPZZ7MTSW4A6XT5QC https://nest.pijul.com/pijul/pijul/changes/2MLOE3FPVT4ZXC4L7... "Solving an order conflict" and "Resurrecting zombie bytes" sound like the tricky parts of the patch-based workflow. I would very much like to understand how to work with these parts of Pijul. How are these actions performed? How does the user interact with the graph to perform these actions? Is the conflict state rendered into the file? It's not mentioned in the video, and I cannot find it in the documentation or in blog posts. I cannot be the only one that's interested in learning about those things, and I think it would be more effective than anything else in raising interest for Pijul. My best guess is that the workflow is still rough around the edges and requires the user to have a good understanding of the underlying graph, and therefore it's not ready to be presented yet. And my hot take is that it's still rough around the edges because there is no easy way, and maybe no way at all, to make it polished and intuitive for a new user. But I want to be proven wrong.
- jsunderland323 3y agoI’d be happy to demo a work in progress if you want to see a structured dvcs. This is a surprisingly tricky problem. There’ve been quite a few attempts throughout the years at a structured/visual dcvs but it’s almost impossible to solve without also building a schema language, an offline browser, and a package manager/app registry. There’s murmurs in some of the offline first/CRDT communities that suggest other solutions but I haven’t seen anything structured and viable emerge. Anyway, let me know if you want to chat. I’ll put my email in my bio
- kadoban 3y agoIs git conflict resolution "intuitive and easy to use not just for programmers, but also for lawyers and artists"? Several programmers I know can barely manage it. I think we might be setting our bar a little high here.
- jensenbox 3y agoIs there a pronunciation key for the name?
- joedrago 3y agoThis appears to be from the official devs on Twitter (and corresponds with how the speaker in this video says it): https://twitter.com/pijul_org/status/764116443550117889 https://twitter.com/pijul_org/status/764116443550117889
- adastra22 3y agoPijul is an actual species of bird, so I presume the way the bird is called?
- mk12 3y agoI recently found out about another project called jj: https://github.com/martinvonz/jj https://github.com/martinvonz/jj. It takes inspiration from Pijul and others but is git-compatible.
- xelxebar 3y agoCame here to praise jj as well. More than just "git-compatible", its backing store is literally git, which means that you can transparently collab with existing git repos to your heart's desire. Other interesting ideas it implements: - Working copy always comitted, - All repo operations are version controlled, which enables painless undo etc. - First-class conflicts, just like pijul, so no merge/rebase/etc ops every fail - Auto-rebase of downstream dependent branches when doing rebase etc. Also, the dev is really active and helpful. Apparently, Google let him make the project his full time job.
- TheAceOfHearts 3y agoTo the author: you should include a parragraph on the repo README explaining how pijul compares to git. Learning git kinda sucks, but once you learn it there's very little incentive to invest time in learning other version control tools. The extra features rarely justify the additional effort. Like I thought Fossil was kinda neat, but I still stuck with git.
- imiric 3y agoI'm not familiar with Pijul, and haven't finished watching this presentation, but IME the problems with modern version control tools is that they still rely on comparing lines of plain text, something we've been doing for decades. Merge conflicts are an issue because our tools are agnostic about the actual content they're tracking. Instead, the tools should be smarter and work on the level of functions, classes, packages, sentences, paragraphs, or whatever primitive makes sense for the project and file that is being changed. In the case of code bases, they need to be aware of the language and the AST of the program. For binary files, they need to be aware of the file format and its binary structure. This would allow them to show actually meaningful diffs, and minimize the chances of conflicts, and of producing a corrupt file after an automatic merge. There has been some research in this area, and there are a few semantic diffing tools[1,2,3], but I'm not aware of this being widely used in any VCS. Nowadays, with all the machine learning advances, the ideal VCS should also use ML to understand the change at a deeper level, and maybe even suggest improvements. If AI can write code for me, it could surely understand what I'm trying to do, and help me so that version control is entirely hands-free, instead of having to fight with it, and be constantly aware of it, as I have to do now. Or, since it's more than likely that humans won't be writing code or text in the near future, we'll skip the next revolution in VCS tools, and AI will be able to version its own software. /sigh I just finished watching the presentation, and Pijul seems like an iterative improvement over Git. Nothing jumped out at me like a killer feature that would make me want to give it a try. It might be because the author focuses too much on technical details and fixing Git's shortcomings, instead of taking a step back and rethinking what a modern VCS tool should look like today. [1]: https://semanticdiff.com/ https://semanticdiff.com/ [2]: https://github.com/trailofbits/graphtage https://github.com/trailofbits/graphtage [3]: https://github.com/GumTreeDiff/gumtree https://github.com/GumTreeDiff/gumtree
- zaphirplane 3y agoHow about a few for examples of how this semantic / AST is a game changer. An extra parameter is added to an existing function, how does this look Similar functionality are extracted to a parameterised new function, how does this look I’m sure diff and code review tools will evolve but it’s helpful for people to talk about how it looks like to make it less nebulous matrix second life
- vhodges 3y agoOn the theory of patches I found this set of posts helpful: https://jneem.github.io/merging/ https://jneem.github.io/merging/ TIL from the first 10 minutes (I didn't have time to watch the whole thing): I've been mis-pronouncing the name of pijul :)
- commandersaki 3y agoSounds elegant, lacks critical mass.
- beefman 3y agoThis product was announced at the end of the talk: https://nest.pijul.org/ https://nest.pijul.org/
- Nathanba 3y agoThe problem with pijul is that the command line interface is not really better than what git has. I tried pijul several times over the years and last month I couldn't even figure out how to make a commit and push it. I think it should be far more intuitive.
- hobofan 3y agoI really like Pijul and its underlying libraries, both from a conceptual and implementation perspective. However over the 6 years of its existence I've lost hope of it ever becoming something that I'll use in my day-to-day, the same way I lost hope on rocket (the Rust web framework). After all those years it's still essentially a single-maintainer side project who is stretched too thin across 3 big projects (Pijul, Sanakirja, Nest). I really hope that at some point a maintainer/team constellation emerges that can put all the lessons learned from Pijul into something that has a shot at being a successor to git.
- evgpbfhnr 3y agoI don't know if it's the server being slow, or inherent to the protocol, but `pijul clone https://nest.pijul.com/pijul/pijul https://nest.pijul.com/pijul/pijul` took an awefully long time (it doesn't say how much it downloaded, but the .pijul directory is 28MB -- which isn't a lot assuming it contains things similar to .git (gathering from the talk said it should contain diff + snapshot of each 'commit'))
- gonzus 3y agoI feel that git is the DVD of source control systems: sccs / rcs / cvs are the equivalent of tapes, clearly inferior and crappy. Other, more modern systems, are like BlueRay: technically superior, but not by a margin wide enough to make me want to change.
- baq 3y agoVery bad analogy if you consider how crap DVDs look like compared to even streamed H264 1080p content. If you wanted to compare full HD with 4k then I’d agree.
- osakr_1245 3y ago[flagged]
- akho 3y agoI think the Git problem that needs solution is, ahem, the problem people solve with extra tools when using Git? That is, forges (GitHub, GitLab, Gitea, …). Issues, PR discussions, &c need to be managed with the code (questions like “is issue #769798 fixed in branch xxx?” need to be answerable, code review discussions are almost as valuable as commit messages). Code needs to be actually distributed (forges introduce centralization). Local development should be possible. Patch theory is nice, but ultimately produces only a moderate improvement. I really like Fossil’s sensibilities.
- activitypea 3y agoGit's lack of centralization, i.e. a "source of truth" repo at the tool level is it's most imaginary, impractical benefit. Times where I've had to skip over origin and push to someone directly are few. Those occasions are far outweighed by a dozen annoyances that consistently come up with having to tell git "no trust me, origin is the remote I care about". Sure, you can fix it with aliases, but it's still misaligned of how most people work. I don't see the Linux kernel benefiting from it much, either.
- pastage 3y agoCan you perhaps give a better example when you need to say that origin is what matters for you? I seldom have these problems myself, but there is a nagging feeling that this is something I have thought about my self.