22 ms·
Mercurial 4.0 Sprint Notes
- steveklabnik 10y agoRelevant quote: > Facebook is writing a Mercurial server in Rust. It will be distributed and > will support pluggable key-value stores for storage (meaning that we could > move hg.mozilla.org to be backed by Amazon S3 or some such). The primary > author also has aspirations for supporting the Git wire protocol on the > server and enabling sub-directories to be git cloned independently of a > large repo. This means you could use Mercurial to back your monorepo while > still providing the illusion of multiple "sub-repos" to Mercurial or Git > clients. The author is also interested in things like GraphQL to query repo > data. Facebook engineers are crazy... in a good way.
- marcinkuzminski 10y agoCTO of RhodeCode here. This is exciting, we need to take a look from our side how this would affect our product. I really like that Mercurial is gaining some traction with the big guys, which tries to solve some nice problems at scale.
- steveklabnik 10y agoOver here: https://www.mercurial-scm.org/wiki/4.0sprint https://www.mercurial-scm.org/wiki/4.0sprint > - Goal is to open source the server, once it's more than just slideware. So you might not be able to look at it just yet, but hopefully soon?
- marcinkuzminski 10y agoHopefully ! We spend a lot of time on our own to scale our Mercurial backend. Currently with the http based vcs-server and gevent we can support a lot of concurrent hg operations, but imho that thing can put it on the next level... I wonder if it will support all things like phases etc ootb.
- Stratoscope 10y agoA suggestion if I may... Don't use monospaced formatting (two-space prefix) as a way to put a '>' on every line of a long quote, as it becomes unreadable on mobile. One '>' at the beginning of each paragraph is good enough (with blank lines to separate paragraphs), or italics are fine too. Here's a copy that should be readable on any device: > Facebook is writing a Mercurial server in Rust. It will be distributed and will support pluggable key-value stores for storage (meaning that we could move hg.mozilla.org to be backed by Amazon S3 or some such). The primary author also has aspirations for supporting the Git wire protocol on the server and enabling sub-directories to be git cloned independently of a large repo. This means you could use Mercurial to back your monorepo while still providing the illusion of multiple "sub-repos" to Mercurial or Git clients. The author is also interested in things like GraphQL to query repo data. Facebook engineers are crazy... in a good way.
- steveklabnik 10y agoThis is offtopic so I'll leave it to just this one reply, but thanks! Lack of quoting in HN's markdown is the only part I have actual frustration with it; I usually use the two-space + wrap format because it makes it much more clear that it's a quote; I find your version hard to tell. But that said, I checked on my phone, and I see what you're saying about mobile. Ugh.
- pohl 10y agoHN markdown does support italics, so italicizing entire paragraphs is a viable alternative to differentiate what you're quoting from your own comments. I've seen folks do it now & then.
- vvanders 10y agoThat's exciting, sounds similar to the git fusion stuff Perforce is doing. I'd be happy to see a bit more diversity in source control. Git is great but falls over in more than a few scenarios(unmergeable large binary files come to mind).
- pjmlp 10y agoCongratulations, this is the kind of stuff that will improve Rust's adoption.
- glandium 10y agoIt's interesting they're doing this in Rust. I would have expected them to do it in D.
- arunc 10y agoWarp [0], which was a huge marketing win for DLang is not maintained anymore. Not sure if it is being used at all. Will there be any D projects in Facebook? [0] https://github.com/facebookarchive/warp https://github.com/facebookarchive/warp
- the_duke 10y agoI had no idea Google and FB were dabbling with Mercurial. I checked it out years ago, but pretty much settled on Git. What are the advantages?
- fra 10y agoThey wrote about it here: https://code.facebook.com/posts/218678814984400/scaling-mercurial-at-facebook https://code.facebook.com/posts/218678814984400/scaling-merc...
- jcranmer 10y agoThe big advantage to call out is that they found Mercurial much easier to add extensions to and more receptive to their issues with large monorepos. (For an idea of the scale, Facebook's repository I have heard is ~10× the size of mozilla-central, which is probably the largest public Mercurial repository.)
- lucideer 10y agoThe biggest technical difference is the mostly immutable history, which is a feature or a drawback depending on who you talk to. More subjectively, most people I've chatted to about it seem to find Mercurial's interface much easier to grok / pick up as a new user than Git's (which is somewhat notorious for its quirks). There are other differences, but these stand out to me. That said, I use Git because adoption + community (and my experience with hg-git has been less successful than some... though it's been a while since I gave it a spin)
- marcinkuzminski 10y agoCTO of RhodeCode here. If you use phases (draft/secret/public) correctly it's really close to being mutable. For example all our devs forks are having non-publishing repositories since it's a private non-shared space. By keeping all commits as draft it's easy to do a rebase and then push new changed commits. While the main repo is publishing and once pushed commits are never mutable, also we keep a workflow that only dev forks can have multiple heads while production repo cant. IMHO it's the nicer form of mutability
- agentgt 10y agoI know this is slightly tangental but I'm always a little shocked that Facebook (and I think Google to some extent) have massive mono repositories. The benefits of having one repository do not seem to be worth the serious performance issues as well as potential coupling that can happen with a gigantic code base as well also making much more difficult to OSS certain parts. e.g. why doesn't FB use dependency management (or binary package management aka maven, npm, etc)? That is have multiple repositories and cut releases. Build tools to help cut and manage the releases and dependency graph of the releases. There are even plugins and shell scripts that will make Mercurial act like a mono repository for many small repositories (I use it for our own Java Maven code base). I must be missing some killer features and would love to see it in action (FB major repository).
- steveklabnik 10y agoI haven't worked on a monorepo, but I will say that Facebook _does_ use at least some of these tools, apparently: enough so that they created a whole replacement for the npm client, yarn. From talking to some companies that use monorepos that want to use Rust, they _do_ use these kinds of tools, but want support for it in the tool. Firefox, for example, is pretty much a monorepo, and we built tooling in Cargo to help. I think that part of it is that "monorepo" can mean slightly different things. Does it only contain your companies' code, or also your dependencies? Do you modify those dependencies in-repo?
- agentgt 10y agoYeah I'm not even sure I understand what FB's monorepository is so it is probably unfair critique. I can definitely see an "app" (Firefox) as a monorepo but not giant web companies (particularly with the surge of microservices). BTW Cargo is fantastic (and I'm fairly picky on package management / build tools).
- gcp 10y agoOSSing certain parts is certainly an issue, but what do you mean with "potential coupling" being an issue? That's exactly the advantage, right?
- shoover 10y agoThat was a busy meeting and the developer mailing list is very busy. It's great to see continued investment from so many interested parties. Judging by the notes in the wiki [1], however, the purveyors of my preferred server, Kiln, are not so engaged lately: > Available hosting solutions: Bitbucket, Kallithea (self-hosted), Kiln (still exists?) I believe it is maintained and even if not maintained would continue to work for ages, but I suspect we will be held to 3.x for a while. [1]: https://www.mercurial-scm.org/wiki/4.0sprint https://www.mercurial-scm.org/wiki/4.0sprint
- marcinkuzminski 10y agoWhat's you best KILN feature ? I wonder if we can adopt it at RhodeCode
- shoover 10y agoHmm. I haven't looked at RhodeCode. For me Kiln has provided a good UI and workflow and just worked ever since their v2.4 or so. I'm not sure there is a particular best feature, but roughly these features are what come to mind: - Code review UI including comment pane with one level of sub-threading and easy click and drag linking to lines of code, small changeset selection pane, and large scrolling code pane are simple and work well. By contrast, I'm less sure about github and bitbucket pull requests UI. - largefiles - Single sign-on and linking with FogBugz (if we reference a FogBugz case in a commit message, then the commits show up in the case notes)
- marcinkuzminski 10y agoThanks for sharing, actually those two first ones was our main focus from beginning. We worked back in the days with Unity which was using kiln to make the code-review workflows similar. If you miss kiln, you should check RhodeCode out, and our community edition is even open-sourced.
- PretzelFisch 10y agoKILN and Fogbugz have a nice integration.
- cm3 10y agohg absorb sounds very useful. I'd like to know details, since I can imagine undesired results it might produce.
- mkj 10y agoLooks like it uses annotate information, seems pretty handy. https://bitbucket.org/facebook/hg-experimental/src/default/hgext3rd/absorb.py https://bitbucket.org/facebook/hg-experimental/src/default/h...
- drewg123 10y agoWhat about speed? Our team just moved from hg to git for an enormous project that has ~25 years of history (CVS -> SVN -> hg|git). The biggest improvement to my daily life is that a git pull takes seconds, while an hg pull takes minutes (or even large fractions of hours when I've spent a week or two away from work).
- PretzelFisch 10y agoYou don't have much history in git at the moment.
- AbacusAvenger 10y agoDidn't he say he imported a repository with 25 years of history?
- drewg123 10y agoWe have 25 years of history. The initial clone time for hg and git is within the same order of magnitude (order of an hour), though git manages to be about 50% faster at that too.
- rjbwork 10y agoWhere were and are you hosting it? SSH vs HTTPS? Client? OS? Any hooks?
- drewg123 10y agoWe were hosting both on an ESX VM and clones / pulls were done via SSH. OS was FreeBSD on both ends. Hookless. Part of the reason for the move is that we wanted to take advantage of the corp-wide infrastructure of an Atlassian stash server hosted in the cloud and professionally maintained, so as to get away from maintaining our own repo. But the speeds I quote above were for the initial phase of the conversion, when both repos ran on the same ESX VM and direct comparisons were meaningful. Now that its hosted professionally in the cloud, it seems even faster. Note that I'm across ~2500 miles of VPN from the home office, and that surely has something to do with it.
- bedros 10y agoanyone knows if mercurial subrepo feature is used by big corporations
- ksec 10y agoSo is this a comeback of Mercurial? I know Subversion is pretty much dead, Mercurial was sidelined and Git seems to have conquered the world. The only thing I think would shake things up a little would be Perforce going Open Source. But i dont see that happening as they seems to be very comfortable in their niche.