9 ms·
SQLAlchemy Migrated from Mercurial to Git
- octo_t 13y ago> Git manages the size of the repository more efficiently; while the Mercurial repository has been approaching 50M in size, the Git repository is only 17M. 17MB vs 50MB - almost a third of the size. That is definitely quite impressive.
- micampe 13y agoI’m not the biggest git fan and I still think it’s right choice, but how is this even a reason to switch? Unless you go in (older) Subversion ridiculousness how are 40MB of disk space even a concern? They even listed it first. At best it would be Oh, and we saved 40MB, the size of two raw pictures from a digital camera.
- myth_drannon 13y agoI would assume people cloning the repository on slower connections.
- zzzeek 13y agoit's the time it takes to clone. Also with git I need to clone a lot less since I can create local feature branches that I can delete if they are abandoned. edit: quick speed test, git clone from my server = 17.4 seconds, hg clone from the same server's hg repo = 25.4 seconds
- rbehrends 13y agoTime to clone is not directly a function of absolutely repository size. Both Git and Mercurial use hardlinks for local clones by default, so the reason why Mercurial is slower than Git for local clones is primarily that its repository data generally contains more files. Cloning/pulling/pushing speed across a network is in large part determined by the protocol used; a few years ago, Git's network performance was inferior to that of Mercurial [1]. I believe that has been largely fixed since then. Finally, Mercurial allows you to have local feature branches that you can delete if they are abandoned just fine. [1] https://code.google.com/p/support/wiki/DVCSAnalysis https://code.google.com/p/support/wiki/DVCSAnalysis -- footnote 1
- jedbrown 13y agoYour citation is comparing the "dumb http" protocol, which was replaced several years ago with the smart http protocol in git-1.7. Dumb http was only ever provided as a method of last resort. The git:// and ssh protocols have always been fast. Another data point: when we converted to Git, I did a number of speed comparisons. Our repository was 77MB in Git versus 178 MB in Mercurial. Clone time from bitbucket over either (smart) http or ssh was 18 seconds with Git, versus 2 minutes with Hg. We can do a shallow clone (--depth 1) in 4 seconds (10 MB transferred) with Git, but Hg has no comparable feature.
- rbehrends 13y agoThe speed was also slower for the native protocol if you read the footnote to the end. Also, as I noted, this was years ago. I was making a point about cloning/pulling/pushing speed being dependent not on just the repository size, not about the relative superiority of one or the other tool [1]. [1] I find both Mercurial and Git adequate, but lacking in some aspects that are important to me (both with respect to architectural design and workflow considerations). For practical work, I consider the differences between Mercurial and Git to be relatively minor in comparison and cannot really get exercised over them.
- cmccabe 13y agoIs it really helpful to provide numbers that you know are many years out of date? If you're really interested in this, try to import a big repository like, say, the Linux kernel using the latest version of Mercurial and see how it compares to the latest version of git.
- rbehrends 13y agoI was making a point about what factors influence clone performance in general, not trying to contribute to the tedious Git vs. Mercurial debate. If I had found any data about, say, Gnu Arch vs. Codeville (or some other abandoned codebase), I would have used that instead.
- bobz 13y agoWhen you consider that distributed repositories are going to be cloned dozens, hundreds, or thousands of times, it starts to add up. Even if this larger project could handle that bandwidth, it is a significant factor for smaller projects, or larger projects like github. Meaning more of a chance that git remains the dominant choice.
- jonknee 13y agoThe repo is hosted by third parties, bandwidth is not an factor (it's all text in any event so the compression factor is huge). There are actual differences worth talking about, file size is not one of them.
- plorkyeran 13y agogit repositories aren't giant masses of text files; they're compressed on disk and the additional gains from compressing them with lzma are minimal (12 MB on a 306 MB git repo I have lying around). I assume Mercurial does something similar as the difference would be a lot larger than 17 vs 50 MB if not.
- etanol 13y agoThe size issues of the SQLAlchemy repository come from the way Mercurial handles copies and renames. I prefer Mercurial because is much easier to use but this file rename issue always make me feel uncomfortable when reorganizing code. These days I'm giving Fossil a try, which still is easier to use than Git and the repository size sits between Git and Mercurial.
- gbog 13y agoFrom the other side of the mirror, I mean from the position of someone used to git, this comment seem weird. Git is not hard to use. It is adding a few articulations in your workflow, and they are just allowing you to run faster. One example: interactive staging with git add -p, this articulation masks it much easier to debug: add print all over the place, try some tweaks, find the one, stage this one snippet, checkout the files, run the test, and you're done.
- tonfa 13y agoIf that was an actual issue (I doubt it is), they could have done a oneoff repack server side (reordering changesets to optimize compression). Edit: I think the third point would have been enough, network effect for free software is important. Size (reorder your repo if it matters), branching (use bookmarks, not named branch unless you know what your are doing), and history rewriting (use evolve) are dubious points.
- SEJeff 13y agoVery smart move. Some time ago Riak did the same thing. Their rationale was more detailed, but came to the same conclusion: http://basho.com/a-few-more-details-on-why-we-switched-to-github/ http://basho.com/a-few-more-details-on-why-we-switched-to-gi...
- etanol 13y agoToo bad there are no comments on the blog: SQLAlchemy's issue repository will remain hosted on Trac; while a Git repository can be mirrored in any number of places, an issue repository cannot (for now! Can someone please create a distributed issue tracker? Should be pretty doable, though getting Github/Bitbucket to use it, not so much...), so SQLAlchemy's long history of issue discussion remains maintained directly by the project. http://fossil-scm.org http://fossil-scm.org !!!
- zzzeek 13y agouh yikes! http://fossil-scm.org/index.html/dir?ci=601c15421a4a5ca5&name=src http://fossil-scm.org/index.html/dir?ci=601c15421a4a5ca5&... although good point, I see they're doing distributed. will take a look at how they approach that.
- etanol 13y agoI think you can customize most parts of the Web UI: http://www.fossil-scm.org/fossil/wiki?name=Cookbook#css http://www.fossil-scm.org/fossil/wiki?name=Cookbook#css
- qznc 13y agoThere are tools like Bugs Everywhere [0], which mix well with distributed VCS. However, the question is if distributed bug tracking makes sense. [0] http://bugseverywhere.org/ http://bugseverywhere.org/
- travisb 13y agoThere are many different projects which implement distributed bug tracking[0], some better than others. The question of how much distributed bug tracking makes sense depends strongly on the structure of the project. If the project is developer heavy (such as developers responding to user help requests) it can work well, but if the project has several strata of developers and support people and users then it might not make as much sense. [0] http://travisbrown.ca/blog.html#TooMuchAboutDistributedBugTracking2013-04-20-83 http://travisbrown.ca/blog.html#TooMuchAboutDistributedBugTr...
- trosenbaum 13y agoWould something like Kiln Harmony work for storing the main upstream repository? https://secure.fogcreek.com/kiln/ https://secure.fogcreek.com/kiln/ It seems that if KH translates seamlessly between HG and Git that a project could accept pushes from both, right? I assume since KH is not open source / free that this would not be an acceptable method for maintaining the main repo, or that the translation would introduce interesting wrinkles, but it does seem like one could potentially have his cake and eat it too.
- qznc 13y agoThanks to hg-git, Mercurial users should have no problem using the git repository.
- gecko 13y agohg-git and Kiln Harmony serve different, though overlapping, purposes. hg-git allows people who know both Git and Mercurial, but who prefer Mercurial, to work with Git repositories from Mercurial. No effort is made to hide the Git model, and no effort is made to ensure that the Mercurial repository generated from the Git one is idempotent (i.e., a given Git repo will always generate the same Mercurial repo) or can round-trip (i.e., I can trivially craft repos that hg-git can work with just fine, but where pushing to a bare Git repo will result in a different Git repository from the original). The benefit to this model is that the Git users don't need to do anything different, ever, and no one needs to know that you were using Mercurial, ever. Kiln Harmony does something different: it's designed to let a team use whatever tools they want. This means that Mercurial users don't have to learn the Git model, Git users don't have to learn the Mercurial model, and generally, everything "just works" in that situation. Doing that requires a lot more processing power than hg-git requires, which is part of why we only offer it as a hosted solution for the moment. It also works best when your central repository is a Kiln Harmony repository. I obviously think that's a fine trade-off, but if part of the main motivation of SQLAlchemy was GitHub, then Kiln Harmony probably isn't a good solution.
- deleted 13y ago[deleted]
- zalew 13y ago> Largely due to the popularity of Github, Git has achieved a much higher userbase, to the degree where we regularly have users requesting us to move to Git so they can provide pull requests. I am too a Mercurial refugee and prefer Git, but how is that even a reason? People don't want to contribute if it's on a Mercurial repo?? Python seems to be doing fine on HG. Meanwhile, thanks for your amazing work on SqlAlchemy(&co), zzzeek!
- gamache 13y agoI think what those users mean is that they want the project on Github. There's no concept of "pull requests" within Git. Anyway, most Git users aren't Mercurial refugees; I wager we're Subversion refugees and kids.
- deleted 13y ago[deleted]
- CmonNoReg 13y agoWhich means (in most cases) "I did not do a conscious choice based on reviews and comparisons and just followed the hype.". A bit sad.
- zalew 13y ago> There's no concept of "pull requests" within Git. http://git-scm.com/book/ch5-2.html http://git-scm.com/book/ch5-2.html "run the git request-pull command and e-mail the output to the project maintainer manually." http://www.wired.com/wiredenterprise/2012/05/torvalds_github/ http://www.wired.com/wiredenterprise/2012/05/torvalds_github... "Git comes with a nice pull-request generation module, but GitHub instead decided to replace it with their own totally inferior version." > I think what those users mean is that they want the project on Github. that would be even worse. luckily as zzzeek explained below, it wasn't the case.
- gamache 13y agoWow. I had no idea about git pull requests, so I even googled it, and didn't see it on page one. My mistake, thanks!
- k_bx 13y agoCould someone please explain me an argument about removing branch in git vs closing it in mercurial? As I understand, when you "remove" branch in git, just as when you close it in mercurial, it doesn't delete commits from tree either, since it can lead to disaster (if those commits were pushed already). So someone else could actually start new branch from commit which is in branch you just "deleted". Am I wrong here?
- 0x0 13y agoIf there are branches with commits that aren't referenced elsewhere, i believe deleting the branch will eventually cause the commits to be "garbage collected". So you can delete abandoned experiments and keep a clean house. On the other hand, commits that are referenced elsewhere (perhaps merged into a main branch) will stay. But you could still delete the branch it was merged from, and avoid having the list of branches grow huge.
- jedbrown 13y agoYou can delete commits from Hg using 'strip' (enabled via the 'mq' extension). But this is strict deletion (as opposed to marking it for gc after a grace period) and you can't push an "unreference this commit", so you need admin access on the remote to strip a commit there. If you push branches to bitbucket for review, the abandoned commits stick around as anonymous heads until you strip them via the web interface, but they still stick around for anyone that pulled from you. The Hg community is working on a system [1] for dealing with this, but it's still not ready for general use and I find it significantly more complicated than Git's model. [1] http://mercurial.selenic.com/wiki/ChangesetEvolution http://mercurial.selenic.com/wiki/ChangesetEvolution
- reinhardt 13y agohg strip saves the stripped commits under .hg/strip-backup as a bundle.
- plorkyeran 13y ago> So someone else could actually start new branch from commit which is in branch you just "deleted" Well yes, of course. They could also start a new branch from any other commit that they happen have in their local repository. Branches aren't really 'things' in git. They're just a pointer to an arbitrary commit that automatically gets updated when you make new commits. It's rarely a useful thing to do, but you don't even have to be on a branch to make commits.
- ezquerra 13y agoEvery project has the right to chose the VCS that they feel is right for them. However I find the reasons they give for the switch are pretty weak except for the last one (github is more popular). 1. Repository size: Such size differences are uncommon, and mostly likely due to the fact that the git repo was a brand new repo created from scratch by converting the original hg repo. If they repacked the mercurial repo it would probably be greatly reduced in size. They could also do: hg convert --config "extensions.convert=" --branchsort sqlalchemy sqlalchemy-smaller Which results in a reduction of the repository size from 45 MB in the original to 24 MB in the converted repo. This is not as good as git but quite close IMHO. 2. This is not correct: Mercurial bookmarks _used_ to be an extension (a long time ago!) but they have been built into mercurial for a long time now. They are basically equivalent to git branches. So nowadays this is just a process issue. If using named branches is a problem just tell your devs to not use them and use bookmarks instead! 3. Mercurial's rebase and mq extensions are part of core mercurial. They are packaged and distributed with mercurial, their test suites are part of the mercurial test suite and they are supported and developed by the core mercurial developers. The only difference between a core command and one of the official extensions is that these must be manually enabled to be user (by adding a line to the mercurial config file). This is a good thing! This makes it harder to mess up your repo if you do not know what you are doing. If you are not knowledgeable enough to find and edit the mercurial config file you should definitely not be using rebase or collapsing your changesets. The last reason on the other hand is one that I can understand and which I think has merit. Is it worth the hassle of changing their DVCS system? Perhaps. However it bothers me when legitimate _social_ reasons to switch from mercurial to git are supported by not quite as clear cut technical reasons such as the ones put forward in this blog post.
- zzzeek 13y ago> If using named branches is a problem just tell your devs to not use them and use bookmarks instead! but they do, because all the devs know git and are only using mercurial because they want to send you a pull request, and now their feature branch is forever. Mecurial is unforgiving of mistakes - once in the repo, they are forever. As for MQ, I've really tried to understand it, but it's just profoundly awkward IMHO. Mercurial's main benefit is that it's much easier to use and understand than git, so if I'm going to need to fire up the extra brainpower, I might as well learn git where these advanced patterns are extremely commonplace, and are part of the core functionality, not as a series of optional extensions that have been bolted on over the years. I've almost never seen anyone using MQ or rebase with mercurial (or even bookmarks for that matter). I've tried carefully in my post to express that my points are all based on an overlap of social with technical issues. I am aware that technically, hg has answers for all these things. But these answers don't pan out in the real social world, for better or for worse. If git didn't exist, I'm sure there would be a much larger knowledge share for mercurial, and I was even betting on that in the beginning (certainly the "easier" tool will be more popular?) but it hasn't turned out that way.
- Ziomislaw 13y agoTL;DR - we didn't read the HG manual, so we had to change as we did't know how to use its features.
- develop7 13y ago1. What a huge saving — 0,002¢ per repo copy. Don't spend it all in one place. 2. Bookmarks is core feature since March 2011. This basically means those who pushed "move to Git" decision, haven't checked if anything was updated in Mercurial (and with bookmarks in particular) since then at least. This is pretty understandable — it is known that only software gets updated is one you're paying attention to, others don't. 3. As ezquerra mentioned, those extensions Mike is probably referring to, are shipped with Mercurial, and are enabled with only single line added to .hgrc. Not mentioning those extensions do not _emulate_ Git features, they _reproduce_ it. 4. Only reason makes sense.