6 ms·

Exercise: what are the downsides of the current generation of DVCS ? (Assume you may be something other than a coder that carries all files in a laptop.)

by ableal 17y ago

chrisbolt 17y ago
Inability to check out a subset of a repository.
- dryicerx 17y ago
  For this reason, I am loving svn-git
- snprbob86 17y ago
  There are two good reasons to checkout only a subset of the repository: 1) Large binary or data files -- For this, there are various solutions of varying hackiness. 2) Your project has grown into several sub-projects -- The best solution here is to clone the repository to new names and then refactor and trim the respective project trees with a lot of deleting. You'll thank yourself later for improving your architecture and there will no longer be a partial check-out problem. The real issue for both of these, is handling unversioned or independently versioned bits with respect to all the other bits. You might want some unversioned binary blobs, but you might also want some versioned giant csv files without having to download all the old versions all the time. Source code has one versioning strategy, data files another, photoshop files yet another. No version control system today really lets you pick which versioning strategy to use for which files. And none of the existing "sub-module" type extensions are really great either.
  - viraptor 17y ago
    3) You have a small patch and want to update it to the HEAD version before submitting. You worked with the source you obtained in some other way (from a source package for example). Now you have to wait 10+ minutes on a large project, downloading all the changes since 1970, just to change the newest version of a single file. 4) The project has grown into several sub-projects - but you don't control that repository.
    - cma 17y ago
      re #3: "shallow repository A shallow repository has an incomplete history some of whose commits have parents cauterized away (in other words, git is told to pretend that these commits do not have the parents, even though they are recorded in the commit object). This is sometimes useful when you are interested only in the recent history of a project even though the real history recorded in the upstream is much larger. A shallow repository is created by giving the —depth option to git-clone(1), and its history can be later deepened with git-fetch(1)." There are still issues; you can't commit for instance, but you can update to HEAD to update your patch, with a depth of 1 you only get the most recent changes.
  - alextp 17y ago
    Actually, the worst problem with submodules in modern DVCSs is handling dependencies. If project A depends on project B and project B and the API exposed by project B varies over time it is pretty easy to keep both in sync and working. Except, when someone needs to check out an older version of A they need to figure out which version of B to check out (and it might not be as easy as checking out the latest B in the date that that version of A was committed). Git allows you to mark a specific revision of a submodule, but then updating the whole project is no longer a single command, and there are some problems with not pushing everything at the same time.
  - sandGorgon 17y ago
    http://mercurial.selenic.com/bts/issue105 http://mercurial.selenic.com/bts/issue105 Two of the features that made it difficult to 'sell' Hg for the main SCM of FreeBSD were the lack of Partial checkouts and Partial history
- dasil003 17y ago
  I agree this is a definite weakness compared to svn, but the problem is that svn gains this facility by having no concept of a project. Let me repeat that: in svn it is technically impossible to identify the project tree. The problem with that, of course, is that merging only makes sense if you are merging branches that occur at the same tree-level in a project. This is why subversion merge tracking is so buggy and half-baked... because any given directory could be a project tree, a subtree, a branch, or a tag, and you could merge any of them. Sure if stick to certain conventions it works pretty well, but technically the whole implementation is a minefield, which is why svn will never be as robust as other systems that don't make this mistake. Even though git's submodules and subtrees leave a lot to be desired, and have significant room for improvement, they will never be as convenient for partial checkouts, because the requirements for svn-style partial checkouts require crippling the entire system.
- prog 17y ago
  bzr has support for filtered views[1] if thats what you mean. [1] http://doc.bazaar.canonical.com/latest/en/user-guide/filtered_views.html http://doc.bazaar.canonical.com/latest/en/user-guide/filtere...
yummyfajitas 17y ago
Lack of a good way to edit recent history. Suppose I sit down for a collaborative session with someone. Over the course of a few hours, we might generate 10-100 commits between the two of us, and several branches, all to implement a single feature. I.e., I commit, say "hey, pull from me, you had an off by one a few minutes ago." At the end of the session, we only want to generate a single commit to the public repositories. Mercurial queues are a tolerable way to handle this, but they aren't great.
- nuclear_eclipse 17y ago
  Git makes this easy. It supports "rebasing" commit trees, and for what you want, a "squashed" rebase is absolutely trivial to do.
- vtail 17y ago
  git rebase --interactive <commit> should help http://stackoverflow.com/questions/598672/git-how-to-squash-the-first-two-commits http://stackoverflow.com/questions/598672/git-how-to-squash-... and http://stackoverflow.com/questions/435646/how-do-i-combine-the-first-two-commits-of-a-git-repository/436530#436530 http://stackoverflow.com/questions/435646/how-do-i-combine-t...
- dalore 17y ago
  I do this all the time with git. I create a branch, do all my incremental commits, get it all working. Then I merge it into the mainline as one squashed commit. There a few guides on how to do it with the command line but I'll admit I've gotten lazy and now just use smartgit which lets me do it easily without having to remember the commands.
- prog 17y ago
  git already has this in core and bzr has a rebase plugin[1]. I think this problem has been solved for quite some time now :) [1] http://doc.bazaar.canonical.com/plugins/en/rebase-plugin.html http://doc.bazaar.canonical.com/plugins/en/rebase-plugin.htm...
nuclear_eclipse 17y ago
Poor or inefficient support for large binary assets, such as compiled objects, images, or video. Especially in the context of video game development, binary assets that constantly change are highly painful to deal with in Git or Mercurial.
- naner 17y ago
  How is it possible that nobody has dealt with this problem yet?
  - pmjordan 17y ago
    Active open source development rarely meets "AAA" game development.
    - sshumaker 17y ago
      One could argue that the traditional source control model (where everyone has a copy on their own machine) isn't a great way to deal with huge binary assets. I've worked on game projects where the binary churn was on the order of gigabytes a day. Even though everyone was physically co-located, when everyone arrived at 10am and started to update to head, a hundred computers all trying to download the days worth of 2 gigs of assets over the network didn't make for a pretty picture. You can try to cronjob the updates, staggered around 4am. Or you can realize that the majority of people are only modifying a subset of these assets, and roll your own link-based asset manager for large binaries.
      - pmjordan 17y ago
        Yes, absolutely, you wouldn't want everyone to keep pulling all data, git-style. The programmers usually only need the assets in the game-ready format, not the source models, textures, etc. which are typically much larger, and only needed by the artists/level designers working with them.
  - ableal 17y ago
    I have been trying to point it out [1], without much luck. Today I got in early by chance, and decided on the Socratic approach ;-) [1] http://news.ycombinator.com/item?id=1201559 http://news.ycombinator.com/item?id=1201559, http://news.ycombinator.com/item?id=1219082 http://news.ycombinator.com/item?id=1219082, http://news.ycombinator.com/item?id=1222905 http://news.ycombinator.com/item?id=1222905, http://news.ycombinator.com/item?id=1242374 http://news.ycombinator.com/item?id=1242374
ionfish 17y ago
No way to move files between repositories and retain their history. Luke Palmer talked about this a little here: http://lukepalmer.wordpress.com/2008/11/12/sketch-of-udon-version-controlpackaging-system/ http://lukepalmer.wordpress.com/2008/11/12/sketch-of-udon-ve...
- windsurfer 17y ago
  git filter-branch --subdirectory-filter foodir -- --all
- blasdel 17y ago
  It is absolutely possible to do this in git. It helps if you forget about "files" as something to "move". I did it in the project I'm working on now! It originally had one main repository, and another for an extension that was used as a git submodule in the main one. Active development proceeded in both, with shitloads of commits in the main one just to update the reference to the submodule. I decided this was retarded, so I took a clone of the extension repository, used filter-branch to rewrite it's entire history so that all the paths were always prefixed with "vendor/extensions/project_name/", and pushed it as an unrelated branch into the main repository. Then in the main repository I made a new branch, removed the old submodule junk there, did a nice clean merge that melded all the commits from the extension branch going back in time, and made that the new master branch.
koenbok 17y ago
- When you work with large binary files that don't diff well (psd), it's a pain to have the entire repository history local. - Hard to explain to even fairly technical designers/copywriters. - With all that easy branching, I sometimes just have a hard time getting a decent version of the codebase together from everyone.
- dustingetz 17y ago
  i used SVN to track large binaries and it was morbidly slow. maybe i was doing it wrong. i thought the solution was to keep binary assets under a separate version control better suited for files that don't diff; a nightly snapshotted filesystem seems just fine.
  - ableal 17y ago
    Seems that's why Perforce is in business (e.g. http://news.ycombinator.com/item?id=1242374 http://news.ycombinator.com/item?id=1242374 )
th 17y ago
Complexity (in terms of the operations that need to be performed regularly). At least with Git, simplicity seems to be traded in favor of flexibility.
- prog 17y ago
  bzr, hg and darcs are not too complex to use.
_delirium 17y ago
Well, a big one is probably the same one: politics. Instead of politics over commit privileges and commit rules, you have politics over who controls mainline, who controls which staging areas, what their rules are for pulling or accepting pushed patches, etc.
- bdonlan 17y ago
  The point is with centralized systems, only committers can do things such as branching or preparing commits. With a DVCS, everyone gets the same tools, so the rights granted by having the patch accepted are simply having it merged, nothing more or less.
  - _delirium 17y ago
    That's true, and I think it is an improvement when it comes to the ability to do local forks. But most of the big fights I've seen in CVS/SVN-based projects ultimately come down to who controls mainline, and a DVCS doesn't solve that problem at all. If anything, it makes it easier to put even more politics on top of it, where instead of having a fight over getting into the single repository, you now have to navigate whole cascading set of repositories (e.g. Linus doesn't generally pull directly from you, so you have to deal with sub-gatekeepers before you get to the main gatekeeper). I suspect Linus likes the approach not because it has no politics, but because it encourages the politics that match the way he'd like to manage the Linux kernel development, with this style of hierarchical, cascading commit approval, which is pretty hard to set up in CVS (though Mozilla's sort of grafted it on via their Bugzilla, which keeps track of cascading approvals of patches based on who owns which areas and sub-areas). Not that that's necessarily a bad thing, it's just different (and for a project as large as Linux, probably necessary).
jedediah 17y ago
Terrible submodule support.
cool-RR 17y ago
I would want something pretty superficial: An ability to organize branches in folders.
- maw 17y ago
  bzr can do that with its shared repos. It's not too hard to fake it with git, either. (This assumes you know how to use git, which without a doubt is hard, despite what its apologists may tell you.)

Exercise: what are the downsides of the current generation of DVCS ? (Assume you may be something other than a coder that carries all files in a laptop.)

by ableal 17y ago

Exercise: what are the downsides of the current generation of DVCS ? (Assume you may be something other than a coder that carries all files in a laptop.)

chrisbolt 17y ago

Inability to check out a subset of a repository.

dryicerx 17y ago
For this reason, I am loving svn-git
snprbob86 17y ago
There are two good reasons to checkout only a subset of the repository: 1) Large binary or data files -- For this, there are various solutions of varying hackiness. 2) Your project has grown into several sub-projects -- The best solution here is to clone the repository to new names and then refactor and trim the respective project trees with a lot of deleting. You'll thank yourself later for improving your architecture and there will no longer be a partial check-out problem. The real issue for both of these, is handling unversioned or independently versioned bits with respect to all the other bits. You might want some unversioned binary blobs, but you might also want some versioned giant csv files without having to download all the old versions all the time. Source code has one versioning strategy, data files another, photoshop files yet another. No version control system today really lets you pick which versioning strategy to use for which files. And none of the existing "sub-module" type extensions are really great either.
- viraptor 17y ago
  3) You have a small patch and want to update it to the HEAD version before submitting. You worked with the source you obtained in some other way (from a source package for example). Now you have to wait 10+ minutes on a large project, downloading all the changes since 1970, just to change the newest version of a single file. 4) The project has grown into several sub-projects - but you don't control that repository.
  - cma 17y ago
    re #3: "shallow repository A shallow repository has an incomplete history some of whose commits have parents cauterized away (in other words, git is told to pretend that these commits do not have the parents, even though they are recorded in the commit object). This is sometimes useful when you are interested only in the recent history of a project even though the real history recorded in the upstream is much larger. A shallow repository is created by giving the —depth option to git-clone(1), and its history can be later deepened with git-fetch(1)." There are still issues; you can't commit for instance, but you can update to HEAD to update your patch, with a depth of 1 you only get the most recent changes.
- alextp 17y ago
  Actually, the worst problem with submodules in modern DVCSs is handling dependencies. If project A depends on project B and project B and the API exposed by project B varies over time it is pretty easy to keep both in sync and working. Except, when someone needs to check out an older version of A they need to figure out which version of B to check out (and it might not be as easy as checking out the latest B in the date that that version of A was committed). Git allows you to mark a specific revision of a submodule, but then updating the whole project is no longer a single command, and there are some problems with not pushing everything at the same time.
- sandGorgon 17y ago
  http://mercurial.selenic.com/bts/issue105 http://mercurial.selenic.com/bts/issue105 Two of the features that made it difficult to 'sell' Hg for the main SCM of FreeBSD were the lack of Partial checkouts and Partial history
dasil003 17y ago
I agree this is a definite weakness compared to svn, but the problem is that svn gains this facility by having no concept of a project. Let me repeat that: in svn it is technically impossible to identify the project tree. The problem with that, of course, is that merging only makes sense if you are merging branches that occur at the same tree-level in a project. This is why subversion merge tracking is so buggy and half-baked... because any given directory could be a project tree, a subtree, a branch, or a tag, and you could merge any of them. Sure if stick to certain conventions it works pretty well, but technically the whole implementation is a minefield, which is why svn will never be as robust as other systems that don't make this mistake. Even though git's submodules and subtrees leave a lot to be desired, and have significant room for improvement, they will never be as convenient for partial checkouts, because the requirements for svn-style partial checkouts require crippling the entire system.
prog 17y ago
bzr has support for filtered views[1] if thats what you mean. [1] http://doc.bazaar.canonical.com/latest/en/user-guide/filtered_views.html http://doc.bazaar.canonical.com/latest/en/user-guide/filtere...

yummyfajitas 17y ago

Lack of a good way to edit recent history. Suppose I sit down for a collaborative session with someone. Over the course of a few hours, we might generate 10-100 commits between the two of us, and several branches, all to implement a single feature. I.e., I commit, say "hey, pull from me, you had an off by one a few minutes ago." At the end of the session, we only want to generate a single commit to the public repositories. Mercurial queues are a tolerable way to handle this, but they aren't great.

nuclear_eclipse 17y ago
Git makes this easy. It supports "rebasing" commit trees, and for what you want, a "squashed" rebase is absolutely trivial to do.
vtail 17y ago
git rebase --interactive <commit> should help http://stackoverflow.com/questions/598672/git-how-to-squash-the-first-two-commits http://stackoverflow.com/questions/598672/git-how-to-squash-... and http://stackoverflow.com/questions/435646/how-do-i-combine-the-first-two-commits-of-a-git-repository/436530#436530 http://stackoverflow.com/questions/435646/how-do-i-combine-t...
dalore 17y ago
I do this all the time with git. I create a branch, do all my incremental commits, get it all working. Then I merge it into the mainline as one squashed commit. There a few guides on how to do it with the command line but I'll admit I've gotten lazy and now just use smartgit which lets me do it easily without having to remember the commands.
prog 17y ago
git already has this in core and bzr has a rebase plugin[1]. I think this problem has been solved for quite some time now :) [1] http://doc.bazaar.canonical.com/plugins/en/rebase-plugin.html http://doc.bazaar.canonical.com/plugins/en/rebase-plugin.htm...

nuclear_eclipse 17y ago

Poor or inefficient support for large binary assets, such as compiled objects, images, or video. Especially in the context of video game development, binary assets that constantly change are highly painful to deal with in Git or Mercurial.

naner 17y ago
How is it possible that nobody has dealt with this problem yet?
- pmjordan 17y ago
  Active open source development rarely meets "AAA" game development.
  - sshumaker 17y ago
    One could argue that the traditional source control model (where everyone has a copy on their own machine) isn't a great way to deal with huge binary assets. I've worked on game projects where the binary churn was on the order of gigabytes a day. Even though everyone was physically co-located, when everyone arrived at 10am and started to update to head, a hundred computers all trying to download the days worth of 2 gigs of assets over the network didn't make for a pretty picture. You can try to cronjob the updates, staggered around 4am. Or you can realize that the majority of people are only modifying a subset of these assets, and roll your own link-based asset manager for large binaries.
    - pmjordan 17y ago
      Yes, absolutely, you wouldn't want everyone to keep pulling all data, git-style. The programmers usually only need the assets in the game-ready format, not the source models, textures, etc. which are typically much larger, and only needed by the artists/level designers working with them.
- ableal 17y ago
  I have been trying to point it out [1], without much luck. Today I got in early by chance, and decided on the Socratic approach ;-) [1] http://news.ycombinator.com/item?id=1201559 http://news.ycombinator.com/item?id=1201559, http://news.ycombinator.com/item?id=1219082 http://news.ycombinator.com/item?id=1219082, http://news.ycombinator.com/item?id=1222905 http://news.ycombinator.com/item?id=1222905, http://news.ycombinator.com/item?id=1242374 http://news.ycombinator.com/item?id=1242374

ionfish 17y ago

No way to move files between repositories and retain their history. Luke Palmer talked about this a little here: http://lukepalmer.wordpress.com/2008/11/12/sketch-of-udon-version-controlpackaging-system/ http://lukepalmer.wordpress.com/2008/11/12/sketch-of-udon-ve...

windsurfer 17y ago
git filter-branch --subdirectory-filter foodir -- --all
blasdel 17y ago
It is absolutely possible to do this in git. It helps if you forget about "files" as something to "move". I did it in the project I'm working on now! It originally had one main repository, and another for an extension that was used as a git submodule in the main one. Active development proceeded in both, with shitloads of commits in the main one just to update the reference to the submodule. I decided this was retarded, so I took a clone of the extension repository, used filter-branch to rewrite it's entire history so that all the paths were always prefixed with "vendor/extensions/project_name/", and pushed it as an unrelated branch into the main repository. Then in the main repository I made a new branch, removed the old submodule junk there, did a nice clean merge that melded all the commits from the extension branch going back in time, and made that the new master branch.

koenbok 17y ago

- When you work with large binary files that don't diff well (psd), it's a pain to have the entire repository history local. - Hard to explain to even fairly technical designers/copywriters. - With all that easy branching, I sometimes just have a hard time getting a decent version of the codebase together from everyone.

dustingetz 17y ago
i used SVN to track large binaries and it was morbidly slow. maybe i was doing it wrong. i thought the solution was to keep binary assets under a separate version control better suited for files that don't diff; a nightly snapshotted filesystem seems just fine.
- ableal 17y ago
  Seems that's why Perforce is in business (e.g. http://news.ycombinator.com/item?id=1242374 http://news.ycombinator.com/item?id=1242374 )

th 17y ago

Complexity (in terms of the operations that need to be performed regularly). At least with Git, simplicity seems to be traded in favor of flexibility.

prog 17y ago
bzr, hg and darcs are not too complex to use.

_delirium 17y ago

Well, a big one is probably the same one: politics. Instead of politics over commit privileges and commit rules, you have politics over who controls mainline, who controls which staging areas, what their rules are for pulling or accepting pushed patches, etc.

bdonlan 17y ago
The point is with centralized systems, only committers can do things such as branching or preparing commits. With a DVCS, everyone gets the same tools, so the rights granted by having the patch accepted are simply having it merged, nothing more or less.
- _delirium 17y ago
  That's true, and I think it is an improvement when it comes to the ability to do local forks. But most of the big fights I've seen in CVS/SVN-based projects ultimately come down to who controls mainline, and a DVCS doesn't solve that problem at all. If anything, it makes it easier to put even more politics on top of it, where instead of having a fight over getting into the single repository, you now have to navigate whole cascading set of repositories (e.g. Linus doesn't generally pull directly from you, so you have to deal with sub-gatekeepers before you get to the main gatekeeper). I suspect Linus likes the approach not because it has no politics, but because it encourages the politics that match the way he'd like to manage the Linux kernel development, with this style of hierarchical, cascading commit approval, which is pretty hard to set up in CVS (though Mozilla's sort of grafted it on via their Bugzilla, which keeps track of cascading approvals of patches based on who owns which areas and sub-areas). Not that that's necessarily a bad thing, it's just different (and for a project as large as Linux, probably necessary).

jedediah 17y ago

Terrible submodule support.

cool-RR 17y ago

I would want something pretty superficial: An ability to organize branches in folders.

maw 17y ago
bzr can do that with its shared repos. It's not too hard to fake it with git, either. (This assumes you know how to use git, which without a doubt is hard, despite what its apologists may tell you.)