8 ms·
Dropbox as a Git Server
- avivo 10y agoImportant thing to note — this may not be the best solution if your goal is to always have an up to date local (dropbox synced) version of a git repo. The docs discourage that: > If you're using the Dropbox client to sync files, it's a good idea to use selective sync and disable syncing of the folder containing the repository to avoid any unexpected conflicts, just in case. Also, in case this wasn't obvious, this is basically a "shim" to use dropbox as a git server (as opposed to say mercurial or SVN). No actual git server is being run and if I understand correctly, some useful features from a normal git server are not implemented. > git-remote-dropbox is a Git remote helper. > git-remote-dropbox stores all objects as loose objects - it does not pack objects. This means that we do not perform delta compression. In addition, we do not perform garbage collection of dangling objects.
- y4mi 10y agoyou shouldn't put your plain git-repo into a dropbox-client synced folder, as this corrupts the git repo eventually. but this project exists exactly because of this and circumvents said problem.
- _ZeD_ 10y agoWhat's wrong with a simple svn server?
- bartvk 10y agoI love svn, but appreciate some git things as well. Particularly the way you can do a couple of commits before pushing; I don't think there's any equivalent in Subversion?
- tacos 10y agoMany people don't seem aware of AWS CodeCommit -- AWS hosted git. $1/user per month, 10GB storage, no bandwidth charges. Free for the first year.
- todd8 10y agoThis is why I love reading HN. Thanks for the tip!
- dingle_thunk 10y agohttp://visualstudio.com http://visualstudio.com Team Services is also pretty nice. Free unlimited hosted git repos for 5 users, also includes free build/test/load test/release/CI management, kanban boards, team chat rooms. eclipse/xcode/xyz integration, etc, etc. If you've got a small team, not bad.
- bartvk 10y agoHey, thanks for that pointer! I'm an iOS developer who will start working with a bunch of C# developers, and this is just the thing I needed!
- deleted 10y ago[deleted]
- wrong_variable 10y ago> I ran out of private GitHub repositories a long time ago why doesn't OP use bit-bucket ?
- m4r71n 10y agoOr GitLab? I really don't understand why anyone would misuse Dropbox as a Git server...
- sytse 10y agoThe OP seems to be interested in a SaaS (alternative for GitHub). For the people that think you always have to install GitLab, instead of running your own GitLab server you can also use GitLab.com with unlimited projects and collaborators, see https://about.gitlab.com/gitlab-com/ https://about.gitlab.com/gitlab-com/
- oblio 10y agoHas Gitlab done any work on the high availability front? I remember reading a while back that gitlab.com was hosted on 2 servers, or something like that.
- quicksilver03 10y agoThey definitely stated that gitlab.com was running on one single server (plus a spare): https://about.gitlab.com/2015/01/03/the-hardware-that-powers-100k-git-repos/ https://about.gitlab.com/2015/01/03/the-hardware-that-powers... https://about.gitlab.com/2015/03/09/moving-all-your-data/ https://about.gitlab.com/2015/03/09/moving-all-your-data/ It looks like this is no longer the case: https://about.gitlab.com/2015/06/04/gitlab-dot-com-outage-on-2015-05-29/ https://about.gitlab.com/2015/06/04/gitlab-dot-com-outage-on... I haven't been able to determine if they're still storing the git repositories on NFS: https://about.gitlab.com/2015/09/01/gitlab-dot-com-outage-on-2015-09-01/ https://about.gitlab.com/2015/09/01/gitlab-dot-com-outage-on... but if that's the case, I would be extremely nervous about the integrity and stability of their platform. I've had too many bad experiences with NFS mounts failing and general instability that I don't want to deploy it anymore.
- jonahx 10y ago> Why shouldn't I keep a bare Git repository in a Dropbox shared folder, use it as a folder-based Git remote, and sync it with the desktop client? > There seem to be some articles on the Internet suggesting that this is a good idea. It's not. Using the desktop client to sync a bare Git repository is not safe. Concurrent changes or delays in syncing can result in a corrupted Git repository. How dangerous would this be in practice? Ie, what are the chances of corrupting a git repo with this method with a small team, and what exactly must happen to cause it?
- tlarkworthy 10y agoIt's not an issue at all and all the articles saying otherwise are FUD. http://edinburghhacklab.com/2012/11/when-git-on-dropbox-conflicts-no-problem/ http://edinburghhacklab.com/2012/11/when-git-on-dropbox-conf... Edit: Wow I got downvoted and I actually ran the experiments myself. Has anyone got experiments showing otherwise?
- justinhj 10y agoI didn't downvote you, but I don't really agree this is not an issue. The article itself shows that the git repo can get into a confused state which you have to fix, even if the fix is simple. The experiment itself doesn't emulate common causes of Dropbox pain. Firstly Dropbox stores and syncs files in a way that gits number of files and layout of files is not kind to, causing thrashing and high cpu when people commit. Different clients have different ways of locking a git repo. Issues occur when two clients make changes and are online at different times. I'm not saying everyone will hit these issues, but these are all issues that someone has run into.
- bognition 10y agoI don't have any hard numbers but I tried using Dropbox as a git remote a few years ago. Everything worked great until it didn't. A HN discussion from a few years ago suggests that Dropbox + GIT is a mixed bag: https://news.ycombinator.com/item?id=5558822 https://news.ycombinator.com/item?id=5558822
- elithrar 10y ago
- alediaferia 10y agoWhy?
- UniZero 10y agoWhy use two services when you can get the job done with one?
- collyw 10y agoWhen all you have is a hammer....
- nicky0 10y agoIf your are paying for a Dropbox account might as well make full use of it.
- dikiaap 10y agohow about free service?
- madeofpalk 10y agoIf you aren't are paying for a Dropbox account might as well make full use of it.
- mseebach 10y agoBecause a hacker felt an itch. Why not?
- 0xCMP 10y agoReading through the code (20 second skim) it does not appear that this remote would reveal any information to the OP such as your dropbox access token or code being pushed.
- rav 10y agoStudying the design document [1], one sentence makes it clear why this project is needed and simply using the Dropbox client does not suffice: > We can perform a compare-and-swap operation in Dropbox by using the "update" write mode with a specific revision number. Since a Git repository basically consists of a hash-addressed content object store alongside a mutable list of references (mapping branch names to hashes of latest commits), you need a compare-and-swap operation on the reference list to update the branch reference when you push new commits. The Dropbox client by design does not do a compare-and-swap whenever a file is updated, but the Dropbox API supports it. [1] https://github.com/anishathalye/git-remote-dropbox/blob/master/DESIGN.rst https://github.com/anishathalye/git-remote-dropbox/blob/mast...
- Mo3 10y agoI'm not sure how this is "needed" just because it is possible. Setting up a small headless git server takes 2 minutes without sharing your code with every intelligence agency on the planet.
- motoboi 10y agoIf that server is on AWS, Google Cloud or Azure, wouldn't it be totally vulnerable to national security letters?
- dahdum 10y agoCould be alluding to the appointment of Condoleezza Rice to the Dropbox board, and implication that Dropbox just gives them access versus requiring pesky NSLs or warrants (given Rice's defense of those NSA programs).
- drunkcatsdgaf 10y agosee Amber Cottle.
- sourcd 10y ago> As far as I know, git-remote-dropbox is the only safe way to host a Git repository on Dropbox. I've another hackish method that's good enough for a single developer, plus it's encrypted. I've got a truecrypt file container that resides in Dropbox. I mount it as read-write when I want to push. On the rest of my machines, its mounted as read-only. Even if the container is 100GB, it takes 10 seconds to sync daily commits - because of block encryption & syncing. This won't work with Google / MS / Amazon drive etc. because they upload the entire container on each incremental change.
- dividuum 10y agoThis feels dangerous. While I'm not sure, I don't think changes to a truecrypted filesystem are guaranteed to be atomic. I'm pretty sure there's no guarantee that the filesystem is consistent every time Dropbox starts syncing. You probably end up with a partially correct file system on the ro-side, which might end in kernel panics if you're unlucky. Also it's not two-way as the git-remote-dropbox solution offers.
- sourcd 10y agoHence the word "hackish". Though as long as TC containers are read-only, Dropbox syncs them all right.
- heavenlyhash 10y agoUh, this is also dangerous to your security. The truecrypt docs contain very large warnings about this[1]: someone who can see your disk at multiple versions may be able to drastically weaken your security. If you had keys in those truecrypt files, you should rotate them; if you had secrets, consider them compromised. Truecrypt uses XTS mode, which is Not Great -- it only exists because someone was trying to hammer ciphers into fitting nicely with fixed sized disk sectors, and it makes a number of serious compromises to do so. [2] has a good discussion of this. You do not want to combine XTS with sharing multiple versions of the ciphertext. tl;dr you've seen the penguin pictures? As [2] cleverly says, you're pretty much doing that to yourself by sharing multiple versions of the file. There's also significantly more powerful attacks that can be mounted by an adversary that can feed you corrupted blocks which will allow them to permanently compromise your key, or perform other hijinks like manipulating your content without setting off warning bells. Dropbox, or someone that compromises dropbox, is in that position, by nature of the service they're providing. These issues are rooted in the fact that XTS is not an authenticated cipher -- this leads to such an endemic category of subtle problems that it's been nicknamed "The Cryptographic Doom Principle" [3]. So yeah. If your life depends on it... don't store a truecrypt volume in dropbox. ----- [1] I found a mirror at http://andryou.com/truecrypt/docs/how-to-back-up-securely.php http://andryou.com/truecrypt/docs/how-to-back-up-securely.ph... -- and I'll quote, in case that too vanishes: > IMPORTANT: If you store the backup volume in any location that > an adversary can repeatedly access (for example, on a device kept > in a bank’s safe deposit box), you should repeat all of the above > steps (including the step 1) each time you want to back up the > volume ... where "step 1" is "Create a new TrueCrypt volume". [2] http://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ http://sockpuppet.org/blog/2014/04/30/you-dont-want-xts/ [3] http://www.thoughtcrime.org/blog/the-cryptographic-doom-principle/ http://www.thoughtcrime.org/blog/the-cryptographic-doom-prin...
- GrinningFool 10y agoIt's worth mentioning that it's also very easy to set up git for use over ssh just about anywhere (digitalocean, linode, VM or container at home, etc) The official docs cover it pretty well: https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-the-Server https://git-scm.com/book/en/v2/Git-on-the-Server-Setting-Up-... If it's for your own use, you can skip the part about creating a `git` user, and host the files in eg `/home/$USER/repositories` instead. The repository setup instructions remain the same, just the path differs, and the `authorized_keys` file to add keys to will be in `/home/$USER` instead of `/home/git`.
- Symbiote 10y agoIt's so simple, I think it's worth showing just how simple. For a single user who has already SSH access to some server, and has installed Git on that server with apt/yum/whatever: On the server: mkdir project.git; cd project.git git init --bare In an existing Git repository on your client: git add remote newserver user@example.net:project.git git push newserver master
- haldean 10y agoAnd then you can `pip install klaus` to get a nice web UI; SSH-based git plus klaus[0] is what I'm using[1] instead of Github now and it's great. [0]: https://github.com/jonashaag/klaus https://github.com/jonashaag/klaus [1]: http://git.haldean.org http://git.haldean.org
- joeblau 10y agoI find it interesting that Dropbox isn't building these types of utilities themselves. This looks pretty cool and also seems like a really good way to up-sell Dropbox storage later.
- antoncohen 10y agoWell, they sort of did build it. Anish was an intern at Dropbox when he wrote it. I'm not sure if he started it as a personal side project, but it was finished and released during a Dropbox Hack Week.
- joeblau 10y agoAh okay, thanks for the insight; that's super interesting.
- nallerooth 10y agoPeople still use Dropbox for this? I guess I shouldn't have stopped developing this 4 years ago.
- lclemente 10y agoShameless plug: I've implemented something similar in the past, but with the addition of client-side encryption. I've been using it for some time without problems. At a first glance at the design docs here, the present project looks more sophisticated in terms of syncing. I'm just using local file operations and hoping for dropbox to do its stuff :) https://github.com/lucas-clemente/git-cr https://github.com/lucas-clemente/git-cr
- erikb 10y agoIf you understand how git works, you don't need a tool to do this, and especially not a python tool. Don't misunderstand me, I love no language more than Python. But putting a directory under Dropbox supervision is the easiest thing you can do. That doesn't change because it's name is ".git/".
- maxerickson 10y agoAre you sure you've correctly identified the "this" in question? This enables using Dropbox as a shared remote repository, which is more than putting the directory under Dropbox.
- segmen 10y ago? Dropbox for this ?
- chmaynard 10y agoI'm shamelessly using GitHub Desktop and loving it. I guess I'll pass on this project unless there is a way to point GD to a Dropbox-hosted repo.
- sudopluto 10y agoI think you misunderstood the point of the project. It is not making dropbox a git gui, but makes it possible to use dropbox as a remote git server.
- chmaynard 10y agoWhat I meant to say is that I generally use git via the GitHub Desktop app, not the command line. I don't think it is possible to point GitHub Desktop to a Dropbox repo. Or is it?
- KayL 10y agoStoring massive small files in Dropbox folders is horrible! It burns my SSD & CPU on each restart.
- amalag 10y agoMany people don't realize it but you can store git repos in S3 using jgit. Super cheap backups of git repos.
- meonkeys 10y agoSyncthing (https://syncthing.net/ https://syncthing.net/) as a git server would be cool too. I use Syncthing to sync around my personal git repo between my machines. All pushes and pulls are instant. I don't have conflicts because I'm only working on one machine at a time. `git-remote-syncthing` would presumably be even better, allowing multiple simultaneous users.