17 ms·

GitTorrent: A Decentralized GitHub

1038 points by luu 11y ago

doragcoder 11y ago
I need to go more in-depth on the proposal. But the first thing that strikes me, is if you're going to use the Blockchain (with a capital "B") as storage of usernames and such. Why not use namecoin? It has the process for name consensus down. Also it won't pollute the main Bitcoin blockchain.
- cjbprime 11y ago
  Hi, author here. I have a mild bias against altcoins, and have heard bad things about Namecoin in particular: that the anti-spam incentives aren't good, leading to illegal files stored in the blockchain itself, and that there's no compact representation (like Bitcoin's Simplified Payment Verification) for determining whether a claimed name is valid without consulting a full history. As I understand it, these two design flaws combine to mean that you have to store some very illegal files to use a namecoin resolver, which doesn't sound good to me. (I may be mistaken, since the bad things I heard about Namecoin came from Bitcoin people..)
  - jMyles 11y ago
    The notion that a file can be illegal makes as much sense to the internet as a plant being illegal makes to the earth. Especially when it has the side effect of inhibiting what otherwise might be a compelling solution.
    - cxxv 11y ago
      Your analogy doesn't hold up, as (for one) we have society and plants don't. I'm glad that child porn is illegal (which is usually distributed in files), as it makes the world safer for children. Also: some plants are serious jerks.
      - nilved 11y ago
        If pi goes on forever, it contains any and all representations of child porn that could exist. The first person to calculate pi that far will be breaking the law. No, that doesn't make sense, nor does any sort of information being illegal. What does make sense is to legislate the actions that can generate that data, or the misappropriation of that data.
        StephenFalken 11y ago
        Great point !
        yoplait_ 11y ago
        no, it's a really contrived point.
        jMyles 11y ago
        Hear hear. This is an easy-to-understand and concise thought experiment; use it often!
        relate 11y ago
        You took a poor example since pi is not known to be a normal number (see eg http://mathworld.wolfram.com/NormalNumber.html http://mathworld.wolfram.com/NormalNumber.html ) This number however is: 0.0 1 10 11 100 101 110 111 1000 ... This number contains all movies that have and will ever exist. However to specify where a 700mb harry potter movie is in this number, you'll need at least 700mb to represent the index. So in some sense harry potter 'exists' in this number - but in another sense its just a silly encoding method.
    - detaro 11y ago
      Most law enforcement and judges don't ask what makes sense to the internet, even if you ignore users personal opinions about the matter.
    - leohutson 11y ago
      Right, and the notion that murder is illegal makes no sense to the universe as humans are "merely" an aggregate of matter no different than any other. Nothing makes any sense to entities not capable of reasoning, that is a pointless tautology with no relevance to the discussion.
  - vezzy-fnord 11y ago
    There aren't any inherent protections to storing illegal data in the Bitcoin blockchain, either (short of evading the issue by using a thin client, I suppose). The nature of what is illegal being so encompassing, in that you can ultimately encode the information into some integer of sorts either way, means it's impractical or unnecessary to provide full protection for such things.
    - cjbprime 11y ago
      No inherent protections, just different incentives. Storing a 4MB image at 80 bytes per $0.08 OP_RETURN transaction would cost you $4000. Why would you do that when you can use namecoin for practically free?
      - krupan 11y ago
        Which makes the statement you made that, "this currency stuff is frankly kind of uninteresting to me," kind of funny (I assume that was on purpose) because the currency stuff is exactly the thing that makes bitcoin work so well.
        cjbprime 11y ago
        Fair point! I made that statement because a lot of people don't realize that Bitcoin, ignoring the fact that you can trade it, has solved a fundamental consensus problem in distributed systems that we should care about and use :)
  - hurin 11y ago
    > I have a mild bias against altcoins, and have heard bad things about Namecoin in particular: that the anti-spam incentives aren't good, leading to illegal files stored in the blockchain itself, and that there's no compact representation (like Bitcoin's Simplified Payment Verification) for determining whether a claimed name is valid without consulting a full history. Security-wise Namecoin is a weaker Blockchain, but I think in this case it's not that important. -- Are not the anti-spam measures hurting users as much as spammers in this case? Since you end up with higher-costs for what is otherwise practically free with a centralized service. From the looks of it there is no reason that Bitcoin's Simplified Payment Verification wouldn't be usable with Namecoin either. edit: Also this may be interesting to take a look at https://people.csail.mit.edu/nickolai/papers/vandenhooff-versum.pdf https://people.csail.mit.edu/nickolai/papers/vandenhooff-ver...
shmerl 11y ago
> There are philosophical reasons, too: GitHub is closed source, so we can’t make it better ourselves. There is GitLab.
- sytse 11y ago
  GitLab CEO here, thanks for mentioning us as the open source alternative. We think in the short term multiple organizations hosting their own GitLab is the way to go. It is hard to do issues and pull/merge requests in a decentralized way (the OP is impressive but it shows distributed git instead of distributed GitHub). I would like to see federated merge requests http://feedback.gitlab.com/forums/176466-general/suggestions/5097708-implement-cross-server-federated-merge-requests http://feedback.gitlab.com/forums/176466-general/suggestions...
  - JoshTriplett 11y ago
    Not just federated merge requests, but federated one-click cloning. Given a "home" GitLab instance, you should be able to click a button on any GitLab repository (or on repositories elsewhere, based on some spec), and end up with a clone of that repository in your "home" instance. That should be possible via browser support for site-based URL handlers; for browsers without such support, you could also have a URL on the main GitLab site that people can register with to redirect to their "home" instance.
    - sytse 11y ago
      Interesting idea. I think it is harder to do this than the federated merge requests because of the permissions, but I love the idea.
      - JoshTriplett 11y ago
        You'd be logged into your "home" instance, which gives you the permissions you need. You click a link on the remote site, which takes you (either directly via URL handler or indirectly via central site redirect) to your home instance with appropriate values filled in, and prompts you to confirm (to avoid CSRF).
alexnewman 11y ago
Stored github in btsync for a startup before. It actually worked ok. We stopped doing it because btsync was trivial to crash with basic fuzzing and was closed source
innovator116 11y ago
Similar to Gitchain http://gitchain.org/ http://gitchain.org/ ?
- cjbprime 11y ago
  (Author here.) Yeah! I was a Gitchain backer. The difference is that Gitchain stored the actual git commits in the blockchain, and I leave the actual commits on the hard disks of each BitTorrent seeder.
dmarti 11y ago
The remaining hard part is the adapter layer to enable the extra applications such as the issue tracker to use the Git repository for storage. Joey Hess has a good article about Git "databranches" here: https://joeyh.name/blog/entry/databranches/ https://joeyh.name/blog/entry/databranches/
ianphughes 11y ago
I have to admit that I started reading this post feeling a little snarky about the concept. However, I think Chris makes an excellent case for the concept.
mrisse 11y ago
Can I clone GitTorrent with GitTorrent?
- cjbprime 11y ago
  Yes, it works. npm install gittorrent git clone gittorrent://github.com/cjb/gittorrent Or for true decentralization (doesn't get the wanted sha1 from github.com): gittorrent://81e24205d4bac8496d3e13282c90ead5045f09ea/gittorrent
  - mrisse 11y ago
    It works! Very cool.
  - colund 11y ago
    I got fatal: Unable to find remote helper for 'gittorrent'
    - cjbprime 11y ago
      How about "sudo npm install -g gittorrent"? It looks like the git-remote-gittorrent binary isn't ending up in your $PATH.
      - viraptor 11y ago
        Or without polluting global namespace: export PATH="$PATH:$HOME/node_modules/.bin"
arxpoetica 11y ago
Major fan of this idea. But how does one address the GUI challenges presented by leaving GitHub behind? It can't be understated that GitHub provides an amazing (communal/social) user experience.
- scotchmi_st 11y ago
  It's the same old problem of trying to build a peer-to-peer social network. How do you ensure that large files are distributed correctly and quickly with minimal security implications in an environment where nodes are constantly joining and leaving the network? Perhaps it's possible, but if it were an easy way of doing it, there would be more of that sort of thing around.
  - amirouche 11y ago
    > How do you ensure that large files are distributed correctly and quickly with minimal security implications in an environment where nodes are constantly joining and leaving the network? The owner of a project will keep a node always online to fight its own churn. There is no big files in the case of issue tracker. This is not video social network. Also, video social network do work, e.g. private trackers, the only thing is that they use the web publish magnet links which can be done over a DHT cf. PPSP and tribler.
- johndevor 11y ago
  Wouldn't it be the equivalent of creating a BitTorrent client? There are lots of attractive and functional clients for that.
  - woah 11y ago
    I think they are talking about the issue tracking etc
- towelguy 11y ago
  I guess the projects that can't be on github won't mind the GUI challenges as long as they have some way to have a central repository without having to maintain a server on their own.
alganet 11y ago
I'd love something like this for fossil[1], which can also handle wiki and issues inside it's repos by default. [1]: http://fossil-scm.org/index.html/doc/trunk/www/index.wiki http://fossil-scm.org/index.html/doc/trunk/www/index.wiki
x5n1 11y ago
There is also Gogs: https://github.com/gogits/gogs https://github.com/gogits/gogs
Kenji 11y ago
Haha, that reminds me of my comment earlier: https://news.ycombinator.com/item?id=9578307 https://news.ycombinator.com/item?id=9578307 And plenty of people schooling me that git is already distributed (git yes, github no). I am happy that someone is working on this.
ackalker 11y ago
A very interesting idea, GitTorrent, but I have one question which comes to me whenever I read about a delta-based distribution scheme: who is going to generate and share all those deltas? Some Linux distributions have experimented with delta-based package repositories, examples are deltup for Arch Linux and rpm-delta for RPM-based distros. Some of the known issues are: - choosing the number and spacing between deltas. Fine-grained deltas require more storage space, coarse-grained deltas require more download bandwidth. - retiring old deltas: periodically deleting all deltas older than a certain version, replacing them with the full package of that version. Again a trade-off between storage space and download bandwidth. For Git repositories, this would roughly translate to: - choosing the number, history spacing, and size of the Git packs per repository. - retiring old Git packs: periodically deleting Git packs older than a particular revision, replacing them with a bare repository at that revision.
- pjc50 11y ago
  Git is built out of deltas. You're already storing all of them.
  - ackalker 11y ago
    The problem is not with deltas between revisions (i.e. the commits themselves) but with the Git packs spanning multiple revisions. At the time of a `git pull`, a user's repository can be at any revision between initial and latest. Who is going to seed (and keep seeding) packs for all of those possible revision intervals?
  - deleted 11y ago
    [deleted]
  - peterwaller 11y ago
    This is isn't quite how I would describe it. Git does do delta compression in packfiles, but the fundamental primitives lack deltas. It's just: * The contents of this directory is this list of of files whose contents have these SHAs. This is called a "tree". The SHA of a tree is also an object, and can appear in another tree. To see this for yourself, in any git repository run `git cat-file -p HEAD`. You'll see the (more or less) raw commit object for HEAD, which will point at a tree SHA. To see the contents of that tree-sha, run `git cat-file -p <the tree SHA>`. That tree object has a one-to-one correspondence with what you'll see on-disk in the objects directory, (if the object has not been put in a pack file). Above I have more or less fully described the contents of the files found in `.git/objects`. The delta'ing doesn't happen until later, if and when packfiles are constructed. But they're just a storage/bandwidth optimisation. AFAICT, these deltas have nothing to do with what you might think of as "git diff", which is just some fancy porcelain which looks at objects. The nice property of the construction is that given a large tree, even if nested, if you change a single file in that tree, you will only change as many trees as the file is deep in the tree, so computing changes between two nearby trees can usually be done quickly.
rektide 11y ago
Lovely work! We ask for the commit we want and connect to a node with BitTorrent, but once connected we conduct this Smart Protocol negotiation in an overlay connection on top of the BitTorrent wire protocol, in what’s called a BitTorrent Extension. Then the remote node makes us a packfile and tells us the hash of that packfile, and then we start downloading that packfile from it and any other nodes who are seeding it using Standard BitTorrent. The disadvantage here is that any given file in the repo could be stored in an ever increasing number of packfiles. Each existing version of the repo will generate a new packfile to get it to the newest version, and it's up to the authenticated masters to generate and seed each of those packfiles, while peers either do or do not cache these replicated datas. In short, this means of syndicating updates ignores the Merkle-DAG-ness (DAG-osity?) of Git. The un-updateability of torrents is something that seems to seriously limit it's use. There are a lot of interesting attempts to hack around this- LiveStreaming and Nightweb are two that spring to mind. https://www.tribler.org/StreamingExperiment/ https://www.tribler.org/StreamingExperiment/ https://sekao.net/nightweb/protocol.html https://sekao.net/nightweb/protocol.html
- minot 11y ago
  > The un-updateability of torrents is something that seems to seriously limit it's use. I am not involved with the development of torrents at all but (please bear with me until the end) my initial reaction is that we should think of the lack of ability of torrents to update as a feature and not as a bug. Perhaps if ability of torrents to update is a concern then it warrants a new peer to peer protocol? (Please note that this is not the case of http://xkcd.com/927/?cmpid=pscau http://xkcd.com/927/?cmpid=pscau as I am not advocating a new protocol for every use case) It seems like we can sign torrent files with gpg keys. Perhaps I am wrong. Perhaps, we can allow updating in torrents if we require that the updates be signed with the same private key as the original torrent? Am I barking up the right tree here? Edit: Oops. I edited this post before I saw the reply about BEP-0039. Apologies.
  - dpiers 11y ago
    Support for mutable/updateable torrents was proposed in BEP-0039 (http://www.bittorrent.org/beps/bep_0039.html http://www.bittorrent.org/beps/bep_0039.html) and is the basis of how BitTorrent Sync functions.
durin42 11y ago
> Google Code announced its shutdown a few months ago, and their rationale was explicitly along the lines of “everyone’s using GitHub anyway, so we don’t need to exist anymore”. I mean, sorta. It was also because running a service is expensive, and containing abuse is a constant thankless treadmill. > We’re quickly heading towards a single central service for all of the world’s source code. Far from it? Not that a fully-decentralized system seems bad, but there are many things that aren't github. I don't even have anything of interest on github.
DannoHung 11y ago
Sweet as hell. My only quibble is the idea of using the blockchain for validated naming. I think it'd end up in a landgrab which is nasty. As much as we all hate DNS, having the ability to kick a squatter off someone's name is probably a good thing. Personally I don't think having a crazy hash for identification is a bad thing. Rather, what you need to do is just have some sort of reasonable personal contact book so you only have to deal with the crazy hash once (when you decide you want to remember that someone is who they said they were).
- cjbprime 11y ago
  I don't really understand the landgrab complaint: this costs $0.08 per username registered ($0.16 if you want to avoid races with miners), in comparison to centralized sites, which usually cost $0 per username registered. Why does this lose?
  - DannoHung 11y ago
    Centralization is about equivalent to ICANN being able to kick off a squatter. A truly decentralized system with first to register doesn't have a way to kick off squatters.
    - icebraining 11y ago
      Well, the current ones don't, but it could. Decentralized networks just need decentralized decision, ie., you could have a process by which the majority of the nodes decide to kick off a squatter.
      - tw04 11y ago
        At which point someone uses a botnet to game the system.
- namecast 11y ago
  Hmm. How about mixing and matching, like by having a DNS record with a special TXT record syntax, e.g. 'IN TXT "gt=yoursha256hash"'. That way you could find your destination via DNS and verify the hash in the TXT record against the blockchain.
pjc50 11y ago
I'm very impressed by this, and I'm a bitcoin skeptic and git usability critic.
- ossreality 11y ago
  so brave
tyuwan 11y ago
sdfs
Skunkleton 11y ago
I could see this integrating really nicely with mailing lists. The commit could be sent out to the mailing list, and anyone who is interested in reviewing the code or doing a merge would already have the information they need, no blockchain required.
physcab 11y ago
This is an aside but I like the author's writing style. Not only is he clear but also describes why he's doing what he's doing and how it's important to him and others. Really helps me give the ideas more thought!
tr4s 11y ago
How is the new value for a mutable key propagated through the DHT? (as in how sure am I to get the newest commit hash?)
kodisha 11y ago
decentralized web interface? Isn it better to develop native, cross platform app for that? We have both electron and NW.js now, that should be easy.
deleted 11y ago
[deleted]
simplemath 11y ago
Well, thats a winning name, to be certain. Pretty great work too.
k2enemy 11y ago
Forgive me if my comment too readily reveals my ignorance on these topics, but would the IPFS project [0] be useful for this? [0] http://ipfs.io http://ipfs.io
- tperson 11y ago
  Definitely, checkout their example on how to host git on ipfs[0] [0] http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/git/readme.md http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsN...
  - e12e 11y ago
    While integration between git and ipfs is good -- I'm not sure I see how this is much different from just tar-ing up a git-archive and creating a torrent/magnet (setting aside many of the other aspects of how ipfs and torrents work). Decentralized git pull for only a given hash isn't all that interesting? If one could pull in updates and/or push changes -- that would be "decentralized git". This is example is more "ipfs.io as a transport for git-releases", rather than "ipfs.io as a transport for git"?
  - chriscool 11y ago
    There is this script I worked on to help doing that: https://github.com/whyrusleeping/git-ipfs-rehost https://github.com/whyrusleeping/git-ipfs-rehost
hellbanner 11y ago
Well done. I think this is a great piece to build a "pay for branch merging" way of promoting open source.. use decentralized currency and voila, automated programming.
shea256 11y ago
Love this. The post mentions using the blockchain for unique username registration and mapping to public key hashes, and as it turns out there's a project I and others have been working on that does exactly this called Blockstore. Here's the link if anyone wants to check it out: https://github.com/namesystem/blockstore https://github.com/namesystem/blockstore The way it works is there's a mapping between a unique name and a hash in the blockchain, and then there's a mapping in a DHT from the hash to the data to be associated (which can be a plain old public key and can also be a JSON file that references a public key and other identity information).
- cjbprime 11y ago
  (GitTorrent author here.) That's great, thanks! I should just use this (preferably with the DHT I'm already using to look up Git commits) instead of reimplementing myself. What do you think about the idea of making pluggable modules to connect Blockstore with web frameworks (Django, Rails), without the framework/website authors having to get involved in understanding Bitcoin themselves?
  - deleted 11y ago
    [deleted]
  - shea256 11y ago
    Oh I love that idea. As far as Django, Blockstore is on Pypi so you can just install and import the library. I think it'd be great to have modules for Rails, Node, etc. I saw your tweet and I'll shoot you an email. Also feel free to open an issue on github.com/namesystem/blockstore and we can discuss the idea openly there.
ArneBab 11y ago
> It surprised me that nothing like this seems to exist already in the decentralization community. There was a GSoC project in 2013 which did exactly this, using Freenet as decentralized storage backend with a Web of Trust for Spam resistant and updatable identities (note that in the gittorrent scheme once someone claimed a username, that username will stick there forever). It works and compared to GitTorrent it adds anonymity and upload-and-run. A current article describing it is here: http://draketo.de/english/freenet/real-life-infocalypse http://draketo.de/english/freenet/real-life-infocalypse (it got referenced here, too: https://news.ycombinator.com/item?id=9562749 https://news.ycombinator.com/item?id=9562749 ) The GSoC project was done by Steve Dougherty: http://www.google-melange.com/gsoc/project/details/google/gsoc2013/sdough/5650716672655360 http://www.google-melange.com/gsoc/project/details/google/gs... > I’d be happy to work on a project like this and make GitTorrent sit on top of it, so please let me know if you’re interested in helping with that. Have a look at Gitocalypse: https://github.com/SeekingFor/gitocalypse https://github.com/SeekingFor/gitocalypse
comex 11y ago
If a version of this with friendly name support is released, I will mirror all my active GitHub repositories there. If someone builds on this, as discussed elsewhere in the thread, to make a decentralized service that mimics 'social' functionality such as issues and pull requests, I will strongly consider using it instead of GitHub (depending on the UI, stability, etc.). I don't even have any particularly popular repos, so there is no real reason for anyone to care about the above, but, y'know, HN comments approving of the idea don't necessarily translate into actual interest in the product, so now you know there's at least one person in the latter category. :)
erikb 11y ago
Git itself is already distributed. Do we really need more distribution or do we simply need to learn to work more distributed?
- ajkjk 11y ago
  It's clear, though, that there are aspects of 'distributed' that Git does not accomplish. This gives you a global namespace and a global identity, just like a website, without a single point of failure or a corporate interest, and with a closed source system and a trivial option for forking the whole structure into your own [friend group|company|private cluster|etc]. An argument could be made that git ought to be augmented with something like this: why run distributed protocol A on distributed protocol B; maybe we should just run distributed protocol AB from the get-go?
placebo 11y ago
Great. Centralism and monopoly in any form makes me uneasy. This is another point for freedom...
laumars 11y ago
I don't understand why so many people are in awe of this project when it seems to be based on a number of falsehoods: > "imagine someone arguing that we can do without BitTorrent because we have FTP. We would not advocate replacing BitTorrent with FTP, and the suggestion doesn’t even make sense! First — there’s no index of which hosts have which files in FTP, so we wouldn’t know where to look for anything." Actually there is. It's called a mirror list. Most FTP-based repositories support this. > "And second — even if we knew who owned copies of the file we wanted, those computers aren’t going to be running an anonymous FTP server." Except bittorrent does turn your client into a server. Many clients silently punch holes in your firewall via uPnP, so you don't always realise you're running a server, but it does still happen. And as for anonymous FTP servers, it depends on what you mean there. If you mean anonymous access, then that's not only supported, but actually the norm. If you mean the server itself is anonymous, then it should be noted that neither github nor torrent seeding peers are anonymous either. > "Just like Git, FTP doesn’t turn clients into servers in the way that a peer-to-peer protocol does. So that’s why Git isn’t already the decentralized GitHub — you don’t know where anything’s stored, and even if you did, those machines aren’t running Git servers that you’re allowed to talk to. I think we can fix that." Hang on, a moment ago you _didn't_ want to run servers. Now you're complaining that git clones aren't servers? ----- Then there's the matter of the github competitors, of which there are many. gitlab, gitbucket, etc. Some open source, some closed but free, but all of them largely offer the same features as github. These days it seems trendy to use bittorrent as a bootstrap for all kinds of wacky and wonderful problems, but using bittorrent for a protocol that's already distributed and already pretty saturated with github competitors; well it just seems redundant.
- ajkjk 11y ago
  I think your complaints are disingenuous. [edit: what I mean is that you've pointed out problems with small components of the overall article and used them as an argument against the article as a whole, which I think is unfair. I don't think the components' validity affects the overall idea.] His argument may have some holes, and people are probably mostly ignorant of those holes so they can't critique them, but I don't think they're in awe of the idea because their awe hasn't been dispelled by identifying those holes. This is a very neat idea, and none of the issues with the setup argument dispel that. "using bittorrent for a protocol that's already distributed and already pretty saturated with github competitors; well it just seems redundant." no, no, it doesn't seem redundant. DHT based distributed indexing is so incredibly fundamentally different than a mirror list for files in FTP or from a series of GitHub clones. It's owner-agnostic. It just exists, by virtue of having participants, with no overhead. I don't have to select a target host or find a server or even identify where my particular file (or git repository, or username, or whatever) lives. It's .. unification. It's elegant and reduces complexity and makes the whole ecosystem more simple. Maybe I'm blinded by my own awe, but, I love this idea. Also, yes, some people have thought of components of this before, but I haven't really seen the full stack laid out vertically like this, combined with a narrative that makes me so excited about it.
4684499 11y ago
I don't think this is new. http://code.google.com/p/gittorrent/ http://code.google.com/p/gittorrent/
ajjai 11y ago
nice
mixmastamyk 11y ago
Interesting, I'm also very interested in the other project mentioned "bugs everywhere." Anyone have more details on day to day usage of it?
joe563323 11y ago
Just awesome idea. The idea of having a torrent for git repos blew my mind.
decentrality 11y ago
This is awesome, but it centralizes on JavaScript. It is an implementation of a standard without the standard being defined so other implementations can spring up. Git, one could argue, is language centralized also, which is technically true. That I don't have an answer for. But I don't believe handing off so much dependence to a JavaScript application fits for me. A C/C++ application like Git I can overlook, at least for a decade or so, but JavaScript feels like a perpetual beta/prototype only. Granted, that's my subjective feeling. Raised issue: https://github.com/cjb/GitTorrent/issues/12 https://github.com/cjb/GitTorrent/issues/12
- swombat 11y ago
  The word "centralize" in the way you're using it is very misleading. Every piece of programming in existence uses some sort of language. What you're really stating, I think, is that this is written in Javascript and you don't like Javascript. That's totally fine, and it's your prerogative. I'm sure that, like any other piece of programming, GitTorrent can be rewritten in other languages. If Javascript bothers you that much, then do this: rewrite it in Ruby if that's what you prefer. But please stop attacking the claim that this is "decentralised github" by claiming that this "centralises" on Javascript. It doesn't "centralise" on any language. It's just a first implementation written in Javascript.
  - decentrality 11y ago
    That this implementation being uncomfortable for me is my subjective feeling, I've mentioned several times. That I don't want to depend on JavaScript is clear. I don't want to depend on JavaScript. Using the word centralize the way I have, I can't see otherwise. But I'm not defaming anyone. I'm not attacking anything. I'm very motivated to help the standard advance, but I would never for a moment install this implementation. Stupid of me? Sure, maybe. But it's not an attack. What I wanted I already got, confirmation by the OP that this is in fact a reference implementation and not the all-end-all. Were that clear to start with, I wouldn't have commented other than to say, awesome! This is extremely exciting to me.
z3t4 11y ago
I always thought Github was open source and that you could fork it, like in the spirit of Github. I find that slightly ironic =) What's stopping a decentralized github from ending up in the same fate as the newsgroups, that the data set gets too large to handle !? I think that Github is more then just a repository, it's a community. I kinda quit Facebook and signed up to Github instead :P And if it weren't for Github I would have never touched git. Startup idea: Create something like Github and assembly.com, but with a complete tool-set (git+vps)
hobarrera 11y ago
I like the idea of decentralized, but I'm not sure we need p2p as well. It'd be interesting to see something like gitlab to have some sort of federation support; where multiple instances can talk to one another (a la xmpp/smtp), so as to clone/send pull-requests, etc across different instances.
- sytse 11y ago
  We would love to see features like that in GitLab http://feedback.gitlab.com/forums/176466-general/suggestions/5097708-implement-cross-server-federated-merge-requests http://feedback.gitlab.com/forums/176466-general/suggestions...
pdudits 11y ago
What I miss in the article or discussion is consistency of refs. The moment your remote is a distributed one, it may have two different values for HEAD in single point of time in different parts of network. If two clients push in this time, HEADs diverge even further. So as in distributed databases, you either: * need to acquire exclusive lock on the repository metadata, or * accept, that your push will be eventually discarded because you did not have up to date metadata

1038 points by luu 11y ago

I need to go more in-depth on the proposal. But the first thing that strikes me, is if you're going to use the Blockchain (with a capital "B") as storage of usernames and such. Why not use namecoin? It has the process for name consensus down. Also it won't pollute the main Bitcoin blockchain.

cjbprime 11y ago
Hi, author here. I have a mild bias against altcoins, and have heard bad things about Namecoin in particular: that the anti-spam incentives aren't good, leading to illegal files stored in the blockchain itself, and that there's no compact representation (like Bitcoin's Simplified Payment Verification) for determining whether a claimed name is valid without consulting a full history. As I understand it, these two design flaws combine to mean that you have to store some very illegal files to use a namecoin resolver, which doesn't sound good to me. (I may be mistaken, since the bad things I heard about Namecoin came from Bitcoin people..)
- jMyles 11y ago
  The notion that a file can be illegal makes as much sense to the internet as a plant being illegal makes to the earth. Especially when it has the side effect of inhibiting what otherwise might be a compelling solution.
  - cxxv 11y ago
    Your analogy doesn't hold up, as (for one) we have society and plants don't. I'm glad that child porn is illegal (which is usually distributed in files), as it makes the world safer for children. Also: some plants are serious jerks.
    - nilved 11y ago
      If pi goes on forever, it contains any and all representations of child porn that could exist. The first person to calculate pi that far will be breaking the law. No, that doesn't make sense, nor does any sort of information being illegal. What does make sense is to legislate the actions that can generate that data, or the misappropriation of that data.
      - StephenFalken 11y ago
        Great point !
        yoplait_ 11y ago
        no, it's a really contrived point.
      - jMyles 11y ago
        Hear hear. This is an easy-to-understand and concise thought experiment; use it often!
      - relate 11y ago
        You took a poor example since pi is not known to be a normal number (see eg http://mathworld.wolfram.com/NormalNumber.html http://mathworld.wolfram.com/NormalNumber.html ) This number however is: 0.0 1 10 11 100 101 110 111 1000 ... This number contains all movies that have and will ever exist. However to specify where a 700mb harry potter movie is in this number, you'll need at least 700mb to represent the index. So in some sense harry potter 'exists' in this number - but in another sense its just a silly encoding method.
  - detaro 11y ago
    Most law enforcement and judges don't ask what makes sense to the internet, even if you ignore users personal opinions about the matter.
  - leohutson 11y ago
    Right, and the notion that murder is illegal makes no sense to the universe as humans are "merely" an aggregate of matter no different than any other. Nothing makes any sense to entities not capable of reasoning, that is a pointless tautology with no relevance to the discussion.
- vezzy-fnord 11y ago
  There aren't any inherent protections to storing illegal data in the Bitcoin blockchain, either (short of evading the issue by using a thin client, I suppose). The nature of what is illegal being so encompassing, in that you can ultimately encode the information into some integer of sorts either way, means it's impractical or unnecessary to provide full protection for such things.
  - cjbprime 11y ago
    No inherent protections, just different incentives. Storing a 4MB image at 80 bytes per $0.08 OP_RETURN transaction would cost you $4000. Why would you do that when you can use namecoin for practically free?
    - krupan 11y ago
      Which makes the statement you made that, "this currency stuff is frankly kind of uninteresting to me," kind of funny (I assume that was on purpose) because the currency stuff is exactly the thing that makes bitcoin work so well.
      - cjbprime 11y ago
        Fair point! I made that statement because a lot of people don't realize that Bitcoin, ignoring the fact that you can trade it, has solved a fundamental consensus problem in distributed systems that we should care about and use :)
- hurin 11y ago
  > I have a mild bias against altcoins, and have heard bad things about Namecoin in particular: that the anti-spam incentives aren't good, leading to illegal files stored in the blockchain itself, and that there's no compact representation (like Bitcoin's Simplified Payment Verification) for determining whether a claimed name is valid without consulting a full history. Security-wise Namecoin is a weaker Blockchain, but I think in this case it's not that important. -- Are not the anti-spam measures hurting users as much as spammers in this case? Since you end up with higher-costs for what is otherwise practically free with a centralized service. From the looks of it there is no reason that Bitcoin's Simplified Payment Verification wouldn't be usable with Namecoin either. edit: Also this may be interesting to take a look at https://people.csail.mit.edu/nickolai/papers/vandenhooff-versum.pdf https://people.csail.mit.edu/nickolai/papers/vandenhooff-ver...

shmerl 11y ago

> There are philosophical reasons, too: GitHub is closed source, so we can’t make it better ourselves. There is GitLab.

sytse 11y ago
GitLab CEO here, thanks for mentioning us as the open source alternative. We think in the short term multiple organizations hosting their own GitLab is the way to go. It is hard to do issues and pull/merge requests in a decentralized way (the OP is impressive but it shows distributed git instead of distributed GitHub). I would like to see federated merge requests http://feedback.gitlab.com/forums/176466-general/suggestions/5097708-implement-cross-server-federated-merge-requests http://feedback.gitlab.com/forums/176466-general/suggestions...
- JoshTriplett 11y ago
  Not just federated merge requests, but federated one-click cloning. Given a "home" GitLab instance, you should be able to click a button on any GitLab repository (or on repositories elsewhere, based on some spec), and end up with a clone of that repository in your "home" instance. That should be possible via browser support for site-based URL handlers; for browsers without such support, you could also have a URL on the main GitLab site that people can register with to redirect to their "home" instance.
  - sytse 11y ago
    Interesting idea. I think it is harder to do this than the federated merge requests because of the permissions, but I love the idea.
    - JoshTriplett 11y ago
      You'd be logged into your "home" instance, which gives you the permissions you need. You click a link on the remote site, which takes you (either directly via URL handler or indirectly via central site redirect) to your home instance with appropriate values filled in, and prompts you to confirm (to avoid CSRF).

alexnewman 11y ago

Stored github in btsync for a startup before. It actually worked ok. We stopped doing it because btsync was trivial to crash with basic fuzzing and was closed source

innovator116 11y ago

Similar to Gitchain http://gitchain.org/ http://gitchain.org/ ?

cjbprime 11y ago
(Author here.) Yeah! I was a Gitchain backer. The difference is that Gitchain stored the actual git commits in the blockchain, and I leave the actual commits on the hard disks of each BitTorrent seeder.

dmarti 11y ago

The remaining hard part is the adapter layer to enable the extra applications such as the issue tracker to use the Git repository for storage. Joey Hess has a good article about Git "databranches" here: https://joeyh.name/blog/entry/databranches/ https://joeyh.name/blog/entry/databranches/

ianphughes 11y ago

I have to admit that I started reading this post feeling a little snarky about the concept. However, I think Chris makes an excellent case for the concept.

mrisse 11y ago

Can I clone GitTorrent with GitTorrent?

cjbprime 11y ago
Yes, it works. npm install gittorrent git clone gittorrent://github.com/cjb/gittorrent Or for true decentralization (doesn't get the wanted sha1 from github.com): gittorrent://81e24205d4bac8496d3e13282c90ead5045f09ea/gittorrent
- mrisse 11y ago
  It works! Very cool.
- colund 11y ago
  I got fatal: Unable to find remote helper for 'gittorrent'
  - cjbprime 11y ago
    How about "sudo npm install -g gittorrent"? It looks like the git-remote-gittorrent binary isn't ending up in your $PATH.
    - viraptor 11y ago
      Or without polluting global namespace: export PATH="$PATH:$HOME/node_modules/.bin"

arxpoetica 11y ago

Major fan of this idea. But how does one address the GUI challenges presented by leaving GitHub behind? It can't be understated that GitHub provides an amazing (communal/social) user experience.

scotchmi_st 11y ago
It's the same old problem of trying to build a peer-to-peer social network. How do you ensure that large files are distributed correctly and quickly with minimal security implications in an environment where nodes are constantly joining and leaving the network? Perhaps it's possible, but if it were an easy way of doing it, there would be more of that sort of thing around.
- amirouche 11y ago
  > How do you ensure that large files are distributed correctly and quickly with minimal security implications in an environment where nodes are constantly joining and leaving the network? The owner of a project will keep a node always online to fight its own churn. There is no big files in the case of issue tracker. This is not video social network. Also, video social network do work, e.g. private trackers, the only thing is that they use the web publish magnet links which can be done over a DHT cf. PPSP and tribler.
johndevor 11y ago
Wouldn't it be the equivalent of creating a BitTorrent client? There are lots of attractive and functional clients for that.
- woah 11y ago
  I think they are talking about the issue tracking etc
towelguy 11y ago
I guess the projects that can't be on github won't mind the GUI challenges as long as they have some way to have a central repository without having to maintain a server on their own.

alganet 11y ago

I'd love something like this for fossil[1], which can also handle wiki and issues inside it's repos by default. [1]: http://fossil-scm.org/index.html/doc/trunk/www/index.wiki http://fossil-scm.org/index.html/doc/trunk/www/index.wiki

x5n1 11y ago

There is also Gogs: https://github.com/gogits/gogs https://github.com/gogits/gogs

Kenji 11y ago

Haha, that reminds me of my comment earlier: https://news.ycombinator.com/item?id=9578307 https://news.ycombinator.com/item?id=9578307 And plenty of people schooling me that git is already distributed (git yes, github no). I am happy that someone is working on this.

ackalker 11y ago

A very interesting idea, GitTorrent, but I have one question which comes to me whenever I read about a delta-based distribution scheme: who is going to generate and share all those deltas? Some Linux distributions have experimented with delta-based package repositories, examples are deltup for Arch Linux and rpm-delta for RPM-based distros. Some of the known issues are: - choosing the number and spacing between deltas. Fine-grained deltas require more storage space, coarse-grained deltas require more download bandwidth. - retiring old deltas: periodically deleting all deltas older than a certain version, replacing them with the full package of that version. Again a trade-off between storage space and download bandwidth. For Git repositories, this would roughly translate to: - choosing the number, history spacing, and size of the Git packs per repository. - retiring old Git packs: periodically deleting Git packs older than a particular revision, replacing them with a bare repository at that revision.

pjc50 11y ago
Git is built out of deltas. You're already storing all of them.
- ackalker 11y ago
  The problem is not with deltas between revisions (i.e. the commits themselves) but with the Git packs spanning multiple revisions. At the time of a `git pull`, a user's repository can be at any revision between initial and latest. Who is going to seed (and keep seeding) packs for all of those possible revision intervals?
- deleted 11y ago
  [deleted]
- peterwaller 11y ago
  This is isn't quite how I would describe it. Git does do delta compression in packfiles, but the fundamental primitives lack deltas. It's just: * The contents of this directory is this list of of files whose contents have these SHAs. This is called a "tree". The SHA of a tree is also an object, and can appear in another tree. To see this for yourself, in any git repository run `git cat-file -p HEAD`. You'll see the (more or less) raw commit object for HEAD, which will point at a tree SHA. To see the contents of that tree-sha, run `git cat-file -p <the tree SHA>`. That tree object has a one-to-one correspondence with what you'll see on-disk in the objects directory, (if the object has not been put in a pack file). Above I have more or less fully described the contents of the files found in `.git/objects`. The delta'ing doesn't happen until later, if and when packfiles are constructed. But they're just a storage/bandwidth optimisation. AFAICT, these deltas have nothing to do with what you might think of as "git diff", which is just some fancy porcelain which looks at objects. The nice property of the construction is that given a large tree, even if nested, if you change a single file in that tree, you will only change as many trees as the file is deep in the tree, so computing changes between two nearby trees can usually be done quickly.

rektide 11y ago

Lovely work! We ask for the commit we want and connect to a node with BitTorrent, but once connected we conduct this Smart Protocol negotiation in an overlay connection on top of the BitTorrent wire protocol, in what’s called a BitTorrent Extension. Then the remote node makes us a packfile and tells us the hash of that packfile, and then we start downloading that packfile from it and any other nodes who are seeding it using Standard BitTorrent. The disadvantage here is that any given file in the repo could be stored in an ever increasing number of packfiles. Each existing version of the repo will generate a new packfile to get it to the newest version, and it's up to the authenticated masters to generate and seed each of those packfiles, while peers either do or do not cache these replicated datas. In short, this means of syndicating updates ignores the Merkle-DAG-ness (DAG-osity?) of Git. The un-updateability of torrents is something that seems to seriously limit it's use. There are a lot of interesting attempts to hack around this- LiveStreaming and Nightweb are two that spring to mind. https://www.tribler.org/StreamingExperiment/ https://www.tribler.org/StreamingExperiment/ https://sekao.net/nightweb/protocol.html https://sekao.net/nightweb/protocol.html

minot 11y ago
> The un-updateability of torrents is something that seems to seriously limit it's use. I am not involved with the development of torrents at all but (please bear with me until the end) my initial reaction is that we should think of the lack of ability of torrents to update as a feature and not as a bug. Perhaps if ability of torrents to update is a concern then it warrants a new peer to peer protocol? (Please note that this is not the case of http://xkcd.com/927/?cmpid=pscau http://xkcd.com/927/?cmpid=pscau as I am not advocating a new protocol for every use case) It seems like we can sign torrent files with gpg keys. Perhaps I am wrong. Perhaps, we can allow updating in torrents if we require that the updates be signed with the same private key as the original torrent? Am I barking up the right tree here? Edit: Oops. I edited this post before I saw the reply about BEP-0039. Apologies.
- dpiers 11y ago
  Support for mutable/updateable torrents was proposed in BEP-0039 (http://www.bittorrent.org/beps/bep_0039.html http://www.bittorrent.org/beps/bep_0039.html) and is the basis of how BitTorrent Sync functions.

durin42 11y ago

> Google Code announced its shutdown a few months ago, and their rationale was explicitly along the lines of “everyone’s using GitHub anyway, so we don’t need to exist anymore”. I mean, sorta. It was also because running a service is expensive, and containing abuse is a constant thankless treadmill. > We’re quickly heading towards a single central service for all of the world’s source code. Far from it? Not that a fully-decentralized system seems bad, but there are many things that aren't github. I don't even have anything of interest on github.

DannoHung 11y ago

Sweet as hell. My only quibble is the idea of using the blockchain for validated naming. I think it'd end up in a landgrab which is nasty. As much as we all hate DNS, having the ability to kick a squatter off someone's name is probably a good thing. Personally I don't think having a crazy hash for identification is a bad thing. Rather, what you need to do is just have some sort of reasonable personal contact book so you only have to deal with the crazy hash once (when you decide you want to remember that someone is who they said they were).

cjbprime 11y ago
I don't really understand the landgrab complaint: this costs $0.08 per username registered ($0.16 if you want to avoid races with miners), in comparison to centralized sites, which usually cost $0 per username registered. Why does this lose?
- DannoHung 11y ago
  Centralization is about equivalent to ICANN being able to kick off a squatter. A truly decentralized system with first to register doesn't have a way to kick off squatters.
  - icebraining 11y ago
    Well, the current ones don't, but it could. Decentralized networks just need decentralized decision, ie., you could have a process by which the majority of the nodes decide to kick off a squatter.
    - tw04 11y ago
      At which point someone uses a botnet to game the system.
namecast 11y ago
Hmm. How about mixing and matching, like by having a DNS record with a special TXT record syntax, e.g. 'IN TXT "gt=yoursha256hash"'. That way you could find your destination via DNS and verify the hash in the TXT record against the blockchain.

pjc50 11y ago

I'm very impressed by this, and I'm a bitcoin skeptic and git usability critic.

ossreality 11y ago
so brave

tyuwan 11y ago

sdfs

Skunkleton 11y ago

I could see this integrating really nicely with mailing lists. The commit could be sent out to the mailing list, and anyone who is interested in reviewing the code or doing a merge would already have the information they need, no blockchain required.

physcab 11y ago

This is an aside but I like the author's writing style. Not only is he clear but also describes why he's doing what he's doing and how it's important to him and others. Really helps me give the ideas more thought!

tr4s 11y ago

How is the new value for a mutable key propagated through the DHT? (as in how sure am I to get the newest commit hash?)

kodisha 11y ago

decentralized web interface? Isn it better to develop native, cross platform app for that? We have both electron and NW.js now, that should be easy.

deleted 11y ago

[deleted]

simplemath 11y ago

Well, thats a winning name, to be certain. Pretty great work too.

k2enemy 11y ago

Forgive me if my comment too readily reveals my ignorance on these topics, but would the IPFS project [0] be useful for this? [0] http://ipfs.io http://ipfs.io

tperson 11y ago
Definitely, checkout their example on how to host git on ipfs[0] [0] http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/git/readme.md http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsN...
- e12e 11y ago
  While integration between git and ipfs is good -- I'm not sure I see how this is much different from just tar-ing up a git-archive and creating a torrent/magnet (setting aside many of the other aspects of how ipfs and torrents work). Decentralized git pull for only a given hash isn't all that interesting? If one could pull in updates and/or push changes -- that would be "decentralized git". This is example is more "ipfs.io as a transport for git-releases", rather than "ipfs.io as a transport for git"?
- chriscool 11y ago
  There is this script I worked on to help doing that: https://github.com/whyrusleeping/git-ipfs-rehost https://github.com/whyrusleeping/git-ipfs-rehost

hellbanner 11y ago

Well done. I think this is a great piece to build a "pay for branch merging" way of promoting open source.. use decentralized currency and voila, automated programming.

shea256 11y ago

Love this. The post mentions using the blockchain for unique username registration and mapping to public key hashes, and as it turns out there's a project I and others have been working on that does exactly this called Blockstore. Here's the link if anyone wants to check it out: https://github.com/namesystem/blockstore https://github.com/namesystem/blockstore The way it works is there's a mapping between a unique name and a hash in the blockchain, and then there's a mapping in a DHT from the hash to the data to be associated (which can be a plain old public key and can also be a JSON file that references a public key and other identity information).

cjbprime 11y ago
(GitTorrent author here.) That's great, thanks! I should just use this (preferably with the DHT I'm already using to look up Git commits) instead of reimplementing myself. What do you think about the idea of making pluggable modules to connect Blockstore with web frameworks (Django, Rails), without the framework/website authors having to get involved in understanding Bitcoin themselves?
- deleted 11y ago
  [deleted]
- shea256 11y ago
  Oh I love that idea. As far as Django, Blockstore is on Pypi so you can just install and import the library. I think it'd be great to have modules for Rails, Node, etc. I saw your tweet and I'll shoot you an email. Also feel free to open an issue on github.com/namesystem/blockstore and we can discuss the idea openly there.

ArneBab 11y ago

> It surprised me that nothing like this seems to exist already in the decentralization community. There was a GSoC project in 2013 which did exactly this, using Freenet as decentralized storage backend with a Web of Trust for Spam resistant and updatable identities (note that in the gittorrent scheme once someone claimed a username, that username will stick there forever). It works and compared to GitTorrent it adds anonymity and upload-and-run. A current article describing it is here: http://draketo.de/english/freenet/real-life-infocalypse http://draketo.de/english/freenet/real-life-infocalypse (it got referenced here, too: https://news.ycombinator.com/item?id=9562749 https://news.ycombinator.com/item?id=9562749 ) The GSoC project was done by Steve Dougherty: http://www.google-melange.com/gsoc/project/details/google/gsoc2013/sdough/5650716672655360 http://www.google-melange.com/gsoc/project/details/google/gs... > I’d be happy to work on a project like this and make GitTorrent sit on top of it, so please let me know if you’re interested in helping with that. Have a look at Gitocalypse: https://github.com/SeekingFor/gitocalypse https://github.com/SeekingFor/gitocalypse

comex 11y ago

If a version of this with friendly name support is released, I will mirror all my active GitHub repositories there. If someone builds on this, as discussed elsewhere in the thread, to make a decentralized service that mimics 'social' functionality such as issues and pull requests, I will strongly consider using it instead of GitHub (depending on the UI, stability, etc.). I don't even have any particularly popular repos, so there is no real reason for anyone to care about the above, but, y'know, HN comments approving of the idea don't necessarily translate into actual interest in the product, so now you know there's at least one person in the latter category. :)

erikb 11y ago

Git itself is already distributed. Do we really need more distribution or do we simply need to learn to work more distributed?

ajkjk 11y ago
It's clear, though, that there are aspects of 'distributed' that Git does not accomplish. This gives you a global namespace and a global identity, just like a website, without a single point of failure or a corporate interest, and with a closed source system and a trivial option for forking the whole structure into your own [friend group|company|private cluster|etc]. An argument could be made that git ought to be augmented with something like this: why run distributed protocol A on distributed protocol B; maybe we should just run distributed protocol AB from the get-go?

placebo 11y ago

Great. Centralism and monopoly in any form makes me uneasy. This is another point for freedom...

laumars 11y ago

I don't understand why so many people are in awe of this project when it seems to be based on a number of falsehoods: > "imagine someone arguing that we can do without BitTorrent because we have FTP. We would not advocate replacing BitTorrent with FTP, and the suggestion doesn’t even make sense! First — there’s no index of which hosts have which files in FTP, so we wouldn’t know where to look for anything." Actually there is. It's called a mirror list. Most FTP-based repositories support this. > "And second — even if we knew who owned copies of the file we wanted, those computers aren’t going to be running an anonymous FTP server." Except bittorrent does turn your client into a server. Many clients silently punch holes in your firewall via uPnP, so you don't always realise you're running a server, but it does still happen. And as for anonymous FTP servers, it depends on what you mean there. If you mean anonymous access, then that's not only supported, but actually the norm. If you mean the server itself is anonymous, then it should be noted that neither github nor torrent seeding peers are anonymous either. > "Just like Git, FTP doesn’t turn clients into servers in the way that a peer-to-peer protocol does. So that’s why Git isn’t already the decentralized GitHub — you don’t know where anything’s stored, and even if you did, those machines aren’t running Git servers that you’re allowed to talk to. I think we can fix that." Hang on, a moment ago you _didn't_ want to run servers. Now you're complaining that git clones aren't servers? ----- Then there's the matter of the github competitors, of which there are many. gitlab, gitbucket, etc. Some open source, some closed but free, but all of them largely offer the same features as github. These days it seems trendy to use bittorrent as a bootstrap for all kinds of wacky and wonderful problems, but using bittorrent for a protocol that's already distributed and already pretty saturated with github competitors; well it just seems redundant.

ajkjk 11y ago
I think your complaints are disingenuous. [edit: what I mean is that you've pointed out problems with small components of the overall article and used them as an argument against the article as a whole, which I think is unfair. I don't think the components' validity affects the overall idea.] His argument may have some holes, and people are probably mostly ignorant of those holes so they can't critique them, but I don't think they're in awe of the idea because their awe hasn't been dispelled by identifying those holes. This is a very neat idea, and none of the issues with the setup argument dispel that. "using bittorrent for a protocol that's already distributed and already pretty saturated with github competitors; well it just seems redundant." no, no, it doesn't seem redundant. DHT based distributed indexing is so incredibly fundamentally different than a mirror list for files in FTP or from a series of GitHub clones. It's owner-agnostic. It just exists, by virtue of having participants, with no overhead. I don't have to select a target host or find a server or even identify where my particular file (or git repository, or username, or whatever) lives. It's .. unification. It's elegant and reduces complexity and makes the whole ecosystem more simple. Maybe I'm blinded by my own awe, but, I love this idea. Also, yes, some people have thought of components of this before, but I haven't really seen the full stack laid out vertically like this, combined with a narrative that makes me so excited about it.

4684499 11y ago

I don't think this is new. http://code.google.com/p/gittorrent/ http://code.google.com/p/gittorrent/

ajjai 11y ago

nice

mixmastamyk 11y ago

Interesting, I'm also very interested in the other project mentioned "bugs everywhere." Anyone have more details on day to day usage of it?

joe563323 11y ago

Just awesome idea. The idea of having a torrent for git repos blew my mind.

decentrality 11y ago

This is awesome, but it centralizes on JavaScript. It is an implementation of a standard without the standard being defined so other implementations can spring up. Git, one could argue, is language centralized also, which is technically true. That I don't have an answer for. But I don't believe handing off so much dependence to a JavaScript application fits for me. A C/C++ application like Git I can overlook, at least for a decade or so, but JavaScript feels like a perpetual beta/prototype only. Granted, that's my subjective feeling. Raised issue: https://github.com/cjb/GitTorrent/issues/12 https://github.com/cjb/GitTorrent/issues/12

swombat 11y ago
The word "centralize" in the way you're using it is very misleading. Every piece of programming in existence uses some sort of language. What you're really stating, I think, is that this is written in Javascript and you don't like Javascript. That's totally fine, and it's your prerogative. I'm sure that, like any other piece of programming, GitTorrent can be rewritten in other languages. If Javascript bothers you that much, then do this: rewrite it in Ruby if that's what you prefer. But please stop attacking the claim that this is "decentralised github" by claiming that this "centralises" on Javascript. It doesn't "centralise" on any language. It's just a first implementation written in Javascript.
- decentrality 11y ago
  That this implementation being uncomfortable for me is my subjective feeling, I've mentioned several times. That I don't want to depend on JavaScript is clear. I don't want to depend on JavaScript. Using the word centralize the way I have, I can't see otherwise. But I'm not defaming anyone. I'm not attacking anything. I'm very motivated to help the standard advance, but I would never for a moment install this implementation. Stupid of me? Sure, maybe. But it's not an attack. What I wanted I already got, confirmation by the OP that this is in fact a reference implementation and not the all-end-all. Were that clear to start with, I wouldn't have commented other than to say, awesome! This is extremely exciting to me.

z3t4 11y ago

I always thought Github was open source and that you could fork it, like in the spirit of Github. I find that slightly ironic =) What's stopping a decentralized github from ending up in the same fate as the newsgroups, that the data set gets too large to handle !? I think that Github is more then just a repository, it's a community. I kinda quit Facebook and signed up to Github instead :P And if it weren't for Github I would have never touched git. Startup idea: Create something like Github and assembly.com, but with a complete tool-set (git+vps)

hobarrera 11y ago

I like the idea of decentralized, but I'm not sure we need p2p as well. It'd be interesting to see something like gitlab to have some sort of federation support; where multiple instances can talk to one another (a la xmpp/smtp), so as to clone/send pull-requests, etc across different instances.

sytse 11y ago
We would love to see features like that in GitLab http://feedback.gitlab.com/forums/176466-general/suggestions/5097708-implement-cross-server-federated-merge-requests http://feedback.gitlab.com/forums/176466-general/suggestions...

pdudits 11y ago

What I miss in the article or discussion is consistency of refs. The moment your remote is a distributed one, it may have two different values for HEAD in single point of time in different parts of network. If two clients push in this time, HEADs diverge even further. So as in distributed databases, you either: * need to acquire exclusive lock on the repository metadata, or * accept, that your push will be eventually discarded because you did not have up to date metadata