8 ms·

GitTorrent: A Decentralized GitHub (2015)

314 points by mbroncano 6y ago

blitblitblit 6y ago
This project really needs more attention. Would have resolved the issue we had earlier today. Some others ideas: * Write a client in C, JS sucks * Make a web-frontend for this so average users can jump on quickly.
- RMPR 6y ago
  > This project really needs more attention. Would have resolved the issue we had earlier today. Some others ideas: The author doesn't think so though https://news.ycombinator.com/item?id=24876815 https://news.ycombinator.com/item?id=24876815
  - onelovetwo 6y ago
    did you actually read the comment or are your just spam replying that on every comment?
- 3np 6y ago
  There’s also git-ssb https://github.com/noffle/git-ssb-intro https://github.com/noffle/git-ssb-intro
  - cryptoquick 6y ago
    Unfortunately, SSB uses far more JS than they really should.
    - 3np 6y ago
      Regardless, just learned about https://radicle.xyz/ https://radicle.xyz/, which looks more active and mature than what I linked
- IggleSniggle 6y ago
  I understand the sentiment “JS sucks,” but not in relation to C for most things. Modern js is rather nice in syntax, and quite performant in V8. Heck, you can transpile it to C with Bellard’s QuickJS (or just compile directly to binary) if there’s some C library interop that your comment is based on (although nodejs has a perfectly good C++ plugin system for this).
nwah1 6y ago
Bittorrent V2 may offer some advantages that improve this idea.
makz 6y ago
Looks like a good fit for youtube-dl
- adammenges 6y ago
  Yeah it’s an apt time. Of course I’m not sure the legal request will stick.
- inquirerofsorts 6y ago
  The maintainers should probably just run their own git server as a hidden service, change their git name/email/signing keys and be done with it. Going after youtube-dl so publicly has a solid Streisand effect. Feel as though the lawyers aren't going to give up easily on this one and will hound them one way or another. Regardless, there is some github alternatives available through tor: Darktea: http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzhvyd.onion/ http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh... Rootgit: http://rootgit4rghbuenb.onion/explore/repos http://rootgit4rghbuenb.onion/explore/repos And on I2P: http://git.volatile.i2p/ http://git.volatile.i2p/ http://git.psi.i2p http://git.psi.i2p
  - anon9001 6y ago
    Switching to hidden services while we're building with decentralization does seem to be the pragmatic thing to do.
  - sj_krueger 6y ago
    Thank you. I have created the repo at Darktea, feel free to create your anonymous accounts and join the project, until we find a better/decentralized place. For now this should be sufficient: http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzhvyd.onion/sj.krueger/youtube-dl http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh... I will give the main contributors the neccessary access for reviewing/merging/etc.. or will entirely hand over the project if you can prove you are the maintainer.
- RMPR 6y ago
  The author doesn't agree with you though. https://news.ycombinator.com/item?id=24876815 https://news.ycombinator.com/item?id=24876815
nyolfen 6y ago
another promising project: https://pijul.org/ https://pijul.org/
- LEARAX 6y ago
  I believe the main developer of pijul has temporarily made his work closed-source for the past few years while he rewrites it to avoid receiving issues while it's unfinished. Looks like it's supposed to release soon, though [1]. [1] https://twitter.com/pijul_org/status/1319159283938983936 https://twitter.com/pijul_org/status/1319159283938983936
m-p-3 6y ago
And another angle, if you want to store a snapshot of a repo on IPFS https://github.com/whyrusleeping/git-ipfs-rehost https://github.com/whyrusleeping/git-ipfs-rehost
- avodonosov 6y ago
  This proviedes just read only publishing currently. Can one simply place a git repo in an IPFS directory (e. g. with some IPFS fuse implementation) and share an IPNS / DNSLink so that future changes can be found under the same address? In this aporoach github issues and pull requests can be replaced by discussions and patches in a mailing list.
- einpoklum 6y ago
  1. That's basically just a bash script though, not something very fancy. 2. How popular is IPFS? How easily usable by lay users?
  - m-p-3 6y ago
    How popular it is I don't know. It's easy to access through the ipfs.io public gateway, and you can use your own gateway if you host a node.
cjbprime 6y ago
Hi! I'm the GitTorrent author. It feels strange to have it show up on the front page whenever GitHub does something bad, more than five years later. Bitcoin and BitTorrent v1 were plausible substrates in 2015, but no longer. I think Radicle (https://radicle.xyz https://radicle.xyz) is the project most deserving of attention at the moment. The hardest thing about such a project is economic: how do you handle issue/comment spammers? Or someone creating as many accounts as they can? How do you incentivize someone to mirror your repo? Or to build you a delta to satisfy your `git pull`, without making the network fall over when someone realizes they can ask you to spend infinite CPU building packfiles for them? And what happens when someone forgets their password or loses their private key? This space was a humbling experience in the necessity of incentive alignment for me.
- ethn 6y ago
  Usenet solved this with a Bitcoin precursor, Adam Back's hashcash. In effect you can do the same thing better by using a cryptocurrency with cheap transactions, e.g. Bitcoin Cash's Memo.cash. This way git hosts are compensated while at the same time making it too costly to spam. Longer packfile creation processes will require a greater transaction fee or otherwise be ignored based on heuristics created by the client. This will prevent infinite CPU building packfiles issues in addition to fair compensation for increased computation cost. All while the miner is incentivized to minimize the time to be able to reap that block reward in the context of market competition.
  - marco_craveiro 6y ago
    Wrt crypto, this is the LBRY [1] approach I believe, and seems to work quite well. [1] https://lbry.tv/ https://lbry.tv/
- jeswin 6y ago
  Absolutely, because it is truly innovative - and you're mostly in uncharted waters. We're at peak cloud right now, and what projects such as yours offer is a glimpse into the post-cloud future. We're going to look back and say - what were we thinking when we decided to move compute and personal data storage into datacenters while carrying powerful supercomputers in our pockets. Add: > And what happens when someone forgets their password or loses their private key? Here's an approach I found interesting: https://darkcrystal.pw/ https://darkcrystal.pw/
Boulth6 6y ago
This is a solution to a non-problem. Hosting taken down git repos is easy an due to gits design all developers already have the source code. The real problem is hosting issues and PRs in such a way. Github has an API and it's possible to script the backup but source code gets backup automatically so when the takedown strikes it's not a big problem.
- hashworks 6y ago
  While others could have a backup of a Git repository, I need to find them first — so discoverability is a problem. Next thing is integrity - how do I know that the git history I receive has not been tampered with?
  - IanCal 6y ago
    Do signed commits solve the integrity problem?
    - hashworks 6y ago
      Yes. Git commits include the hash of the previous commit, creating a cryptographic chain. If you can verify the signature at the tip of a branch you effectively verify the complete history of the branch. Sadly not every maintainer signs their commits or tags.
  - Proven 6y ago
    1) Discoverability is not a problem. You don't need to discover anything, you just need to download the same repo again. If you forgot the name you're probably a super casual user (and not a dev) 2) Git history hasn't been tampered with just like it hasn't been tampered with in any of centralized Github repos. Doh.
- michaelmure 6y ago
  Shameless plug, but that's exactly the aim of https://github.com/MichaelMure/git-bug https://github.com/MichaelMure/git-bug. It's a distributed bug-tracker: it stores issues (and one day, PRs) within git. You can work offline and you always have a full copy of everything. It also has bridges for Github, Gitlab and Jira.
happybeing 6y ago
After thinking about this for a couple of years while helping the Safe Network (a decentralisation project worth getting to know) I began my own effort to decentralise github based on Safe, just last Sunday! Even before Friday's censorship, I was pleasantly surprised by the level of support and offers to help I received in mastodon. People there are aware, skilled and ready. BTW I've got ideas on how to handle the issue/comment spam and other problems rightly highlighted by @cjbprime in his reply to the OP. But first I have to get git-bug (really worth of support too) compiled to WASM and running in the browser. It's early days, but you can see what I'm up to here: https://safenetforum.org/t/safe-git-ui-discussion/32793?u=happybeing https://safenetforum.org/t/safe-git-ui-discussion/32793?u=ha... Or follow: https://mastodon.technology/@happybeing https://mastodon.technology/@happybeing https://twitter.com/safepress https://twitter.com/safepress
withinboredom 6y ago
I don’t understand the need for crypto and/or a global user name. I’m “withinboredom” in a lot of places and it doesn’t bother me when someone gets there first. I also have other names that are better well known, but I digress. Why not just stick a txt record on a domain? You can clone gittorrent://awesome.withinboredom.info. You can easily find a valid gpg key for me in the usual places.
- remram 6y ago
  Honestly some hash or public key would probably work fine. "withinboredom" means nothing to me (no offense) and it would be easy to add your ID as a contact or git remote and never deal with it again.
johndoe42377 6y ago
Why, yes, as long as each cloned repo is a copy of some set of revisions, it makes perfect sense to use peer to peer downloads. Internal hash consistently could be used for verification.
sschueller 6y ago
Wouldn't an activitypub[1] type of github be the perfect solution? Like Mastadon or Peertube but for git. Anyone can setup their own hosting and interconnect to others. There can be global index like peertube is building and spam can be dealt with on each instance. [1] https://www.w3.org/TR/activitypub/ https://www.w3.org/TR/activitypub/
- remram 6y ago
  This is being worked on: "ForgeFed" https://github.com/forgefed/forgefed https://github.com/forgefed/forgefed

314 points by mbroncano 6y ago

This project really needs more attention. Would have resolved the issue we had earlier today. Some others ideas: * Write a client in C, JS sucks * Make a web-frontend for this so average users can jump on quickly.

RMPR 6y ago
> This project really needs more attention. Would have resolved the issue we had earlier today. Some others ideas: The author doesn't think so though https://news.ycombinator.com/item?id=24876815 https://news.ycombinator.com/item?id=24876815
- onelovetwo 6y ago
  did you actually read the comment or are your just spam replying that on every comment?
3np 6y ago
There’s also git-ssb https://github.com/noffle/git-ssb-intro https://github.com/noffle/git-ssb-intro
- cryptoquick 6y ago
  Unfortunately, SSB uses far more JS than they really should.
  - 3np 6y ago
    Regardless, just learned about https://radicle.xyz/ https://radicle.xyz/, which looks more active and mature than what I linked
IggleSniggle 6y ago
I understand the sentiment “JS sucks,” but not in relation to C for most things. Modern js is rather nice in syntax, and quite performant in V8. Heck, you can transpile it to C with Bellard’s QuickJS (or just compile directly to binary) if there’s some C library interop that your comment is based on (although nodejs has a perfectly good C++ plugin system for this).

nwah1 6y ago

Bittorrent V2 may offer some advantages that improve this idea.

makz 6y ago

Looks like a good fit for youtube-dl

adammenges 6y ago
Yeah it’s an apt time. Of course I’m not sure the legal request will stick.
inquirerofsorts 6y ago
The maintainers should probably just run their own git server as a hidden service, change their git name/email/signing keys and be done with it. Going after youtube-dl so publicly has a solid Streisand effect. Feel as though the lawyers aren't going to give up easily on this one and will hound them one way or another. Regardless, there is some github alternatives available through tor: Darktea: http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzhvyd.onion/ http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh... Rootgit: http://rootgit4rghbuenb.onion/explore/repos http://rootgit4rghbuenb.onion/explore/repos And on I2P: http://git.volatile.i2p/ http://git.volatile.i2p/ http://git.psi.i2p http://git.psi.i2p
- anon9001 6y ago
  Switching to hidden services while we're building with decentralization does seem to be the pragmatic thing to do.
- sj_krueger 6y ago
  Thank you. I have created the repo at Darktea, feel free to create your anonymous accounts and join the project, until we find a better/decentralized place. For now this should be sufficient: http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzhvyd.onion/sj.krueger/youtube-dl http://it7otdanqu7ktntxzm427cba6i53w6wlanlh23v5i3siqmos47pzh... I will give the main contributors the neccessary access for reviewing/merging/etc.. or will entirely hand over the project if you can prove you are the maintainer.
RMPR 6y ago
The author doesn't agree with you though. https://news.ycombinator.com/item?id=24876815 https://news.ycombinator.com/item?id=24876815

nyolfen 6y ago

another promising project: https://pijul.org/ https://pijul.org/

LEARAX 6y ago
I believe the main developer of pijul has temporarily made his work closed-source for the past few years while he rewrites it to avoid receiving issues while it's unfinished. Looks like it's supposed to release soon, though [1]. [1] https://twitter.com/pijul_org/status/1319159283938983936 https://twitter.com/pijul_org/status/1319159283938983936

m-p-3 6y ago

And another angle, if you want to store a snapshot of a repo on IPFS https://github.com/whyrusleeping/git-ipfs-rehost https://github.com/whyrusleeping/git-ipfs-rehost

avodonosov 6y ago
This proviedes just read only publishing currently. Can one simply place a git repo in an IPFS directory (e. g. with some IPFS fuse implementation) and share an IPNS / DNSLink so that future changes can be found under the same address? In this aporoach github issues and pull requests can be replaced by discussions and patches in a mailing list.
einpoklum 6y ago
1. That's basically just a bash script though, not something very fancy. 2. How popular is IPFS? How easily usable by lay users?
- m-p-3 6y ago
  How popular it is I don't know. It's easy to access through the ipfs.io public gateway, and you can use your own gateway if you host a node.

cjbprime 6y ago

Hi! I'm the GitTorrent author. It feels strange to have it show up on the front page whenever GitHub does something bad, more than five years later. Bitcoin and BitTorrent v1 were plausible substrates in 2015, but no longer. I think Radicle (https://radicle.xyz https://radicle.xyz) is the project most deserving of attention at the moment. The hardest thing about such a project is economic: how do you handle issue/comment spammers? Or someone creating as many accounts as they can? How do you incentivize someone to mirror your repo? Or to build you a delta to satisfy your `git pull`, without making the network fall over when someone realizes they can ask you to spend infinite CPU building packfiles for them? And what happens when someone forgets their password or loses their private key? This space was a humbling experience in the necessity of incentive alignment for me.

ethn 6y ago
Usenet solved this with a Bitcoin precursor, Adam Back's hashcash. In effect you can do the same thing better by using a cryptocurrency with cheap transactions, e.g. Bitcoin Cash's Memo.cash. This way git hosts are compensated while at the same time making it too costly to spam. Longer packfile creation processes will require a greater transaction fee or otherwise be ignored based on heuristics created by the client. This will prevent infinite CPU building packfiles issues in addition to fair compensation for increased computation cost. All while the miner is incentivized to minimize the time to be able to reap that block reward in the context of market competition.
- marco_craveiro 6y ago
  Wrt crypto, this is the LBRY [1] approach I believe, and seems to work quite well. [1] https://lbry.tv/ https://lbry.tv/
jeswin 6y ago
Absolutely, because it is truly innovative - and you're mostly in uncharted waters. We're at peak cloud right now, and what projects such as yours offer is a glimpse into the post-cloud future. We're going to look back and say - what were we thinking when we decided to move compute and personal data storage into datacenters while carrying powerful supercomputers in our pockets. Add: > And what happens when someone forgets their password or loses their private key? Here's an approach I found interesting: https://darkcrystal.pw/ https://darkcrystal.pw/

Boulth6 6y ago

This is a solution to a non-problem. Hosting taken down git repos is easy an due to gits design all developers already have the source code. The real problem is hosting issues and PRs in such a way. Github has an API and it's possible to script the backup but source code gets backup automatically so when the takedown strikes it's not a big problem.

hashworks 6y ago
While others could have a backup of a Git repository, I need to find them first — so discoverability is a problem. Next thing is integrity - how do I know that the git history I receive has not been tampered with?
- IanCal 6y ago
  Do signed commits solve the integrity problem?
  - hashworks 6y ago
    Yes. Git commits include the hash of the previous commit, creating a cryptographic chain. If you can verify the signature at the tip of a branch you effectively verify the complete history of the branch. Sadly not every maintainer signs their commits or tags.
- Proven 6y ago
  1) Discoverability is not a problem. You don't need to discover anything, you just need to download the same repo again. If you forgot the name you're probably a super casual user (and not a dev) 2) Git history hasn't been tampered with just like it hasn't been tampered with in any of centralized Github repos. Doh.
michaelmure 6y ago
Shameless plug, but that's exactly the aim of https://github.com/MichaelMure/git-bug https://github.com/MichaelMure/git-bug. It's a distributed bug-tracker: it stores issues (and one day, PRs) within git. You can work offline and you always have a full copy of everything. It also has bridges for Github, Gitlab and Jira.

happybeing 6y ago

After thinking about this for a couple of years while helping the Safe Network (a decentralisation project worth getting to know) I began my own effort to decentralise github based on Safe, just last Sunday! Even before Friday's censorship, I was pleasantly surprised by the level of support and offers to help I received in mastodon. People there are aware, skilled and ready. BTW I've got ideas on how to handle the issue/comment spam and other problems rightly highlighted by @cjbprime in his reply to the OP. But first I have to get git-bug (really worth of support too) compiled to WASM and running in the browser. It's early days, but you can see what I'm up to here: https://safenetforum.org/t/safe-git-ui-discussion/32793?u=happybeing https://safenetforum.org/t/safe-git-ui-discussion/32793?u=ha... Or follow: https://mastodon.technology/@happybeing https://mastodon.technology/@happybeing https://twitter.com/safepress https://twitter.com/safepress

withinboredom 6y ago

I don’t understand the need for crypto and/or a global user name. I’m “withinboredom” in a lot of places and it doesn’t bother me when someone gets there first. I also have other names that are better well known, but I digress. Why not just stick a txt record on a domain? You can clone gittorrent://awesome.withinboredom.info. You can easily find a valid gpg key for me in the usual places.

remram 6y ago
Honestly some hash or public key would probably work fine. "withinboredom" means nothing to me (no offense) and it would be easy to add your ID as a contact or git remote and never deal with it again.

johndoe42377 6y ago

Why, yes, as long as each cloned repo is a copy of some set of revisions, it makes perfect sense to use peer to peer downloads. Internal hash consistently could be used for verification.

sschueller 6y ago

Wouldn't an activitypub[1] type of github be the perfect solution? Like Mastadon or Peertube but for git. Anyone can setup their own hosting and interconnect to others. There can be global index like peertube is building and spam can be dealt with on each instance. [1] https://www.w3.org/TR/activitypub/ https://www.w3.org/TR/activitypub/

remram 6y ago
This is being worked on: "ForgeFed" https://github.com/forgefed/forgefed https://github.com/forgefed/forgefed