16 ms·

Show HN: Decentralized, Mutable, Serverless Torrent Swarm Websites

433 points by publiush 5y ago

peterth3 5y ago
> This is great for blogging, whistle blowing, and other things. Could this support user authentication for uploading content? Maybe with something like metamask? Using this tech to make a medium/substack/twitter-style site would lower the barrier to entry for non-technical journalists.
petre 5y ago
WebTorrent has never worked for me. It's included in Brave now but whenever I try to open a magnet link it just sits and waits while qbittorrent downlads everything i before I can manage to make a cup of coffee.
- nabakin 5y ago
  > Can WebTorrent clients connect to normal BitTorrent clients? > In the browser, WebTorrent can only download torrents that are seeded by a WebRTC-capable torrent client. https://webtorrent.io/faq https://webtorrent.io/faq
publiush 5y ago
This is a project I've been working on to create decentralized, peer to peer, "serverless" websites using several technologies including webtorrent, dmt (mutable torrent BEP-46), and handshake decentralized domains. You can create either an immutable (uneditable) torrent site, or you can create a ed25519 keypair and create a mutable (updatable) torrent site. This is great for blogging, whistle blowing, and other things. It also scales well since torrent technology is great although it had previous been pigeonholed to other use cases. Please give it a try and let me know your thoughts! I don't take any credit for this since I just weaved the great technologies others already made together!
michaelscrypt 5y ago
How is this different from ZeroNet [https://zeronet.io/ https://zeronet.io/]?
- ccakes 5y ago
  At a quick glance, no Bitcoin
  - nephanth 5y ago
    Iirc, zeronet used to not have bitcoin before the craze, so this would be like early zeronet I guess
  - zcw100 5y ago
    No but Handshake does use coins which I assume to be some sort of blockchain "Handshake uses a coin system for name registration"
- generalizations 5y ago
  Also, it's a variation on the concept. It's really good when we have multiple implementations of a general design.
- sprash 5y ago
  Seems to be not much different from zeronet. It even makes the exact same mistakes like not building in anonymization by default (preferably using i2p). Zeronet hovever is much more mature and has a very active community.
- xrd 5y ago
  Is it just me, or is that site down?
algo_trader 5y ago
Well done, good luck. Always wanted to implement something like this. > dmt (mutable torrent BEP-46), Is dmt supported (made scalable?!) by the existing torrent infra structure (DHT/trackers/etc??). Sorry if this question doesnt make sense.
- algo_trader 5y ago
  Also, is this suitable for publishing an RSS-like feed which out polluting the world with a new torrent for each atom?
  - publiush 5y ago
    Yes and no - DMT (https://github.com/lmatteis/dmt https://github.com/lmatteis/dmt) is implemented into this so you can use a single hash in the DHT for the 'site', but that will be updated to point to a new torrent infohash on every update.
  - easrng 5y ago
    It depends. You an either have your mutable torrent point to the head of a liked list like this: head | |- post.txt +- prev.torrent prev | |- post.txt +- prev.torrent You get the idea. With the not yet widely-supported BitTorrent v2, you can just add files to a new torrent and seeders of the old torrent will seed the files that are also in the new torrent too.
    - toomuchtodo 5y ago
      What are the challenges slowing the v2 deployment? Client uptake?
      - bscphil 5y ago
        My reading of the situation is that for 99.99% of data currently transferred by Bittorrent, v1 works just fine. And for many use cases, the ability to do something like change an already created torrent would actually be detrimental. It's not that v2 is actively bad, it's just that it's designed for a different audience than currently uses Bittorrent, and will take time for that new audience to find it.
f0e4c2f7 5y ago
Seems cool at first glance! I've never heard of Handshake, will be looking at that more in particular. So I see the address in the screenshot starts with federalist:// Is it possible to access these sites with a normal web browser from the internet too?
- oscargrouch 5y ago
  from the look of it, you will need to have the handshake resolver installed and replaced as your local dns resolver. With that it will return you a the public key address on the bittorrent DHT which can be solved by some torrent client. There it should have a torrent info payload that your client can turn into a ordinary torrent, where you can proceed to download the files, and can open in your browser on your local filesystem.
- publiush 5y ago
  Thank you! I do think there is a way - since the beautiful WebTorrent (https://github.com/webtorrent/webtorrent https://github.com/webtorrent/webtorrent) can do so in browser. I'm keen to see something like this in a normal web browser (if possible as an extension even), hopefully developed by someone with better skills than me haha!
nynx 5y ago
This is what decentralization is about, not crypto.
- xwvvvvwx 5y ago
  this literally uses a cryptocurrency (handshake) for domains
  - publiush 5y ago
    There's a dilemma as the magnet links/hashes aren't easily shareable. One option is to create a DMT directory, but this would be centralized. Handshake is the most mature decentralized domain name project, and I opted for it. It uses coins to limit abuse, since anyone can flood a decentralized system. You don't need coins to browse and use federalist. That said, if there are any other immutable DNS systems that aren't centrally controlled that I could review, I'll definitely take a look!
    - panick21_ 5y ago
      Have you checkout out the Gnu Name System from GNUNet?
      - publiush 5y ago
        The website for GNUnet seems to be down/404, but it looks like ownership of names is controlled by a central authority (https://manpages.debian.org/unstable/gnunet/gnunet-namestore-fcfsd.1.en.html https://manpages.debian.org/unstable/gnunet/gnunet-namestore...).
        summm 5y ago
        If you want to do anything decentralized, I highly recommend to read all of the scientific papers from the gnunet authors. They theoretically solved a lot of problems a long time ago that modern distributed projects keep repeating. About the name system: It is a bit more complex than that. Everybody can "create" a domain, but others must import it's public key, somewhat compatible to a hosts file. The trick is that each domain can have arbitrary subdomains, also stored in the DHT. Now one can construct arbitrary deep trees. And everyone can choose a list of their trusted TLDs, and use them to resolve names. Say site.alice.bob.gnu (where .gnu is shipped with the client as default) and if I personally decide to trust Alice directly, I put her public key into my config file and from now on I can use site.alice instead without ever touching bob.gnu or .gnu again. Their DHT comes with unusual privacy guarantees due to clever cryptography. Furthermore gnunet already researched filesharing using content addressed blocks to allow for deduplication. Very clever. Unfortunately the implementation is not very usable and kind of stale. Some of these file sharing concepts are also better implemented by Freenet, which is a kind of anonymous decentralized web, even somewhat usable. A pity that those projects seem kind of stuck.
timbit42 5y ago
Would this work over I2P?
- publiush 5y ago
  BitTorrent works on I2P so I believe this should as well but I haven't tried.
  - throwaway41597 5y ago
    Hi, thank you for this! Is it wrong to say that without going through a privacy layer like I2P, you'd publicly link your IP to the decentralized websites you visit?
    - publiush 5y ago
      That is correct without I2P, Tor or a VPN.
      - timbit42 5y ago
        Tor doesn't support UDP so I don't think it would work.
RobLach 5y ago
Very cool and actualized proof-of-concept. "Decentralization" beyond a marketing term.
oscargrouch 5y ago
This is cool. I'm also working on a decentralized distribution mechanism based on torrents, and while i've being working on a different architecture, the network mechanism are basically the same with a couple of differences on the network level. In my case i'm working on a very customized version of chrome where the web api is actually available for native applications beyond Javascript.
- publiush 5y ago
  That sounds interesting! I would love to hear more!
  - lachstar-x 5y ago
    Have you considered routing through or utilizing the Session network as well? It's an improved version of Tor and you could possibly build this into a dApp on the Oxen network (oxen.io)
  - oscargrouch 5y ago
    Sure. is there a way to contact you?
    - publiush 5y ago
      Using SMTP, publiusfederalist and then the email provider is tutanota (.com)!
champagnois 5y ago
These sorts of things are still blockable through DNS and IP filters cutting off access to the root nodes and such I would guess. As such, what is the use case?
- publiush 5y ago
  Seeds for the torrents can change and, although not recommended, in terms of bloating the Tor network, using Tor will also help. With the recent attack on exposing Tor users, it may not be a bad thing if everyone starts torrenting on the Tor network actually. Everything, of course, can be blocked at some point, but the thing to remember is that there are other kinds of contracts that existed long before smart contracts - and these things are already leveraged in society. The internet isn't going to disappear tomorrow, but I would agree there are longer term risks, so let's build today. There's been a lot of talk about decentralization lately, and Cypherpunks write code. So I wrote code instead of a blog post.
- rektide 5y ago
  Wow that's a big huge dollop of nihilism, of why even bother. It's very unclear what your slam even means, what you are trying, technically, to express as the problem. Which makes your rejection even harder to handle with faith & respect. I also don't think it's accurate. Handshake is a cryptographically certified way of establishing identity . Since torrents are now mutable via handshake, it seems like webtorrents can be updated & moved as needed. Further, peer-exchange processes mean that having the initial seeds up probably isn't even a requirement. Even if one particular ip address or site gets cut down, the swarm can use other webtorrent trackers to re-spawn & carry-on. Aside from your criticism being either inaccureate or misleading, I'd also say the use case doesn't need to, imo, be 100% perfect in every way to be worthwhile. I'm glad someone did seek better, & bothered. A decentralized, updateable, browser-based torrent is incredible leap for a web of data, for interconnection. Even if it's not 100% completely uncensorable, it's many leaps in the right direction, towards decentralization. Especially decentralization without coordination/consensus, which I think is great & vast improvement over the harsh & strict type of computing that *coins have dominated the field with. This work is far more interesting to me.
derbOac 5y ago
How would you compare this in a use-case sense to something like Beaker browser? Also -- and I'm unfamiliar with handshake -- but is the sort of thing that could work over ad hoc networks?
- publiush 5y ago
  The Beaker browser looks interesting -- it looks a lot like DHT/DMT (https://hypercore-protocol.org/ https://hypercore-protocol.org/), but I speculate it's likely more specialized for the use case. Handshake names are decentralized and on chain, so as long as you have access to read the chain, it would work over ad hoc networks as well (and offline/local).
throwaway94294 5y ago
Isn't this just trusting whoever runs https://query.hdns.io https://query.hdns.io ? https://github.com/publiusfederalist/federalist/blob/3670867764ef8a914cbcbd9affcb2efa24700814/include/hresolver.js#L2 https://github.com/publiusfederalist/federalist/blob/3670867...
- publiush 5y ago
  Good catch! It's temporarily using hdns.io as many people still do not have an hsd node installed. A later version will be shipped with a light weight SPV resolver, at which point, the last piece of the puzzle will be complete.
mrtksn 5y ago
What are some cool websites that I can visit right now?
- publiush 5y ago
  It's new, so I don't think there are many out there yet. I do hope that this changes the landscape of the ecosystem for 'decentralization.' It's not supposed to be about tokens or organizational marketing hype. First, it's about free speech and freedom of information. "De-platforming" is now extinct. Only then, can you even begin to discuss anything else. I'm very thankful to WebTorrent, DMT, Handshake and Electron for making things possible.
  - mrtksn 5y ago
    Thanks, the work done is definitely cool but, as with any other decentralization project, I’m yet to find the content use cases. The only two kinds of successful content types that I’m aware of are pirated movies on Torrent and cryptocurrencies on the blockchain. They all depend on centralized discovery(torrent websites and exchanges). Which makes me wonder, are these decentralized websites or social media platforms attacking the right problems?
    - publiush 5y ago
      If someone wants to make a new kind of torrent website with federalist that doesn't rely on centralized servers, they now can! It would be helpful for me when I download my Ubuntu ISOs. ;)
    - rektide 5y ago
      > I’m yet to find the content use cases. It's an alternative transport, not application layer. If you can imagine doing anything with the world wide web , or ftp, you can imagine what this is for. It can transport hypertext files around, or whatever other file type you want. In the web case, it'd just be some local http files you could open in a local file origin. There'd be no server. But that's still a way to exchange whatever art or media you could ever imagine. We are bounded only by imagination. The internet is built around the Internet Protocol (IP), a way of streaming data arbitrarily from one computer to another. It has been up to us to imagine uses, to chase new possibilities. What do you think the use cases for IP are? Can you see what that made possible? Your question is in effect that, and trying to grasp at how broad, how possible, how potentiated this great work is is dauntingly hard, for we could share any type of content we want with either.
conradev 5y ago
BitTorrent is so good and I am surprised that I don’t see it in more places Using a private key for mutations is a great idea. I believe it is similar to how BitTorrent Live (undocumented) was implemented, where new content was added to a merkle tree. That is at least how RFC7574 proposed doing it
nubela 5y ago
Is there a mobile client?
gpm 5y ago
How do you handle security updates to Chromium?
- publiush 5y ago
  This project has just begun and is still in its infancy. My hope is that the community will come together to take it to another level. Chromium updates are generally handled by the Electron team!
betwixthewires 5y ago
Bravo dude! Absolutely phenomenal work. I'm going to star this and attempt to tinker with it in a few days. I love seeing stuff like this get built, IMO the more interesting tools like this get built the better the world becomes.
- publiush 5y ago
  Thank you so much!
newguyhere2 5y ago
this is actually nice, imagine a world where devices have software like this and acts as a node, so like install it on your smart coffee pot, tv, etc. seems silly but i think as time progresses and as technology advances and gets cheaper and cheaper to build that we will one day have normal everyday appliances os's in them, maybe even a type of wifi that will eventually create mesh networks
- bordercases 5y ago
  Suck It Jian Yang
dane-pgp 5y ago
This looks like a great project, thank you. One aspect of censorship resistance is the ability for people to publish and read content anonymously (or pseudonymously). Could you add something to the README about that threat model? Also, I see that your IRC channel is hosted on Freenode. That seems an odd choice for a project that is opposed to censorship. Please consider a network with a better track record in that regard.
- publiush 5y ago
  Thank you! It doesn't actually allow you to publish and read anonymously without Tor or a VPN, but I will definitely recommend using both of those on the README. I do want to be careful about what is and isn't claimed, as I believe the world has become a bit dissatisfied with overclaims from decentralization projects lately. I don't think I've come to the same conclusion about freenode (https://news.itsfoss.com/freenode-controversy/ https://news.itsfoss.com/freenode-controversy/), and IRC is only decentralized if there isn't just one network anyway. I also don't think this is the right forum for this anyhow! You're definitely free to setup the same channel (#scarywater) on another network, and I'd be happy to be present there as well!
  - southerntofu 5y ago
    > I don't think I've come to the same conclusion about freenode (https://news.itsfoss.com/freenode-controversy/ https://news.itsfoss.com/freenode-controversy/) This article really obscures some facts. For example, it implies that Lee was the person providing for freenode, when in fact he was just one of many donors: most servers in the network were donated by non-profits. Likewise, the article says Freenode operators were not staff because they were not paid: they still donated their time and resources to keeping the network alive. Also, it says Freenode was "taken over" by Christel, which to my knowledge is not exactly correct: from my understanding, she was indeed granted responsibilities, but was not supposed to govern (as in, give orders) to the network. Also, most of the drama took place after this article was published. Many chans which remained on Freenode were pushed out by the new management who seemingly ran regexes on all incoming messages, automatically taking over or banning any chan which had mentions of libera.chat. That is, until new management actually dropped the entire database (a huge "fuck you" to the faithful who remained) and started again from scratch. All in all, Freenode's story is one of a collective project run by volunteers on their donated time/hardware turned into a top-down startup. It also seems the freenode.net homepage has been turned into some kind of newsboard without a lot of people but where lots of content appears to be "deleted". Strange new world: https://freenode.net/n/freenode/57/there-are-literally-78-users-in-this-website-lmao https://freenode.net/n/freenode/57/there-are-literally-78-us... > IRC is only decentralized if there isn't just one network anyway. Not exactly, as there is server-to-server federation to distribute the load (but it's not exactly decentralized in this case). But yes, decentralizing is better. But there's plenty of IRC servers where to host your chans: OFTC, tilde.chat, indymedia, hackint...
lolspace 5y ago
What would happen if I created an illegal site? CP or selling guns?
- Aeolun 5y ago
  Presumably nobody would know it was yours until you added contact information to it.
- vgb2k18 5y ago
  If you created and seeded long enough to get active traffic, the host and DHT peer IPs might be flagged by LE. Once flagged the IP info passed to different investigators for prosecution purposes. Depending on your level of anonymity (VPN, tor, none, other) the investigation is either a dead-end or success. If success, warrant granted, home searched, PC seized. That's one potential scenario.
  - lolspace 5y ago
    But they can't take down the website?
- serverholic 5y ago
  People don’t have to use your website.
naanbread 5y ago
Does this mitigate any aspect of DDOS?
warabe 5y ago
Seems a great project! I've always dreamed creating something like this! One thing I want to know is, your project support users' anonymity?
- publiush 5y ago
  Out of the box, there is no anonymity built into federalist. You can use Tor or VPN to for anonymity. As this develops from a POC for developers to an end user software application, I imagine federalist will have many of these things built in directly or support to hook in (eg Tor and SOCKS) natively.
blooalien 5y ago
> I don't take any credit for this since I just weaved the great technologies others already made together! You should absolutely take credit for your part in this. Others may have created the technologies you used, but you tied them together into something "more" than the individual pieces. There's room enough for all players to take their credit for their part in this existing.
vgb2k18 5y ago
I would love it if someone could change my mind on this: The words unblockable and uncensorable have been misused in the documentation. Real-world example: 99% of my colleagues cannot bring their own devices into our corporate workplace to access torrents via DHT. There are some network rules and filters in place to block most non 443 traffic. Those 99% are effectively blocked. Setting a remote HAProxy server to mask miscelaneous traffic as https was a practical way out through the firewall, which then exposed my HAProxy server to a block by IP rules. This is an idea I'd love to be wrong about. Bock-resistant is the most appropriate statement for this project in its present form. Change my mind :)
- publiush 5y ago
  There are projects who are using collateral freedom to block censorship like Wireleap (https://wireleap.com/blog/routing-layer/ https://wireleap.com/blog/routing-layer/). This would help in the circumstance you describe.
- sudosysgen 5y ago
  You have to set up your https tunnel on AWS or GCP and then tunnel that to your own server. They will not range block AWS, and you can vary the IP every day.
everfree 5y ago
How does this compare to IPFS+ENS (InterPlanetary File System + Ethereum Name Service)?
xrd 5y ago
I cloned it. I don't see how to get to the publius or federalist scripts. I wanted to try to create a blog; is this documented somewhere? It's an exciting project.
- publiush 5y ago
  For federalist: git clone https://github.com/publiusfederalist/federalist https://github.com/publiusfederalist/federalist cd federalist npm install npm start For publius, there are two options, I'll start with option 1, which is an immutable (no updates) model: git clone https://github.com/publiusfederalist/publius https://github.com/publiusfederalist/publius cd publius npm install webtorrent supercop.js ./key ./publius & Then you can either share that magnet link/browse that link, or take the further step to add it as a TXT record on the Handshake blockchain for a name. If you would like to do it mutably, there is an additional step: ./seed & Then, you can share the link created by this and create the TXT record with this one instead. When you update your blog, you can edit the files in the web3root folder, and then simply restart publius and seed.
Geee 5y ago
This doesn't solve the problem that users have to host files themselves, which makes the network unsuitable for many use cases. With torrents and IPFS, only popular content will be hosted by voluntarists with varying levels of service. This is the drawback of traditional decentralized networks compared to blockchain-based networks such as Sia/Skynet, where users can pay to get their files hosted with guaranteed service level.
- publiush 5y ago
  You can pay seedbox services for hosting as well, but information the swarm cares about will continue to exist in the swarm. That being said, I think Sia/Skynet is a beautiful project and recommend everyone to check it out as well. I also saw a few other great projects listed in this thread which all sound fascinating.
neiman 5y ago
This is a fantastic work! The best part of Federalist is reading and displaying the websites from BitTorrent using BEP-46. The Handshake part is less impressive for me personally, since it basically reads Handshake values from a service (query.hdns.io/dns-query). If Federalist adds a Handshake daemon, it will be much more decentralized, but at the same time much heavier on resources. For historical perspective, decentralized websites (or p2p websites) are here since like 20 years. Here's a super concise list of what I recall off the top of my head. - Freenet is the first project I'm aware of doing that - ENS+IPFS websites is a strong movement now (see esteroids.eth.limo, I'm a co-creator of this) - ZeroNet was going strong like 5 years ago, and even though it's less popular now it's still functional and a nice project. - Agregor browser brings decentralized website the Hypercore (a successor of DAT protocol) - BitTorrent was supposed to have a project especially for decentralized websites (Maelstrom web browser), but it didn't took off eventually - Magnetico is a search engine built on top of BitTorrent, very similar to Federalists, but without the Handshake part. - Skynet is a project meant almost only for decentralized websites, based on Sia. They have integration with ENS, and afaik, also with Handshake (check it out!) - Swarm, Namecoin etc. could also be used for decentralized websites. I saw some people doing demos with it, but can't recall any links/names atm.
peter_retief 5y ago
Downloading, setting up and having a look. Using IPFS/IPLD at the moment. Will give feedback later
account-5 5y ago
How did this thread about a decentralised serverless I internet protocol end up in a debate about US gun policy?
- rvz 5y ago
  As the saying goes - ‘The road to hell is paved with good intensions’.
tluyben2 5y ago
Instead of blockchain/web3, I wish these type of projects would pop up more. They are drowned (in my circles) out by 'decentralized everything blockchain' hype... we need this indeed.
- serverholic 5y ago
  This does use blockchain...
  - tluyben2 5y ago
    Only the 'DNS' though right? Just the full-in-for-everything should be less as it does not work. But thanks for the correction.
rakoo 5y ago
Very nice work ! I wanted to do something similar a long time ago but never took the time. Congratulations ! A few points: - is this using webtorrent-the-library or webtorrent-the-protocol ? If it's the latter it's a bit useless. The only reason the protocol exists is to make web browsers speak a similar language as bittorrent, but both protocols are strictly different. Which means if you want to talk both you need to implement both (and fortunately webtorrent-the-library does so) - The Readme says this is good for whistleblowing. Actually this is one of the worst ways of doing it. When using DHT you tell the whole world your IP is interested in a specific content. When using bittorrent your IP connects with other peers interested in a specific content. There is nothing easier for LE than watching a specific content of interest and see who is interested in it: journalists, sources, ... An anonymisation layer is mandatory for this use case
- walteweiss 5y ago
  What would an anonymisation layer be in this case? Would a dedicated VPN or Tor be enough?
  - rakoo 5y ago
    Depends on your threat model. A VPN is enough if you're protecting against your neighbor, but whistleblowing usually means you're acting against a company or a government. Tor is the minimum for this, but is not even enough.
nyolfen 5y ago
i remember dreaming about something like this almost 20 years ago, when understanding torrents drew me into learning about networking and computing generally. huge props, it fills me with a warm feeling to see that these systems are not just possible but manifest.

433 points by publiush 5y ago

> This is great for blogging, whistle blowing, and other things. Could this support user authentication for uploading content? Maybe with something like metamask? Using this tech to make a medium/substack/twitter-style site would lower the barrier to entry for non-technical journalists.

petre 5y ago

WebTorrent has never worked for me. It's included in Brave now but whenever I try to open a magnet link it just sits and waits while qbittorrent downlads everything i before I can manage to make a cup of coffee.

nabakin 5y ago
> Can WebTorrent clients connect to normal BitTorrent clients? > In the browser, WebTorrent can only download torrents that are seeded by a WebRTC-capable torrent client. https://webtorrent.io/faq https://webtorrent.io/faq

publiush 5y ago

This is a project I've been working on to create decentralized, peer to peer, "serverless" websites using several technologies including webtorrent, dmt (mutable torrent BEP-46), and handshake decentralized domains. You can create either an immutable (uneditable) torrent site, or you can create a ed25519 keypair and create a mutable (updatable) torrent site. This is great for blogging, whistle blowing, and other things. It also scales well since torrent technology is great although it had previous been pigeonholed to other use cases. Please give it a try and let me know your thoughts! I don't take any credit for this since I just weaved the great technologies others already made together!

michaelscrypt 5y ago

How is this different from ZeroNet [https://zeronet.io/ https://zeronet.io/]?

ccakes 5y ago
At a quick glance, no Bitcoin
- nephanth 5y ago
  Iirc, zeronet used to not have bitcoin before the craze, so this would be like early zeronet I guess
- zcw100 5y ago
  No but Handshake does use coins which I assume to be some sort of blockchain "Handshake uses a coin system for name registration"
generalizations 5y ago
Also, it's a variation on the concept. It's really good when we have multiple implementations of a general design.
sprash 5y ago
Seems to be not much different from zeronet. It even makes the exact same mistakes like not building in anonymization by default (preferably using i2p). Zeronet hovever is much more mature and has a very active community.
xrd 5y ago
Is it just me, or is that site down?

algo_trader 5y ago

Well done, good luck. Always wanted to implement something like this. > dmt (mutable torrent BEP-46), Is dmt supported (made scalable?!) by the existing torrent infra structure (DHT/trackers/etc??). Sorry if this question doesnt make sense.

algo_trader 5y ago
Also, is this suitable for publishing an RSS-like feed which out polluting the world with a new torrent for each atom?
- publiush 5y ago
  Yes and no - DMT (https://github.com/lmatteis/dmt https://github.com/lmatteis/dmt) is implemented into this so you can use a single hash in the DHT for the 'site', but that will be updated to point to a new torrent infohash on every update.
- easrng 5y ago
  It depends. You an either have your mutable torrent point to the head of a liked list like this: head | |- post.txt +- prev.torrent prev | |- post.txt +- prev.torrent You get the idea. With the not yet widely-supported BitTorrent v2, you can just add files to a new torrent and seeders of the old torrent will seed the files that are also in the new torrent too.
  - toomuchtodo 5y ago
    What are the challenges slowing the v2 deployment? Client uptake?
    - bscphil 5y ago
      My reading of the situation is that for 99.99% of data currently transferred by Bittorrent, v1 works just fine. And for many use cases, the ability to do something like change an already created torrent would actually be detrimental. It's not that v2 is actively bad, it's just that it's designed for a different audience than currently uses Bittorrent, and will take time for that new audience to find it.

f0e4c2f7 5y ago

Seems cool at first glance! I've never heard of Handshake, will be looking at that more in particular. So I see the address in the screenshot starts with federalist:// Is it possible to access these sites with a normal web browser from the internet too?

oscargrouch 5y ago
from the look of it, you will need to have the handshake resolver installed and replaced as your local dns resolver. With that it will return you a the public key address on the bittorrent DHT which can be solved by some torrent client. There it should have a torrent info payload that your client can turn into a ordinary torrent, where you can proceed to download the files, and can open in your browser on your local filesystem.
publiush 5y ago
Thank you! I do think there is a way - since the beautiful WebTorrent (https://github.com/webtorrent/webtorrent https://github.com/webtorrent/webtorrent) can do so in browser. I'm keen to see something like this in a normal web browser (if possible as an extension even), hopefully developed by someone with better skills than me haha!

nynx 5y ago

This is what decentralization is about, not crypto.

xwvvvvwx 5y ago
this literally uses a cryptocurrency (handshake) for domains
- publiush 5y ago
  There's a dilemma as the magnet links/hashes aren't easily shareable. One option is to create a DMT directory, but this would be centralized. Handshake is the most mature decentralized domain name project, and I opted for it. It uses coins to limit abuse, since anyone can flood a decentralized system. You don't need coins to browse and use federalist. That said, if there are any other immutable DNS systems that aren't centrally controlled that I could review, I'll definitely take a look!
  - panick21_ 5y ago
    Have you checkout out the Gnu Name System from GNUNet?
    - publiush 5y ago
      The website for GNUnet seems to be down/404, but it looks like ownership of names is controlled by a central authority (https://manpages.debian.org/unstable/gnunet/gnunet-namestore-fcfsd.1.en.html https://manpages.debian.org/unstable/gnunet/gnunet-namestore...).
      - summm 5y ago
        If you want to do anything decentralized, I highly recommend to read all of the scientific papers from the gnunet authors. They theoretically solved a lot of problems a long time ago that modern distributed projects keep repeating. About the name system: It is a bit more complex than that. Everybody can "create" a domain, but others must import it's public key, somewhat compatible to a hosts file. The trick is that each domain can have arbitrary subdomains, also stored in the DHT. Now one can construct arbitrary deep trees. And everyone can choose a list of their trusted TLDs, and use them to resolve names. Say site.alice.bob.gnu (where .gnu is shipped with the client as default) and if I personally decide to trust Alice directly, I put her public key into my config file and from now on I can use site.alice instead without ever touching bob.gnu or .gnu again. Their DHT comes with unusual privacy guarantees due to clever cryptography. Furthermore gnunet already researched filesharing using content addressed blocks to allow for deduplication. Very clever. Unfortunately the implementation is not very usable and kind of stale. Some of these file sharing concepts are also better implemented by Freenet, which is a kind of anonymous decentralized web, even somewhat usable. A pity that those projects seem kind of stuck.

timbit42 5y ago

Would this work over I2P?

publiush 5y ago
BitTorrent works on I2P so I believe this should as well but I haven't tried.
- throwaway41597 5y ago
  Hi, thank you for this! Is it wrong to say that without going through a privacy layer like I2P, you'd publicly link your IP to the decentralized websites you visit?
  - publiush 5y ago
    That is correct without I2P, Tor or a VPN.
    - timbit42 5y ago
      Tor doesn't support UDP so I don't think it would work.

RobLach 5y ago

Very cool and actualized proof-of-concept. "Decentralization" beyond a marketing term.

oscargrouch 5y ago

This is cool. I'm also working on a decentralized distribution mechanism based on torrents, and while i've being working on a different architecture, the network mechanism are basically the same with a couple of differences on the network level. In my case i'm working on a very customized version of chrome where the web api is actually available for native applications beyond Javascript.

publiush 5y ago
That sounds interesting! I would love to hear more!
- lachstar-x 5y ago
  Have you considered routing through or utilizing the Session network as well? It's an improved version of Tor and you could possibly build this into a dApp on the Oxen network (oxen.io)
- oscargrouch 5y ago
  Sure. is there a way to contact you?
  - publiush 5y ago
    Using SMTP, publiusfederalist and then the email provider is tutanota (.com)!

champagnois 5y ago

These sorts of things are still blockable through DNS and IP filters cutting off access to the root nodes and such I would guess. As such, what is the use case?

publiush 5y ago
Seeds for the torrents can change and, although not recommended, in terms of bloating the Tor network, using Tor will also help. With the recent attack on exposing Tor users, it may not be a bad thing if everyone starts torrenting on the Tor network actually. Everything, of course, can be blocked at some point, but the thing to remember is that there are other kinds of contracts that existed long before smart contracts - and these things are already leveraged in society. The internet isn't going to disappear tomorrow, but I would agree there are longer term risks, so let's build today. There's been a lot of talk about decentralization lately, and Cypherpunks write code. So I wrote code instead of a blog post.
rektide 5y ago
Wow that's a big huge dollop of nihilism, of why even bother. It's very unclear what your slam even means, what you are trying, technically, to express as the problem. Which makes your rejection even harder to handle with faith & respect. I also don't think it's accurate. Handshake is a cryptographically certified way of establishing identity . Since torrents are now mutable via handshake, it seems like webtorrents can be updated & moved as needed. Further, peer-exchange processes mean that having the initial seeds up probably isn't even a requirement. Even if one particular ip address or site gets cut down, the swarm can use other webtorrent trackers to re-spawn & carry-on. Aside from your criticism being either inaccureate or misleading, I'd also say the use case doesn't need to, imo, be 100% perfect in every way to be worthwhile. I'm glad someone did seek better, & bothered. A decentralized, updateable, browser-based torrent is incredible leap for a web of data, for interconnection. Even if it's not 100% completely uncensorable, it's many leaps in the right direction, towards decentralization. Especially decentralization without coordination/consensus, which I think is great & vast improvement over the harsh & strict type of computing that *coins have dominated the field with. This work is far more interesting to me.

derbOac 5y ago

How would you compare this in a use-case sense to something like Beaker browser? Also -- and I'm unfamiliar with handshake -- but is the sort of thing that could work over ad hoc networks?

publiush 5y ago
The Beaker browser looks interesting -- it looks a lot like DHT/DMT (https://hypercore-protocol.org/ https://hypercore-protocol.org/), but I speculate it's likely more specialized for the use case. Handshake names are decentralized and on chain, so as long as you have access to read the chain, it would work over ad hoc networks as well (and offline/local).

throwaway94294 5y ago

Isn't this just trusting whoever runs https://query.hdns.io https://query.hdns.io ? https://github.com/publiusfederalist/federalist/blob/3670867764ef8a914cbcbd9affcb2efa24700814/include/hresolver.js#L2 https://github.com/publiusfederalist/federalist/blob/3670867...

publiush 5y ago
Good catch! It's temporarily using hdns.io as many people still do not have an hsd node installed. A later version will be shipped with a light weight SPV resolver, at which point, the last piece of the puzzle will be complete.

mrtksn 5y ago

What are some cool websites that I can visit right now?

publiush 5y ago
It's new, so I don't think there are many out there yet. I do hope that this changes the landscape of the ecosystem for 'decentralization.' It's not supposed to be about tokens or organizational marketing hype. First, it's about free speech and freedom of information. "De-platforming" is now extinct. Only then, can you even begin to discuss anything else. I'm very thankful to WebTorrent, DMT, Handshake and Electron for making things possible.
- mrtksn 5y ago
  Thanks, the work done is definitely cool but, as with any other decentralization project, I’m yet to find the content use cases. The only two kinds of successful content types that I’m aware of are pirated movies on Torrent and cryptocurrencies on the blockchain. They all depend on centralized discovery(torrent websites and exchanges). Which makes me wonder, are these decentralized websites or social media platforms attacking the right problems?
  - publiush 5y ago
    If someone wants to make a new kind of torrent website with federalist that doesn't rely on centralized servers, they now can! It would be helpful for me when I download my Ubuntu ISOs. ;)
  - rektide 5y ago
    > I’m yet to find the content use cases. It's an alternative transport, not application layer. If you can imagine doing anything with the world wide web , or ftp, you can imagine what this is for. It can transport hypertext files around, or whatever other file type you want. In the web case, it'd just be some local http files you could open in a local file origin. There'd be no server. But that's still a way to exchange whatever art or media you could ever imagine. We are bounded only by imagination. The internet is built around the Internet Protocol (IP), a way of streaming data arbitrarily from one computer to another. It has been up to us to imagine uses, to chase new possibilities. What do you think the use cases for IP are? Can you see what that made possible? Your question is in effect that, and trying to grasp at how broad, how possible, how potentiated this great work is is dauntingly hard, for we could share any type of content we want with either.

conradev 5y ago

BitTorrent is so good and I am surprised that I don’t see it in more places Using a private key for mutations is a great idea. I believe it is similar to how BitTorrent Live (undocumented) was implemented, where new content was added to a merkle tree. That is at least how RFC7574 proposed doing it

nubela 5y ago

Is there a mobile client?

gpm 5y ago

How do you handle security updates to Chromium?

publiush 5y ago
This project has just begun and is still in its infancy. My hope is that the community will come together to take it to another level. Chromium updates are generally handled by the Electron team!

betwixthewires 5y ago

Bravo dude! Absolutely phenomenal work. I'm going to star this and attempt to tinker with it in a few days. I love seeing stuff like this get built, IMO the more interesting tools like this get built the better the world becomes.

publiush 5y ago
Thank you so much!

newguyhere2 5y ago

this is actually nice, imagine a world where devices have software like this and acts as a node, so like install it on your smart coffee pot, tv, etc. seems silly but i think as time progresses and as technology advances and gets cheaper and cheaper to build that we will one day have normal everyday appliances os's in them, maybe even a type of wifi that will eventually create mesh networks

bordercases 5y ago
Suck It Jian Yang

dane-pgp 5y ago

This looks like a great project, thank you. One aspect of censorship resistance is the ability for people to publish and read content anonymously (or pseudonymously). Could you add something to the README about that threat model? Also, I see that your IRC channel is hosted on Freenode. That seems an odd choice for a project that is opposed to censorship. Please consider a network with a better track record in that regard.

publiush 5y ago
Thank you! It doesn't actually allow you to publish and read anonymously without Tor or a VPN, but I will definitely recommend using both of those on the README. I do want to be careful about what is and isn't claimed, as I believe the world has become a bit dissatisfied with overclaims from decentralization projects lately. I don't think I've come to the same conclusion about freenode (https://news.itsfoss.com/freenode-controversy/ https://news.itsfoss.com/freenode-controversy/), and IRC is only decentralized if there isn't just one network anyway. I also don't think this is the right forum for this anyhow! You're definitely free to setup the same channel (#scarywater) on another network, and I'd be happy to be present there as well!
- southerntofu 5y ago
  > I don't think I've come to the same conclusion about freenode (https://news.itsfoss.com/freenode-controversy/ https://news.itsfoss.com/freenode-controversy/) This article really obscures some facts. For example, it implies that Lee was the person providing for freenode, when in fact he was just one of many donors: most servers in the network were donated by non-profits. Likewise, the article says Freenode operators were not staff because they were not paid: they still donated their time and resources to keeping the network alive. Also, it says Freenode was "taken over" by Christel, which to my knowledge is not exactly correct: from my understanding, she was indeed granted responsibilities, but was not supposed to govern (as in, give orders) to the network. Also, most of the drama took place after this article was published. Many chans which remained on Freenode were pushed out by the new management who seemingly ran regexes on all incoming messages, automatically taking over or banning any chan which had mentions of libera.chat. That is, until new management actually dropped the entire database (a huge "fuck you" to the faithful who remained) and started again from scratch. All in all, Freenode's story is one of a collective project run by volunteers on their donated time/hardware turned into a top-down startup. It also seems the freenode.net homepage has been turned into some kind of newsboard without a lot of people but where lots of content appears to be "deleted". Strange new world: https://freenode.net/n/freenode/57/there-are-literally-78-users-in-this-website-lmao https://freenode.net/n/freenode/57/there-are-literally-78-us... > IRC is only decentralized if there isn't just one network anyway. Not exactly, as there is server-to-server federation to distribute the load (but it's not exactly decentralized in this case). But yes, decentralizing is better. But there's plenty of IRC servers where to host your chans: OFTC, tilde.chat, indymedia, hackint...

lolspace 5y ago

What would happen if I created an illegal site? CP or selling guns?

Aeolun 5y ago
Presumably nobody would know it was yours until you added contact information to it.
vgb2k18 5y ago
If you created and seeded long enough to get active traffic, the host and DHT peer IPs might be flagged by LE. Once flagged the IP info passed to different investigators for prosecution purposes. Depending on your level of anonymity (VPN, tor, none, other) the investigation is either a dead-end or success. If success, warrant granted, home searched, PC seized. That's one potential scenario.
- lolspace 5y ago
  But they can't take down the website?
serverholic 5y ago
People don’t have to use your website.

naanbread 5y ago

Does this mitigate any aspect of DDOS?

warabe 5y ago

Seems a great project! I've always dreamed creating something like this! One thing I want to know is, your project support users' anonymity?

publiush 5y ago
Out of the box, there is no anonymity built into federalist. You can use Tor or VPN to for anonymity. As this develops from a POC for developers to an end user software application, I imagine federalist will have many of these things built in directly or support to hook in (eg Tor and SOCKS) natively.

blooalien 5y ago

> I don't take any credit for this since I just weaved the great technologies others already made together! You should absolutely take credit for your part in this. Others may have created the technologies you used, but you tied them together into something "more" than the individual pieces. There's room enough for all players to take their credit for their part in this existing.

vgb2k18 5y ago

I would love it if someone could change my mind on this: The words unblockable and uncensorable have been misused in the documentation. Real-world example: 99% of my colleagues cannot bring their own devices into our corporate workplace to access torrents via DHT. There are some network rules and filters in place to block most non 443 traffic. Those 99% are effectively blocked. Setting a remote HAProxy server to mask miscelaneous traffic as https was a practical way out through the firewall, which then exposed my HAProxy server to a block by IP rules. This is an idea I'd love to be wrong about. Bock-resistant is the most appropriate statement for this project in its present form. Change my mind :)

publiush 5y ago
There are projects who are using collateral freedom to block censorship like Wireleap (https://wireleap.com/blog/routing-layer/ https://wireleap.com/blog/routing-layer/). This would help in the circumstance you describe.
sudosysgen 5y ago
You have to set up your https tunnel on AWS or GCP and then tunnel that to your own server. They will not range block AWS, and you can vary the IP every day.

everfree 5y ago

How does this compare to IPFS+ENS (InterPlanetary File System + Ethereum Name Service)?

xrd 5y ago

I cloned it. I don't see how to get to the publius or federalist scripts. I wanted to try to create a blog; is this documented somewhere? It's an exciting project.

publiush 5y ago
For federalist: git clone https://github.com/publiusfederalist/federalist https://github.com/publiusfederalist/federalist cd federalist npm install npm start For publius, there are two options, I'll start with option 1, which is an immutable (no updates) model: git clone https://github.com/publiusfederalist/publius https://github.com/publiusfederalist/publius cd publius npm install webtorrent supercop.js ./key ./publius & Then you can either share that magnet link/browse that link, or take the further step to add it as a TXT record on the Handshake blockchain for a name. If you would like to do it mutably, there is an additional step: ./seed & Then, you can share the link created by this and create the TXT record with this one instead. When you update your blog, you can edit the files in the web3root folder, and then simply restart publius and seed.

Geee 5y ago

This doesn't solve the problem that users have to host files themselves, which makes the network unsuitable for many use cases. With torrents and IPFS, only popular content will be hosted by voluntarists with varying levels of service. This is the drawback of traditional decentralized networks compared to blockchain-based networks such as Sia/Skynet, where users can pay to get their files hosted with guaranteed service level.

publiush 5y ago
You can pay seedbox services for hosting as well, but information the swarm cares about will continue to exist in the swarm. That being said, I think Sia/Skynet is a beautiful project and recommend everyone to check it out as well. I also saw a few other great projects listed in this thread which all sound fascinating.

neiman 5y ago

This is a fantastic work! The best part of Federalist is reading and displaying the websites from BitTorrent using BEP-46. The Handshake part is less impressive for me personally, since it basically reads Handshake values from a service (query.hdns.io/dns-query). If Federalist adds a Handshake daemon, it will be much more decentralized, but at the same time much heavier on resources. For historical perspective, decentralized websites (or p2p websites) are here since like 20 years. Here's a super concise list of what I recall off the top of my head. - Freenet is the first project I'm aware of doing that - ENS+IPFS websites is a strong movement now (see esteroids.eth.limo, I'm a co-creator of this) - ZeroNet was going strong like 5 years ago, and even though it's less popular now it's still functional and a nice project. - Agregor browser brings decentralized website the Hypercore (a successor of DAT protocol) - BitTorrent was supposed to have a project especially for decentralized websites (Maelstrom web browser), but it didn't took off eventually - Magnetico is a search engine built on top of BitTorrent, very similar to Federalists, but without the Handshake part. - Skynet is a project meant almost only for decentralized websites, based on Sia. They have integration with ENS, and afaik, also with Handshake (check it out!) - Swarm, Namecoin etc. could also be used for decentralized websites. I saw some people doing demos with it, but can't recall any links/names atm.

peter_retief 5y ago

Downloading, setting up and having a look. Using IPFS/IPLD at the moment. Will give feedback later

account-5 5y ago

How did this thread about a decentralised serverless I internet protocol end up in a debate about US gun policy?

rvz 5y ago
As the saying goes - ‘The road to hell is paved with good intensions’.

tluyben2 5y ago

Instead of blockchain/web3, I wish these type of projects would pop up more. They are drowned (in my circles) out by 'decentralized everything blockchain' hype... we need this indeed.

serverholic 5y ago
This does use blockchain...
- tluyben2 5y ago
  Only the 'DNS' though right? Just the full-in-for-everything should be less as it does not work. But thanks for the correction.

rakoo 5y ago

Very nice work ! I wanted to do something similar a long time ago but never took the time. Congratulations ! A few points: - is this using webtorrent-the-library or webtorrent-the-protocol ? If it's the latter it's a bit useless. The only reason the protocol exists is to make web browsers speak a similar language as bittorrent, but both protocols are strictly different. Which means if you want to talk both you need to implement both (and fortunately webtorrent-the-library does so) - The Readme says this is good for whistleblowing. Actually this is one of the worst ways of doing it. When using DHT you tell the whole world your IP is interested in a specific content. When using bittorrent your IP connects with other peers interested in a specific content. There is nothing easier for LE than watching a specific content of interest and see who is interested in it: journalists, sources, ... An anonymisation layer is mandatory for this use case

walteweiss 5y ago
What would an anonymisation layer be in this case? Would a dedicated VPN or Tor be enough?
- rakoo 5y ago
  Depends on your threat model. A VPN is enough if you're protecting against your neighbor, but whistleblowing usually means you're acting against a company or a government. Tor is the minimum for this, but is not even enough.

nyolfen 5y ago

i remember dreaming about something like this almost 20 years ago, when understanding torrents drew me into learning about networking and computing generally. huge props, it fills me with a warm feeling to see that these systems are not just possible but manifest.