5 ms·

ZeroNet and IPFS: uncensorable auto-scaling BitTorrent powered websites

97 points by npongratz 10y ago

mynameislegion 10y ago
Quote: "In terms of censorship resistance, IPFS includes blacklists of forbidden content in the default distribution, though it's configurable so you can turn it off 'at your own risk'. Personally I think all of this willful censorship stuff is just a phase to protect the protocol in its infancy. Blacklisting all forbidden content on a global filesystem is ridiculous. The only popular use case I see for that is blocking ads. So censorship-free distributions of IPFS implementations that integrate with Tor will probably end up as the default if this ever catches on." Unfortunately, IPFS' built-in censorship tools are not meant to be optional, nor are they temporary. https://github.com/ipfs/faq/issues/47 https://github.com/ipfs/faq/issues/47 is an FAQ entry written by an IPFS member. Quote: "IPFS has as a strict requirement that content be able to move as fast as the underlying network permits. this rules out designs like freenet's and other oblivious storage platforms, as the base case." "Oblivious storage platforms" are systems like Tahoe-LAFS or OFF, where storage nodes are unaware of the actual content that they are handling, usually due to some sort of encryption. Quote: "IPFS has as a design requirement that nodes be able to only store and/or distribute content they explicitly want to store and/or distribute. This means that computers that run IPFS nodes do not have to host 'other people's stuff', which is a very important thing when you consider that lots of content in the internet is -- in some for or other -- illegal under certain jurisdictions." So, IPFS is explicitly censorable; if enough IPFS nodes refuse to carry some content, then that content is effectively unavailable. Quote: "users and groups can express what content should or should not be stored and/or distributed. This is required by users to (a) comply with legal constraints in their respective countries, (b) required by users with stricter codes of conduct (i.e. content that is legal but undesired by a group -- e.g. a childrens website)." So it's possible to create walled gardens in IPFS. What does this all mean? IPFS is awesome for making content highly redundant and highly available, but it fails at being uncensorable, it fails at being oblivious, and it fails at advancing the state of security for those using it. The analogy to BitTorrent is not just facile, but also very accurate.
- woah 10y ago
  It "fails"? IPFS has never been about obliviousness, it's a distributed CDN.
- api 10y ago
  A more compelling use case than ad blocking is child porn. One of the problems with networks like Freenet is that people do not want to donate resources to replicating and hosting unknown (but known to be substantial) amounts of CP. It's a personal moral thing as well as a legal thing-- it's not something I or most other people want to in any way support. Security is another use case. It would in theory be possible for malware to be blacklisted by members of the network.
  - dublinben 10y ago
    What you call a "problem" I call the fundamental strength of a network like Freenet over IPFS. If you can impose your moral beliefs on what content ought to be available on other users of the network, then the network is useless. Controversial and unpopular speech is the exact kind of content that these networks must be designed to support. If they can't do that, then they fail their primary purpose.
    - wang_li 10y ago
      For a content host (ie the one who bought the hardware, who pays the bills and maintains the equipment) any system that puts all the cost on me and gives all the benefit to someone else is useless. See the Tragedy of the Commons and Free Rider problems. I'm sure for some people the externalization of the costs and risks are the benefit. But since most of those people won't be contributing anyway, who cares what they think.
    - Jtsummers 10y ago
      Unpopular content falls off of freenet as well (or used to, I'll admit I haven't studied it in years). If no one is requesting it it won't end up stored on the various nodes and will eventually disappear. The difference between Freenet and IPFS is that IPFS is opt-in for duplicating data, where as Freenet was not (you couldn't even opt-out, only on the total amount you would store).
akerro 10y ago
Drawback of ZeroNet is that it can host only static pages, page can't be generated dynamically. Max size for a page is 100MB. I tried to wget and convert my 5yo wordpress blog to static files, but it currently takes 1600MB, so... zeronet is no go for me, I2P and Tor work fine. Honestly I can' think of good example what zeronet should host. Maybe plaintext pages with leaks from WikiLeaks?
- marknadal 10y ago
  They have some examples of dynamic apps, like reddit style. See: https://bit.no.com:43110/Talk.ZeroNetwork.bit/ https://bit.no.com:43110/Talk.ZeroNetwork.bit/ However, how they are doing this is by having the owner's bot accept writes and perform a git merge with one big static JSON list. At least that is what I remember reading. Which doesn't seem scalable or fast. We're trying to solve that problem with https://github.com/amark/gun https://github.com/amark/gun , and hopefully will be able to team up with the ZeroNet guys at some point. I love their work.
  - kafke 10y ago
    The owner of a site is unneeded for the site to continue to work in a dynamic fashion (outside of the single 'ZeroBoard' demo that was made before the new system was in place). Interactive sites allow users permission to sign/post in a particular directory of the site (data/userpublickey/stuff.json). They sign/publish this like the owner of the site would. The site can either access the .json files directly, or compile them into an SQLite database locally and read from that. No bots are needed at all. AFAIK, git itself isn't used in ZeroNet at all, besides open-sourcing it's development. Essentially, the entire site's functionality is entirely local. You can fully use any website (except those that do weird requests to outside of zeronet) without any internet connection. Try visiting the zerotalk site, disabling your internet connection, making a post/comment, and then refreshing the page. You'll see that the comment is still there. It's just inaccessible until other users obtain the .json file you've modified. Likewise, many sites are freely cloneable, so you can try running your own local copy and acknowledge that there isn't a bot on the backend.
infinity0 10y ago
IPFS does not, and does not try to, solve the sybil attack, which has been a hard problem in securing decentralised systems for the past decade or so. So using "uncensorable" here is talking it up way too much; it will not survive attacks by nation-states. More research is needed in these security topics, and the IPFS guys don't take those seriously enough. At the very best, this is highly-resilient to random errors; malicious attackers, not so much.
- qwertyuiop924 10y ago
  However, in order to censor data, you'd have to have all the nodes connected to the target be Sybils. This would be hard. If not, I think IPFS uses gossip-style protocols to query the rest of the network if your local node links don't have the data. Correct me if I'm wrong.
qwertyuiop924 10y ago
That's not actually true. IPFS is like BitTorrent, maybe, but it's not BitTorrent.

97 points by npongratz 10y ago

Quote: "In terms of censorship resistance, IPFS includes blacklists of forbidden content in the default distribution, though it's configurable so you can turn it off 'at your own risk'. Personally I think all of this willful censorship stuff is just a phase to protect the protocol in its infancy. Blacklisting all forbidden content on a global filesystem is ridiculous. The only popular use case I see for that is blocking ads. So censorship-free distributions of IPFS implementations that integrate with Tor will probably end up as the default if this ever catches on." Unfortunately, IPFS' built-in censorship tools are not meant to be optional, nor are they temporary. https://github.com/ipfs/faq/issues/47 https://github.com/ipfs/faq/issues/47 is an FAQ entry written by an IPFS member. Quote: "IPFS has as a strict requirement that content be able to move as fast as the underlying network permits. this rules out designs like freenet's and other oblivious storage platforms, as the base case." "Oblivious storage platforms" are systems like Tahoe-LAFS or OFF, where storage nodes are unaware of the actual content that they are handling, usually due to some sort of encryption. Quote: "IPFS has as a design requirement that nodes be able to only store and/or distribute content they explicitly want to store and/or distribute. This means that computers that run IPFS nodes do not have to host 'other people's stuff', which is a very important thing when you consider that lots of content in the internet is -- in some for or other -- illegal under certain jurisdictions." So, IPFS is explicitly censorable; if enough IPFS nodes refuse to carry some content, then that content is effectively unavailable. Quote: "users and groups can express what content should or should not be stored and/or distributed. This is required by users to (a) comply with legal constraints in their respective countries, (b) required by users with stricter codes of conduct (i.e. content that is legal but undesired by a group -- e.g. a childrens website)." So it's possible to create walled gardens in IPFS. What does this all mean? IPFS is awesome for making content highly redundant and highly available, but it fails at being uncensorable, it fails at being oblivious, and it fails at advancing the state of security for those using it. The analogy to BitTorrent is not just facile, but also very accurate.

woah 10y ago
It "fails"? IPFS has never been about obliviousness, it's a distributed CDN.
api 10y ago
A more compelling use case than ad blocking is child porn. One of the problems with networks like Freenet is that people do not want to donate resources to replicating and hosting unknown (but known to be substantial) amounts of CP. It's a personal moral thing as well as a legal thing-- it's not something I or most other people want to in any way support. Security is another use case. It would in theory be possible for malware to be blacklisted by members of the network.
- dublinben 10y ago
  What you call a "problem" I call the fundamental strength of a network like Freenet over IPFS. If you can impose your moral beliefs on what content ought to be available on other users of the network, then the network is useless. Controversial and unpopular speech is the exact kind of content that these networks must be designed to support. If they can't do that, then they fail their primary purpose.
  - wang_li 10y ago
    For a content host (ie the one who bought the hardware, who pays the bills and maintains the equipment) any system that puts all the cost on me and gives all the benefit to someone else is useless. See the Tragedy of the Commons and Free Rider problems. I'm sure for some people the externalization of the costs and risks are the benefit. But since most of those people won't be contributing anyway, who cares what they think.
  - Jtsummers 10y ago
    Unpopular content falls off of freenet as well (or used to, I'll admit I haven't studied it in years). If no one is requesting it it won't end up stored on the various nodes and will eventually disappear. The difference between Freenet and IPFS is that IPFS is opt-in for duplicating data, where as Freenet was not (you couldn't even opt-out, only on the total amount you would store).

akerro 10y ago

Drawback of ZeroNet is that it can host only static pages, page can't be generated dynamically. Max size for a page is 100MB. I tried to wget and convert my 5yo wordpress blog to static files, but it currently takes 1600MB, so... zeronet is no go for me, I2P and Tor work fine. Honestly I can' think of good example what zeronet should host. Maybe plaintext pages with leaks from WikiLeaks?

marknadal 10y ago
They have some examples of dynamic apps, like reddit style. See: https://bit.no.com:43110/Talk.ZeroNetwork.bit/ https://bit.no.com:43110/Talk.ZeroNetwork.bit/ However, how they are doing this is by having the owner's bot accept writes and perform a git merge with one big static JSON list. At least that is what I remember reading. Which doesn't seem scalable or fast. We're trying to solve that problem with https://github.com/amark/gun https://github.com/amark/gun , and hopefully will be able to team up with the ZeroNet guys at some point. I love their work.
- kafke 10y ago
  The owner of a site is unneeded for the site to continue to work in a dynamic fashion (outside of the single 'ZeroBoard' demo that was made before the new system was in place). Interactive sites allow users permission to sign/post in a particular directory of the site (data/userpublickey/stuff.json). They sign/publish this like the owner of the site would. The site can either access the .json files directly, or compile them into an SQLite database locally and read from that. No bots are needed at all. AFAIK, git itself isn't used in ZeroNet at all, besides open-sourcing it's development. Essentially, the entire site's functionality is entirely local. You can fully use any website (except those that do weird requests to outside of zeronet) without any internet connection. Try visiting the zerotalk site, disabling your internet connection, making a post/comment, and then refreshing the page. You'll see that the comment is still there. It's just inaccessible until other users obtain the .json file you've modified. Likewise, many sites are freely cloneable, so you can try running your own local copy and acknowledge that there isn't a bot on the backend.

infinity0 10y ago

IPFS does not, and does not try to, solve the sybil attack, which has been a hard problem in securing decentralised systems for the past decade or so. So using "uncensorable" here is talking it up way too much; it will not survive attacks by nation-states. More research is needed in these security topics, and the IPFS guys don't take those seriously enough. At the very best, this is highly-resilient to random errors; malicious attackers, not so much.

qwertyuiop924 10y ago
However, in order to censor data, you'd have to have all the nodes connected to the target be Sybils. This would be hard. If not, I think IPFS uses gossip-style protocols to query the rest of the network if your local node links don't have the data. Correct me if I'm wrong.

qwertyuiop924 10y ago

That's not actually true. IPFS is like BitTorrent, maybe, but it's not BitTorrent.