14 ms·

Freenet build 1495: new user experience and performance

140 points by ArneBab 4y ago

EGreg 4y ago
Last year, I interviewed Ian Clarke, who started arguably the first decentralized file sharing system: freenet! https://www.youtube.com/watch?v=JWrRqUkJpMQ https://www.youtube.com/watch?v=JWrRqUkJpMQ The interview -- like most of my interviews -- goes into depth about the entire space, and also touches on Locutus, his new project. PS: I like to interview people on topics related to what we build at Qbix and Intercoin. Freedom of Speech and Capitalism (with Noam Chomsky) https://www.youtube.com/watch?v=gv5mI6ClPGc https://www.youtube.com/watch?v=gv5mI6ClPGc Identity and Civic Engagement (with David Boulet) https://www.youtube.com/watch?v=GRzVj2W9WGM https://www.youtube.com/watch?v=GRzVj2W9WGM Free Cities (Patri Friedman, grandson of Milton Friedman) https://www.youtube.com/watch?v=Lgil1M9tAXU https://www.youtube.com/watch?v=Lgil1M9tAXU Community Economics (Thomas Greco) https://www.youtube.com/watch?v=OXTn52kL0Yo https://www.youtube.com/watch?v=OXTn52kL0Yo Crypto and Securities Laws (Sara Hanks) https://www.youtube.com/watch?v=ocrqgkJn4m0 https://www.youtube.com/watch?v=ocrqgkJn4m0
- Diederich 4y ago
  Really cool stuff, thank you for this!
- ArneBab 4y ago
  Freenet has a different concept than MaidSAFE: While MaidSAFE (as it was when I last looked into it — a few years ago) is a hosting and application platform, the focus of Freenet is to be the technical component for censorship resistance. Everything it provides comes from this — even the recently added video and Android support (because Black Lives Matter showed clearly that the information that people need to share the most is video content nowadays). Therefore the overarching design goal of freenet is to make censorship as hard as technically possible. That’s the reason for - providing anonymity (else you could be threatened with repercussions - as seen in the case of the wikileaks informer from the army in the USA), - building it as free software network (else you could just shut down the central website, as people tried with wikileaks), - providing safe pseudonyms and caching of the content on all participating nodes (else people could censor by spamming or overloading nodes), - including good moderation tools (otherwise people could censor by just "flooding the zone with shit"), and even - the darknet mode and enhancements in usability (else freenet could be stopped by just prosecuting everyone who uses it, or it would reach too few people to be able to counter censorship in the clearnet). One core difference between MaidSAFE is that Freenet provides communication. Content in Freenet stays accessible while people access it, without tracking accesses: When you upload something, you can imagine that a clock starts ticking. After a while (depending on file size) the content is no longer accessible. But if someone accesses it, their node "heals" that content, which resets that clock. Also Freenet enables people to connect purely to their friends, so there’s no central point of failure or detection. I once did a comparison to other projects for a funding proposal (that did not succeed, though) which should still give you an idea about the difference between Freenet and other systems: http://www.draketo.de/proj/freenet-funding/#sec-9 http://www.draketo.de/proj/freenet-funding/#sec-9
- notacoward 4y ago
  Having been in that space at the time, I think this point is not arguable at all. Freenet was not the first decentralized file sharing system. Freenet might have been the subject of a PhD thesis in 1999, but by the time it became even slightly usable Napster and Gnutella already had substantial user bases. (Similar to Ceph BTW which had a long gap between being a PhD thesis and being an actual usable distributed filesystem, during which time others had existed in the wild for some time.) The "arguable" part is whether you could include eDonkey, GNUnet, and others in that list. What is true AFAIK is that Freenet was the first such system with things like censorship resistance as a primary design goal (though the MojoNation folks might even disagree with that) and a non-trivial user base. In case anyone doesn't believe me, here's the Freenet thesis which actually mentions both Gnutella and Napster in references. Hard to claim precedence when that's the case. https://www.cs.cornell.edu/people/egs/615/freenet.pdf https://www.cs.cornell.edu/people/egs/615/freenet.pdf
  - s4n1ty 4y ago
    [dead]
  - sanity31415 4y ago
    Napster wasn't decentralized, it relied on a centralized index which is why it was shut down. Gnutella relied on a broadcast search which didn't scale. Freenet's use of small world routing to be both decentralized and scalable was unique, as was it's use of cryptographic contracts (signed subspace keys). That paper isn't the original Freenet paper, it came later. This is the original: https://www.semanticscholar.org/paper/A-Distributed-Decentralised-Information-Storage-and-Clarke/f3ec2adb51b823e4ba7f8c86315c17c5547f88cc https://www.semanticscholar.org/paper/A-Distributed-Decentra...
    - ArneBab 4y ago
      Gnutella moved from broadcast to dynamically structured search early on (in version 0.6, 2002). See https://rfc-gnutella.sourceforge.net/src/rfc-0_6-draft.html https://rfc-gnutella.sourceforge.net/src/rfc-0_6-draft.html Including decentralized download via HTTP using just three added headers and Tiger Tree Hash Exchange: https://rfc-gnutella.sourceforge.net/src/Partial_File_Sharing_Protocol_1.0.txt https://rfc-gnutella.sourceforge.net/src/Partial_File_Sharin... Scalability was limited to global search in around 5 million nodes (with the network spanning 40 million people at its peak), though, due to restrictions in the number of TCP connections Windows (private edition) supported.
- heavyset_go 4y ago
  Great content, by the way.
acapybara 4y ago
These are some of the most underrated devs on the internet.
- deleted 4y ago
  [deleted]
- cutler 4y ago
  Definitely. Ian Clarke's Freenet has been around since March 2000 which is when I discovered Linux and Perl. The only thing I don't understand is why it was written in Java instead of C/C++. Napster was also built with Java, however, so even slow as Java was back then it was still fast enough for building file-sharing networks.
  - dale_glass 4y ago
    Java was a much better choice security-wise back in 2000. But I think it crippled adoption, it required far too much RAM for the computers of the time.
sandworm101 4y ago
Freenet is well designed and well maintained, but it is a solution looking for a problem. We dont yet life in the police state where everyone's political opinions need to be hidden in dark networks. At the moment, freenet has a bad reputation for hosting very illegal content. That rep is so bad that if authorities ever look into your internet traffic, freenet connections are more than enough to get warrants that will rip your life apart. Imho, running freenet is like commuting to work on a sportbike. It is perfectly legal (i used to do it) but dont be suprised if the cops pay closer attention to you. Except those "random" license checks by the roadside arent nearly as embarrassing as them inspecting your computers.
- ArneBab 4y ago
  Just having a Freenet connection does not get a warrant — though there are researchers who use some fishy statistics which may be reason not to run a very fast node in the US (because running a fast node provably causes false positives in their statistics). For some more details, see https://freenetproject.org/the-discredited-levine-2017-approach-is-still-used.html https://freenetproject.org/the-discredited-levine-2017-appro... and the links therein.
  - sandworm101 4y ago
    I think you might underestimate how easy it is to get a search warrant, paticularly in the US. This isnt the standard for arrest, let alone conviction. A cop saying "the machine regularly connected to a suspect network" is more than enough for many judges/magistrates... most of which dont know an IP from an IPA. (Lol, got over v underestimate mixed up.)
    - calvinmorrison 4y ago
      Kyle Rittenhouse trial, the lawyer pinch zooming on an photo gallery on an iPhone and arguing its legit is all you need to know
      - boppo1 4y ago
cdiwadkar 4y ago
Sorry I’m not tech savvy! How is this different from Tor?
- colechristensen 4y ago
  Freenet is slightly older and doesn’t route internet traffic, just distributed static sites.
  - mekster 4y ago
    Thank you for solving my 20 years mystery.
- nulbyte 4y ago
  In basic terms, Tor is used to browse the Internet, while Freenet is used to browse Freenet. You can browse the Internet without Tor, but you cannot browse Freenet without Freenet.
- ArneBab 4y ago
  Tor gives you an anonymous connection to some server. Stuff that’s not linked to is still available to the server admin. Freenet provides an anonymous, encrypted, and distributed cache that does not rely on the security of any one server. Stuff that’s not linked to could just as well not exist, because no one can access it.
- 0xcde4c3db 4y ago
  Like Tor, Freenet attempts to provide anonymity by obscuring the ultimate origin of any given request Unlike Tor, Freenet can't be directly used like a VPN. Unlike Tor, Freenet nodes don't route to specific origin servers, but also provide storage to the network. Data is "inserted" to announce its presence, and then cached along the path of any requests. This is supposed to both provide robust access to popular data and provide plausible deniability against any given node operator being responsible for the data (the extent to which it succeeds at either aim in practice is a subject of quite some discussion).
- raybb 4y ago
jKwZmFpLrNhG 4y ago
While Freenet is a technically well-designed and maintained network, it has a reputation for hosting illegal content and being associated with warrants that can cause significant disruptions in users' lives. Using Freenet may attract more attention from authorities and may not be necessary at the present time, as the current political climate does not necessitate the use of such networks to hide political opinions.
- verdenti 4y ago
  [dead]
- ritonlajoie 4y ago
  where do you live ?
- Taek 4y ago
  This reads like a ChatGPT censor trying to assure me that I don't need to know how to make a Molotov cocktail.
throwaway0x7E6 4y ago
lmao at two nearly identical glowie comments
davidwritesbugs 4y ago
I dabbled with Freenet when it first came out in '99/2000 is but I found the performance just awful - think I once waited 30 mins for a 100k html file. And each year after they said "we solved that, works great now" but it didn't. I'd be curious to know what it's really like now. Well, I would, except that there was so much CP on Freenet I don't think I'd dare install it because of the risk of taint. Is there much legit anon material?
- ArneBab 4y ago
  There is a lot of personal opinion nowadays — along with forums and microblogging and people discussing philosophical arguments in IRC-over-Freenet (FLIP). You’ll wait up to 30s¹ for a 100k html file nowadays. 1 minute for a 2 MiB website. ¹: if you have at least 10 peers. If you try to run with the lowest possible specs (5 peers), you won’t get a good experience, but with 10 peers it works pretty well (and the Japanese with their awesome connection speeds actually get 60 peers). Also see https://news.ycombinator.com/item?id=34180729 https://news.ycombinator.com/item?id=34180729
ilaksh 4y ago
Also check out Locutus https://github.com/freenet/locutus https://github.com/freenet/locutus from the same inventor as Freenet
m4lmmfk4 4y ago
[dead]
rolenthedeep 4y ago
I really want to like Freenet. The idea of a serverless network with resources distributed amongst the nodes is such an elegant solution. It makes even more sense today with the easy access to fiber or high-speed cable and cheap storage. It's everything I wish the internet as a whole could be. But when I've used it the experience was absolutely dreadful. It's difficult to find anything, and there's a lot of emphasis on trusted peers, but I don't know anyone who has even heard of this, and it's unclear how you'd find nodes to trust. Also it's horribly slow and the proxy server eats a ton of resources. I think it's a beautiful concept with massive potential, but I'm extremely skeptical it will ever find widespread use
- Karrot_Kream 4y ago
  The author's Locutus project seems to be a more relevant to today's web needs.

140 points by ArneBab 4y ago

Last year, I interviewed Ian Clarke, who started arguably the first decentralized file sharing system: freenet! https://www.youtube.com/watch?v=JWrRqUkJpMQ https://www.youtube.com/watch?v=JWrRqUkJpMQ The interview -- like most of my interviews -- goes into depth about the entire space, and also touches on Locutus, his new project. PS: I like to interview people on topics related to what we build at Qbix and Intercoin. Freedom of Speech and Capitalism (with Noam Chomsky) https://www.youtube.com/watch?v=gv5mI6ClPGc https://www.youtube.com/watch?v=gv5mI6ClPGc Identity and Civic Engagement (with David Boulet) https://www.youtube.com/watch?v=GRzVj2W9WGM https://www.youtube.com/watch?v=GRzVj2W9WGM Free Cities (Patri Friedman, grandson of Milton Friedman) https://www.youtube.com/watch?v=Lgil1M9tAXU https://www.youtube.com/watch?v=Lgil1M9tAXU Community Economics (Thomas Greco) https://www.youtube.com/watch?v=OXTn52kL0Yo https://www.youtube.com/watch?v=OXTn52kL0Yo Crypto and Securities Laws (Sara Hanks) https://www.youtube.com/watch?v=ocrqgkJn4m0 https://www.youtube.com/watch?v=ocrqgkJn4m0

Diederich 4y ago
Really cool stuff, thank you for this!
ArneBab 4y ago
Freenet has a different concept than MaidSAFE: While MaidSAFE (as it was when I last looked into it — a few years ago) is a hosting and application platform, the focus of Freenet is to be the technical component for censorship resistance. Everything it provides comes from this — even the recently added video and Android support (because Black Lives Matter showed clearly that the information that people need to share the most is video content nowadays). Therefore the overarching design goal of freenet is to make censorship as hard as technically possible. That’s the reason for - providing anonymity (else you could be threatened with repercussions - as seen in the case of the wikileaks informer from the army in the USA), - building it as free software network (else you could just shut down the central website, as people tried with wikileaks), - providing safe pseudonyms and caching of the content on all participating nodes (else people could censor by spamming or overloading nodes), - including good moderation tools (otherwise people could censor by just "flooding the zone with shit"), and even - the darknet mode and enhancements in usability (else freenet could be stopped by just prosecuting everyone who uses it, or it would reach too few people to be able to counter censorship in the clearnet). One core difference between MaidSAFE is that Freenet provides communication. Content in Freenet stays accessible while people access it, without tracking accesses: When you upload something, you can imagine that a clock starts ticking. After a while (depending on file size) the content is no longer accessible. But if someone accesses it, their node "heals" that content, which resets that clock. Also Freenet enables people to connect purely to their friends, so there’s no central point of failure or detection. I once did a comparison to other projects for a funding proposal (that did not succeed, though) which should still give you an idea about the difference between Freenet and other systems: http://www.draketo.de/proj/freenet-funding/#sec-9 http://www.draketo.de/proj/freenet-funding/#sec-9
notacoward 4y ago
Having been in that space at the time, I think this point is not arguable at all. Freenet was not the first decentralized file sharing system. Freenet might have been the subject of a PhD thesis in 1999, but by the time it became even slightly usable Napster and Gnutella already had substantial user bases. (Similar to Ceph BTW which had a long gap between being a PhD thesis and being an actual usable distributed filesystem, during which time others had existed in the wild for some time.) The "arguable" part is whether you could include eDonkey, GNUnet, and others in that list. What is true AFAIK is that Freenet was the first such system with things like censorship resistance as a primary design goal (though the MojoNation folks might even disagree with that) and a non-trivial user base. In case anyone doesn't believe me, here's the Freenet thesis which actually mentions both Gnutella and Napster in references. Hard to claim precedence when that's the case. https://www.cs.cornell.edu/people/egs/615/freenet.pdf https://www.cs.cornell.edu/people/egs/615/freenet.pdf
- s4n1ty 4y ago
  [dead]
- sanity31415 4y ago
  Napster wasn't decentralized, it relied on a centralized index which is why it was shut down. Gnutella relied on a broadcast search which didn't scale. Freenet's use of small world routing to be both decentralized and scalable was unique, as was it's use of cryptographic contracts (signed subspace keys). That paper isn't the original Freenet paper, it came later. This is the original: https://www.semanticscholar.org/paper/A-Distributed-Decentralised-Information-Storage-and-Clarke/f3ec2adb51b823e4ba7f8c86315c17c5547f88cc https://www.semanticscholar.org/paper/A-Distributed-Decentra...
  - ArneBab 4y ago
    Gnutella moved from broadcast to dynamically structured search early on (in version 0.6, 2002). See https://rfc-gnutella.sourceforge.net/src/rfc-0_6-draft.html https://rfc-gnutella.sourceforge.net/src/rfc-0_6-draft.html Including decentralized download via HTTP using just three added headers and Tiger Tree Hash Exchange: https://rfc-gnutella.sourceforge.net/src/Partial_File_Sharing_Protocol_1.0.txt https://rfc-gnutella.sourceforge.net/src/Partial_File_Sharin... Scalability was limited to global search in around 5 million nodes (with the network spanning 40 million people at its peak), though, due to restrictions in the number of TCP connections Windows (private edition) supported.
heavyset_go 4y ago
Great content, by the way.

acapybara 4y ago

These are some of the most underrated devs on the internet.

deleted 4y ago
[deleted]
cutler 4y ago
Definitely. Ian Clarke's Freenet has been around since March 2000 which is when I discovered Linux and Perl. The only thing I don't understand is why it was written in Java instead of C/C++. Napster was also built with Java, however, so even slow as Java was back then it was still fast enough for building file-sharing networks.
- dale_glass 4y ago
  Java was a much better choice security-wise back in 2000. But I think it crippled adoption, it required far too much RAM for the computers of the time.

sandworm101 4y ago

Freenet is well designed and well maintained, but it is a solution looking for a problem. We dont yet life in the police state where everyone's political opinions need to be hidden in dark networks. At the moment, freenet has a bad reputation for hosting very illegal content. That rep is so bad that if authorities ever look into your internet traffic, freenet connections are more than enough to get warrants that will rip your life apart. Imho, running freenet is like commuting to work on a sportbike. It is perfectly legal (i used to do it) but dont be suprised if the cops pay closer attention to you. Except those "random" license checks by the roadside arent nearly as embarrassing as them inspecting your computers.

ArneBab 4y ago
Just having a Freenet connection does not get a warrant — though there are researchers who use some fishy statistics which may be reason not to run a very fast node in the US (because running a fast node provably causes false positives in their statistics). For some more details, see https://freenetproject.org/the-discredited-levine-2017-approach-is-still-used.html https://freenetproject.org/the-discredited-levine-2017-appro... and the links therein.
- sandworm101 4y ago
  I think you might underestimate how easy it is to get a search warrant, paticularly in the US. This isnt the standard for arrest, let alone conviction. A cop saying "the machine regularly connected to a suspect network" is more than enough for many judges/magistrates... most of which dont know an IP from an IPA. (Lol, got over v underestimate mixed up.)
  - calvinmorrison 4y ago
    Kyle Rittenhouse trial, the lawyer pinch zooming on an photo gallery on an iPhone and arguing its legit is all you need to know
    - boppo1 4y ago

cdiwadkar 4y ago

Sorry I’m not tech savvy! How is this different from Tor?

colechristensen 4y ago
Freenet is slightly older and doesn’t route internet traffic, just distributed static sites.
- mekster 4y ago
  Thank you for solving my 20 years mystery.
nulbyte 4y ago
In basic terms, Tor is used to browse the Internet, while Freenet is used to browse Freenet. You can browse the Internet without Tor, but you cannot browse Freenet without Freenet.
ArneBab 4y ago
Tor gives you an anonymous connection to some server. Stuff that’s not linked to is still available to the server admin. Freenet provides an anonymous, encrypted, and distributed cache that does not rely on the security of any one server. Stuff that’s not linked to could just as well not exist, because no one can access it.
0xcde4c3db 4y ago
Like Tor, Freenet attempts to provide anonymity by obscuring the ultimate origin of any given request Unlike Tor, Freenet can't be directly used like a VPN. Unlike Tor, Freenet nodes don't route to specific origin servers, but also provide storage to the network. Data is "inserted" to announce its presence, and then cached along the path of any requests. This is supposed to both provide robust access to popular data and provide plausible deniability against any given node operator being responsible for the data (the extent to which it succeeds at either aim in practice is a subject of quite some discussion).
raybb 4y ago

jKwZmFpLrNhG 4y ago

While Freenet is a technically well-designed and maintained network, it has a reputation for hosting illegal content and being associated with warrants that can cause significant disruptions in users' lives. Using Freenet may attract more attention from authorities and may not be necessary at the present time, as the current political climate does not necessitate the use of such networks to hide political opinions.

verdenti 4y ago
[dead]
ritonlajoie 4y ago
where do you live ?
Taek 4y ago
This reads like a ChatGPT censor trying to assure me that I don't need to know how to make a Molotov cocktail.

throwaway0x7E6 4y ago

lmao at two nearly identical glowie comments

davidwritesbugs 4y ago

I dabbled with Freenet when it first came out in '99/2000 is but I found the performance just awful - think I once waited 30 mins for a 100k html file. And each year after they said "we solved that, works great now" but it didn't. I'd be curious to know what it's really like now. Well, I would, except that there was so much CP on Freenet I don't think I'd dare install it because of the risk of taint. Is there much legit anon material?

ArneBab 4y ago
There is a lot of personal opinion nowadays — along with forums and microblogging and people discussing philosophical arguments in IRC-over-Freenet (FLIP). You’ll wait up to 30s¹ for a 100k html file nowadays. 1 minute for a 2 MiB website. ¹: if you have at least 10 peers. If you try to run with the lowest possible specs (5 peers), you won’t get a good experience, but with 10 peers it works pretty well (and the Japanese with their awesome connection speeds actually get 60 peers). Also see https://news.ycombinator.com/item?id=34180729 https://news.ycombinator.com/item?id=34180729

ilaksh 4y ago

Also check out Locutus https://github.com/freenet/locutus https://github.com/freenet/locutus from the same inventor as Freenet

m4lmmfk4 4y ago

[dead]

rolenthedeep 4y ago

I really want to like Freenet. The idea of a serverless network with resources distributed amongst the nodes is such an elegant solution. It makes even more sense today with the easy access to fiber or high-speed cable and cheap storage. It's everything I wish the internet as a whole could be. But when I've used it the experience was absolutely dreadful. It's difficult to find anything, and there's a lot of emphasis on trusted peers, but I don't know anyone who has even heard of this, and it's unclear how you'd find nodes to trust. Also it's horribly slow and the proxy server eats a ton of resources. I think it's a beautiful concept with massive potential, but I'm extremely skeptical it will ever find widespread use

Karrot_Kream 4y ago
The author's Locutus project seems to be a more relevant to today's web needs.