4 ms·

Lets remind everyone: 1. IPFS attaches ALL network interfaces (internal and external) to your identity. 2. Tor is still "experimental" done by 3rd parties. ht

Lets remind everyone: 1. IPFS attaches ALL network interfaces (internal and external) to your identity. 2. Tor is still "experimental" done by 3rd parties. https://flyingzumwalt.gitbooks.io/decentralized-web-primer/content/avenues-for-access/lessons/tor-transport.html https://flyingzumwalt.gitbooks.io/decentralized-web-primer/c... 3. Due to 1 and 2, any hosted content is EASILY trackable to a user's computer, even behind NATs. A machine cryptokey also helps cement that (but can be changed). This allows easy DDoS'ing of any and all endpoints hosting content you don't like. 4. It is trivial to ask the dHT for *who* has a certain content key, and get all (or the top 50?) computers hosting that content. (this matters with regards to "sensitive" content) 5. Running a node is still high cpu, ram, and network chattiness - so using a VPS to keep IPFS off your local network is still tenuous to run.

px43 4y ago
Yeah, doing crime on IPFS is a really bad idea. I talked with Juan Benet about this in maybe 2016 or so, and he said it's a feature, which is fair enough. Privacy is very much not the goal of IPFS, and never has been. Not only is it obvious who is hosting any given file, but if you listen actively you can usually see who is downloading any given file as well.
- 7v3x3n3sem9vv 4y ago
  sounds like a potential privacy disaster of Google Facebook and others can just listen in on erroneously assumed privacy. I suppose the only reason Google/Facebook aren't pushing for it is it allows anyone to collect this data vs just the big players.
  - chriswarbo 4y ago
    > sounds like a potential privacy disaster of Google Facebook and others can just listen in on erroneously assumed privacy IPFS is an alternative to HTTP. Would you characterise Google's Web crawler as "listening in" and an "potential privacy disaster"? Are those paying for SEO "erroneously assum[ing] privacy"?
    - noasaservice 4y ago
      HTTP doesn't announce ALL the internal networks you're on. It only announces its own public IP in which it responded to.
hecturchi 4y ago
1) To be fair, private IP ranges should no longer be published to the public DHT (there is a LAN DHT for them). And of course what interfaces are announced is and has been configurable for as long as I remember ("NoAnnounce" setting in the config).
forgotmypw17 4y ago
For not "sensitive" content, but just as an alternative to other hosting methods, would you say that these shortcomings fall under "do things that don't scale"?
blamestross 4y ago
Yes on all counts. 4) Just keep watching the dht entry. It isn't "Top" it is "most recent". If you really are in a rush to know, just join the dht with that entry as your node's hash (or one close to it) and they will even advertise themselves to you.
chriswarbo 4y ago
These points also apply to HTTP, except (5). In fact, resource usage is my biggest problem with IPFS at the moment; and the reason I turned off my node :(
- hecturchi 4y ago
  It is relatively new so not many people know about it, but now libp2p (and Kubu/go-ipfs) can be configured with a Resource Manager that effectively can keep a lot of the resource usage at bay: https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgr https://github.com/ipfs/kubo/blob/master/docs/config.md#swar...
- noasaservice 4y ago
  Yeah, you really can't get away from the chattiness of IPFS because that's just how distributed hash tables work. But the CPU and RAM costs... well, those are frankly exorbitant to what amounts to a glorified Apache web server bolted on with a Bittorrent tracker.
  - pclmulqdq 4y ago
    The CPU and RAM costs are all about figuring out how to pay people (and charge them) algorithmically. I remember private trackers that had seeding and uptime requirements (enforced by people) 20 years ago.
    - chriswarbo 4y ago
      > The CPU and RAM costs are all about figuring out how to pay people (and charge them) algorithmically This sounds more like Filecoin than a normal IPFS node. I've never used Filecoin, I just host my static Web sites and git mirrors on IPFS, but the de facto Go implementation of IPFS slows down my laptop noticably and is consistently the highest consumer of RAM.
pclmulqdq 4y ago
A lot of Web 3 stuff is about federating infrastructure in ways that still allow centralization of control. IPFS is just another one of those protocols. One other thing that turned me off about IPFS/Filecoin was the huge amount of compute power needed to run a node.
- Hamuko 4y ago
  I've been under the impression that IPFS has way lighter computational requirements than Filecoin, which is basically impossible to run without a server.
  - pclmulqdq 4y ago
    Filecoin needs a server. IPFS needs a high-end laptop (that you don't take anywhere and keep 100% connected to reasonably fast internet) at least.
- hosh 4y ago
  IPFS was around before people started talking about “web3”. I see it as part of increasing resiliency and data locality. A key characteristic is that the cost for data storage is shifted to the people who care about preserving that data. So a good example is distributing Linux distro packages. Apparently, cpu/mem can be tuned.
  - px43 4y ago
    Not really. The Ethereum whitepaper was 2013, and I'm pretty sure the first Web3 stuff talking about Ethereum(execution)+Swarm(storage)+Whisper(signalling) was 2014, and IPFS was launched in 2015. IPFS worked well enough that people lost motivation to work on Swarm, and IPFS pubsub would have been cool for the signalling layer, but it seemed to fall flat on its face. Ethereum went all in on being the execution layer of Web3 and mostly delegated other components to other projects which are competing to fill in the gaps, but all generally play nice with each other. https://blog.ethereum.org/2014/08/18/building-decentralized-web https://blog.ethereum.org/2014/08/18/building-decentralized-...
rhodorhoades 4y ago
Interesting. Can you explain how that’s different than the normal internet/ hosting solutions currently available?
- chriswarbo 4y ago
  The reasons I like IPFS: - The address of a file has no relation to the machine hosting it (it's just a hash of the contents). Unlike HTTP and DNS, which rely on machine-specific IP addresses - I can host files on multiple unreliable machines (e.g. a powerful desktop, a mostly-always-on raspberry pi and a couple of frequently off/disconnected laptops). As long as at least one machine's online, the files are still available. Doing this for HTTP requires some sort of load-balancer (a single point of failure), plus mechanisms for discovery, authentication, etc. - Since IPFS uses a content hash, all copies of a file will get served from the same address; hence anyone with a copy of a file can bring its address back to life. With HTTP, people can host their own copies of files that get taken down or disappear; but their copy's address will be different from the original, and hence all hyperlinks to the original will remain broken. (Note that it's not perfect, since IPFS can perform hashing in a few ways, e.g. to better support streaming; and that results in different addresses) This is really useful for stuff like git repos, software packages, etc.
- moomoo11 4y ago
  You get way more VC washing I mean funding.
zcw100 4y ago
1. IPFS attaches ALL network interfaces (internal and external) to your identity. I don't know what you mean by attaching a network interface to your identity or even just identity. IPFS identifies a node by PeerID which is mostly a public key, that's it. 2. Tor is still "experimental" done by 3rd parties. https://flyingzumwalt.gitbooks.io/decentralized-web-primer/c https://flyingzumwalt.gitbooks.io/decentralized-web-primer/c... Tor has nothing to do with IPFS. There have been some people that have worked on some integrations to use them together. 3. Due to 1 and 2, any hosted content is EASILY trackable to a user's computer,even behind NATs. A machine cryptokey also helps cement that (but can be changed). This allows easy DDoS'ing of any and all endpoints hosting content you don't like. Yes, it is easily trackable. There is nothing about the design or goals of IPFS to be anonymous. In some ways the entire point is to be found and announce that you have content available. I don't know why people associate decentralized with anonymous. Not sure how you think it's cemented. Just toss the key and make a new one. How does this make DDoS'ing endpoints any easier than anything else hosted on the internet let alone DDoS'ing all of it. In face being distributed and content addressed can mitigate DDoS if it's malicious or just a hug of death. 4. It is trivial to ask the dHT for who has a certain content key, and get all (or the top 50?) computers hosting that content. (this matters with regards to "sensitive" content) I hope so, that's the way IPFS works. Ask for who has what you're looking for and retrieve it. I have no idea what you mean by "sensitive" content. If it's sensitive you can encrypt it. 5. Running a node is still high cpu, ram, and network chattiness - so using a VPS to keep IPFS off your local network is still tenuous to run. There are configurations to mitigate this but just making a blanket statement that it's high cpu, ram, and network is just FUD.
- dannyobrien 4y ago
  (Disclosure: I work for the Filecoin Foundation/Filecoin Foundation for the Decentralized Web). I do actually agree that the privacy and anonymity aspects of IPFS are not well- conveyed. I think people get hooked on the "censorship-resistant" nature of decentralized systems, without understanding that even if you have multiple sources, for instance in a content-addressable network like IPFS, aggressive censorship systems have other strategies to dissuade dissemination or punish readers. You always have to be thinking a few steps ahead. Services like Tor and, I hope, the IPFS network both try to convey what threat models they are useful for, and which they are not, but it's really hard to stop overenthusiastic re-statements that give them super-powers they do not, in fact, possess. That said, there's a bunch of careful thinking right now going on about how IPFS's privacy story could be improved: https://blog.ipfs.tech/ipfs-ping-2022-recap/ https://blog.ipfs.tech/ipfs-ping-2022-recap/ has a couple of sessions on this, and is a great summary of some other recent developments in the space. One of those improvements is in the point about nodes being high CPU, RAM, etc. (I actually find this to be more of a challenge when running the full IPFS Go node locally on my desktop, rather than on a VPS; it requires some tweaking.) The strategy right now is to encourage more implementations of IPFS to cover more use-cases; the original go-ipfs had to do everything, including maintaining some legacy decisions. Nowadays, there's a lot of effort on alternative IPFS implementations that can be slimmer, or optimised for particular scenarios, e.g. on an embedded device, serving a high-load web gateway, or providing millions of files. Protocol Labs recently renamed their canonical go-ipfs to kubo (https://github.com/ipfs/kubo https://github.com/ipfs/kubo ) to make it more of a peer with other implementations. Of course, I love all these new generation implementations EQUALLY, but if you pushed me, I've enjoyed playing around with https://github.com/n0-computer/iroh https://github.com/n0-computer/iroh , a modular rust implementation building off the increasingly robust rust libp2p etc libraries. There's some more to pick from here: https://docs.ipfs.tech/basics/ipfs-implementations/ https://docs.ipfs.tech/basics/ipfs-implementations/

px43 4y ago

Yeah, doing crime on IPFS is a really bad idea. I talked with Juan Benet about this in maybe 2016 or so, and he said it's a feature, which is fair enough. Privacy is very much not the goal of IPFS, and never has been. Not only is it obvious who is hosting any given file, but if you listen actively you can usually see who is downloading any given file as well.

7v3x3n3sem9vv 4y ago
sounds like a potential privacy disaster of Google Facebook and others can just listen in on erroneously assumed privacy. I suppose the only reason Google/Facebook aren't pushing for it is it allows anyone to collect this data vs just the big players.
- chriswarbo 4y ago
  > sounds like a potential privacy disaster of Google Facebook and others can just listen in on erroneously assumed privacy IPFS is an alternative to HTTP. Would you characterise Google's Web crawler as "listening in" and an "potential privacy disaster"? Are those paying for SEO "erroneously assum[ing] privacy"?
  - noasaservice 4y ago
    HTTP doesn't announce ALL the internal networks you're on. It only announces its own public IP in which it responded to.

hecturchi 4y ago

1) To be fair, private IP ranges should no longer be published to the public DHT (there is a LAN DHT for them). And of course what interfaces are announced is and has been configurable for as long as I remember ("NoAnnounce" setting in the config).

forgotmypw17 4y ago

For not "sensitive" content, but just as an alternative to other hosting methods, would you say that these shortcomings fall under "do things that don't scale"?

blamestross 4y ago

Yes on all counts. 4) Just keep watching the dht entry. It isn't "Top" it is "most recent". If you really are in a rush to know, just join the dht with that entry as your node's hash (or one close to it) and they will even advertise themselves to you.

chriswarbo 4y ago

These points also apply to HTTP, except (5). In fact, resource usage is my biggest problem with IPFS at the moment; and the reason I turned off my node :(

hecturchi 4y ago
It is relatively new so not many people know about it, but now libp2p (and Kubu/go-ipfs) can be configured with a Resource Manager that effectively can keep a lot of the resource usage at bay: https://github.com/ipfs/kubo/blob/master/docs/config.md#swarmresourcemgr https://github.com/ipfs/kubo/blob/master/docs/config.md#swar...
noasaservice 4y ago
Yeah, you really can't get away from the chattiness of IPFS because that's just how distributed hash tables work. But the CPU and RAM costs... well, those are frankly exorbitant to what amounts to a glorified Apache web server bolted on with a Bittorrent tracker.
- pclmulqdq 4y ago
  The CPU and RAM costs are all about figuring out how to pay people (and charge them) algorithmically. I remember private trackers that had seeding and uptime requirements (enforced by people) 20 years ago.
  - chriswarbo 4y ago
    > The CPU and RAM costs are all about figuring out how to pay people (and charge them) algorithmically This sounds more like Filecoin than a normal IPFS node. I've never used Filecoin, I just host my static Web sites and git mirrors on IPFS, but the de facto Go implementation of IPFS slows down my laptop noticably and is consistently the highest consumer of RAM.

pclmulqdq 4y ago

A lot of Web 3 stuff is about federating infrastructure in ways that still allow centralization of control. IPFS is just another one of those protocols. One other thing that turned me off about IPFS/Filecoin was the huge amount of compute power needed to run a node.

Hamuko 4y ago
I've been under the impression that IPFS has way lighter computational requirements than Filecoin, which is basically impossible to run without a server.
- pclmulqdq 4y ago
  Filecoin needs a server. IPFS needs a high-end laptop (that you don't take anywhere and keep 100% connected to reasonably fast internet) at least.
hosh 4y ago
IPFS was around before people started talking about “web3”. I see it as part of increasing resiliency and data locality. A key characteristic is that the cost for data storage is shifted to the people who care about preserving that data. So a good example is distributing Linux distro packages. Apparently, cpu/mem can be tuned.
- px43 4y ago
  Not really. The Ethereum whitepaper was 2013, and I'm pretty sure the first Web3 stuff talking about Ethereum(execution)+Swarm(storage)+Whisper(signalling) was 2014, and IPFS was launched in 2015. IPFS worked well enough that people lost motivation to work on Swarm, and IPFS pubsub would have been cool for the signalling layer, but it seemed to fall flat on its face. Ethereum went all in on being the execution layer of Web3 and mostly delegated other components to other projects which are competing to fill in the gaps, but all generally play nice with each other. https://blog.ethereum.org/2014/08/18/building-decentralized-web https://blog.ethereum.org/2014/08/18/building-decentralized-...

rhodorhoades 4y ago

Interesting. Can you explain how that’s different than the normal internet/ hosting solutions currently available?

chriswarbo 4y ago
The reasons I like IPFS: - The address of a file has no relation to the machine hosting it (it's just a hash of the contents). Unlike HTTP and DNS, which rely on machine-specific IP addresses - I can host files on multiple unreliable machines (e.g. a powerful desktop, a mostly-always-on raspberry pi and a couple of frequently off/disconnected laptops). As long as at least one machine's online, the files are still available. Doing this for HTTP requires some sort of load-balancer (a single point of failure), plus mechanisms for discovery, authentication, etc. - Since IPFS uses a content hash, all copies of a file will get served from the same address; hence anyone with a copy of a file can bring its address back to life. With HTTP, people can host their own copies of files that get taken down or disappear; but their copy's address will be different from the original, and hence all hyperlinks to the original will remain broken. (Note that it's not perfect, since IPFS can perform hashing in a few ways, e.g. to better support streaming; and that results in different addresses) This is really useful for stuff like git repos, software packages, etc.
moomoo11 4y ago
You get way more VC washing I mean funding.

zcw100 4y ago

1. IPFS attaches ALL network interfaces (internal and external) to your identity. I don't know what you mean by attaching a network interface to your identity or even just identity. IPFS identifies a node by PeerID which is mostly a public key, that's it. 2. Tor is still "experimental" done by 3rd parties. https://flyingzumwalt.gitbooks.io/decentralized-web-primer/c https://flyingzumwalt.gitbooks.io/decentralized-web-primer/c... Tor has nothing to do with IPFS. There have been some people that have worked on some integrations to use them together. 3. Due to 1 and 2, any hosted content is EASILY trackable to a user's computer,even behind NATs. A machine cryptokey also helps cement that (but can be changed). This allows easy DDoS'ing of any and all endpoints hosting content you don't like. Yes, it is easily trackable. There is nothing about the design or goals of IPFS to be anonymous. In some ways the entire point is to be found and announce that you have content available. I don't know why people associate decentralized with anonymous. Not sure how you think it's cemented. Just toss the key and make a new one. How does this make DDoS'ing endpoints any easier than anything else hosted on the internet let alone DDoS'ing all of it. In face being distributed and content addressed can mitigate DDoS if it's malicious or just a hug of death. 4. It is trivial to ask the dHT for who has a certain content key, and get all (or the top 50?) computers hosting that content. (this matters with regards to "sensitive" content) I hope so, that's the way IPFS works. Ask for who has what you're looking for and retrieve it. I have no idea what you mean by "sensitive" content. If it's sensitive you can encrypt it. 5. Running a node is still high cpu, ram, and network chattiness - so using a VPS to keep IPFS off your local network is still tenuous to run. There are configurations to mitigate this but just making a blanket statement that it's high cpu, ram, and network is just FUD.

dannyobrien 4y ago
(Disclosure: I work for the Filecoin Foundation/Filecoin Foundation for the Decentralized Web). I do actually agree that the privacy and anonymity aspects of IPFS are not well- conveyed. I think people get hooked on the "censorship-resistant" nature of decentralized systems, without understanding that even if you have multiple sources, for instance in a content-addressable network like IPFS, aggressive censorship systems have other strategies to dissuade dissemination or punish readers. You always have to be thinking a few steps ahead. Services like Tor and, I hope, the IPFS network both try to convey what threat models they are useful for, and which they are not, but it's really hard to stop overenthusiastic re-statements that give them super-powers they do not, in fact, possess. That said, there's a bunch of careful thinking right now going on about how IPFS's privacy story could be improved: https://blog.ipfs.tech/ipfs-ping-2022-recap/ https://blog.ipfs.tech/ipfs-ping-2022-recap/ has a couple of sessions on this, and is a great summary of some other recent developments in the space. One of those improvements is in the point about nodes being high CPU, RAM, etc. (I actually find this to be more of a challenge when running the full IPFS Go node locally on my desktop, rather than on a VPS; it requires some tweaking.) The strategy right now is to encourage more implementations of IPFS to cover more use-cases; the original go-ipfs had to do everything, including maintaining some legacy decisions. Nowadays, there's a lot of effort on alternative IPFS implementations that can be slimmer, or optimised for particular scenarios, e.g. on an embedded device, serving a high-load web gateway, or providing millions of files. Protocol Labs recently renamed their canonical go-ipfs to kubo (https://github.com/ipfs/kubo https://github.com/ipfs/kubo ) to make it more of a peer with other implementations. Of course, I love all these new generation implementations EQUALLY, but if you pushed me, I've enjoyed playing around with https://github.com/n0-computer/iroh https://github.com/n0-computer/iroh , a modular rust implementation building off the increasingly robust rust libp2p etc libraries. There's some more to pick from here: https://docs.ipfs.tech/basics/ipfs-implementations/ https://docs.ipfs.tech/basics/ipfs-implementations/