5 ms·

IPFS is just too slow for it to be usable on a mass scale. It's a neat idea but unfortunately, p2p file storage is tough, you absolutely need a central model to

by purim 4y ago

IPFS is just too slow for it to be usable on a mass scale. It's a neat idea but unfortunately, p2p file storage is tough, you absolutely need a central model to scale up. Offering coins to cash out at casinos where the other side of the order book are people with unlimited supply of it. Doesn't work.

yamtaddle 4y ago
The fundamental problem with any of these p2p-hosting solutions is that most devices folks use these days are battery-powered. Any solution that requires a persistent connection or intermittent network wake-ups to achieve even semi-decent performance won't make it, and if you overcome that hurdle you still won't be able to count on most of the devices downloading content to also serve content. No-one's gonna sacrifice battery life to make the distributed Web dream happen. They seem a lot more useful for internal infra of hosting providers, though for IPFS in particular I expect the performance isn't consistent enough to be a suitable solution for most of those use cases.
- tinalumfoil 4y ago
  Having used ipfs casually the big draw for me has been the gateways. Running a full ipfs node on client devices is mostly not practical, but I can choose a trusted (or not so trusted since I can check the hash) gateway and that can handle the heavy lifting for me. And even though the ipfs network is slow, the distributed nature lends itself to very heavy caching, so the gateway doesn't have to be any slower than if it was serving me its own static files. Plus the gateways provide compatibility to www. Given, things are far from ideal now. But looking over the water it's good to see Mastodon taking off and I think that's largely because you have the option of just choosing a single trusted provider ("instances") from which you can access the rest of the network. The trusted provider does the heavy lifting for you.
- zozbot234 4y ago
  IPFS is built to work reasonably well with ephemeral connections. Sure, not everyone is interested in serving content but for those who are it seems to be a reasonable choice.
hinkley 4y ago
When I worked for a National Supercomputing Center, I discovered that they had a distributed filesystem called Andrew Filesystem (AFS) which they used for archival purposes. With every new generation of distributed filesystem I always wondered why the older ones failed. It wasn't until maybe 10 years ago that I finally got my answer: It turns out that Amdahl's Law kills AFS. There's a total throughput wall that becomes very painfully visible once you move to gigabit networking, and any one client can pretty much saturate the network.
- pjmlp 4y ago
  That is also what CERN was using in early 2000 before the Grid efforts.
rakoo 4y ago
Bittorrent has clearly shown that p2p can perfectly scale with very little centralization, that is now becoming unneeded. There is an issue with IPFS, not with p2p. My uneducated guess: content in IPFS is split in too small blocks, making the data-to-control ratio way too low.
jhunter1016 4y ago
Disclaimer, I am head of product for Pinata I think of IPFS as being an open data platform first. You can connect to it and disconnect as frequently as you like. The underlying p2p capabilities don't have to necessarily be fast. It just has to do the job it was designed to do—get content in a permissionless fashion. Speed, convenience, reliability, and more are not the problems protocol need to solve for. Providers can solve for these problems without infringing on your ability to "take your ball and go home." Take Pinata for example. We provide dedicated IPFS gateways that provide essentially the same experience you would expect from traditional cloud providers. But if you ever want to leave Pinata or back your data up or just inspect your data, you don't need Pinata's permission. IPFS media is public and open. Convenience is a layer on top of that. IPFS also doesn't need or have tokens. Filecoin is a separate entity. IPFS is especially powerful because it is not linked to a specific blockchain or currency.
- dmitriid 4y ago
  > Speed, convenience, reliability, and more are not the problems protocol need to solve for. Providers can solve for these problems They can't if the protocol doesn't allow it. Otherwise it will be regular centralized storage providers with IPFS bolted on top for one or two geeks who care about it.
  - zeroclip 4y ago
    That’s kinda the whole point. A couple of centralized gateways can be the main hosts, like Protocol Labs and Cloudflare, and provide the best performance, and also compete with each other on the same content hash. If one or both of them goes down, any user in the world can re-host the data by pinning it, if they still happen to have the file on disk or if they’ve backed it up.
    - deleted 4y ago
      [deleted]
    - dmitriid 4y ago
      I don't understand the point, really. Especially not from the user's point of view. IPFS doesn't add anything in n this equation except additional complexity and "well, if this already centralized service goes down, you're still screwed because the chances of someone storing your files are asymptotically approaching zero"
xani_ 4y ago
You don't need central central, but the more you distribute the more you lose on performance. It could be interesting idea on smaller scale, say you start a "virtual hosting provider", where each of 10, 100, 1000 people connects into a mesh and store eachother's data so in event of failure it just keeps working.
VonGuard 4y ago
I am told this is very specifically a default setting in the app that makes the minimum connections something like 100 or so. If you crank that down to 5 to 20 it goes much faster. YMMV.
matrix_overload 4y ago
Yeah, but Filecoin raised $250M two years ago. They have enough capital to keep writing these articles until the thermodynamic death of the Universe.
- simonebrunozzi 4y ago
  In 2017, not in 2020.

by purim 4y ago

yamtaddle 4y ago

The fundamental problem with any of these p2p-hosting solutions is that most devices folks use these days are battery-powered. Any solution that requires a persistent connection or intermittent network wake-ups to achieve even semi-decent performance won't make it, and if you overcome that hurdle you still won't be able to count on most of the devices downloading content to also serve content. No-one's gonna sacrifice battery life to make the distributed Web dream happen. They seem a lot more useful for internal infra of hosting providers, though for IPFS in particular I expect the performance isn't consistent enough to be a suitable solution for most of those use cases.

tinalumfoil 4y ago
Having used ipfs casually the big draw for me has been the gateways. Running a full ipfs node on client devices is mostly not practical, but I can choose a trusted (or not so trusted since I can check the hash) gateway and that can handle the heavy lifting for me. And even though the ipfs network is slow, the distributed nature lends itself to very heavy caching, so the gateway doesn't have to be any slower than if it was serving me its own static files. Plus the gateways provide compatibility to www. Given, things are far from ideal now. But looking over the water it's good to see Mastodon taking off and I think that's largely because you have the option of just choosing a single trusted provider ("instances") from which you can access the rest of the network. The trusted provider does the heavy lifting for you.
zozbot234 4y ago
IPFS is built to work reasonably well with ephemeral connections. Sure, not everyone is interested in serving content but for those who are it seems to be a reasonable choice.

hinkley 4y ago

When I worked for a National Supercomputing Center, I discovered that they had a distributed filesystem called Andrew Filesystem (AFS) which they used for archival purposes. With every new generation of distributed filesystem I always wondered why the older ones failed. It wasn't until maybe 10 years ago that I finally got my answer: It turns out that Amdahl's Law kills AFS. There's a total throughput wall that becomes very painfully visible once you move to gigabit networking, and any one client can pretty much saturate the network.

pjmlp 4y ago
That is also what CERN was using in early 2000 before the Grid efforts.

rakoo 4y ago

Bittorrent has clearly shown that p2p can perfectly scale with very little centralization, that is now becoming unneeded. There is an issue with IPFS, not with p2p. My uneducated guess: content in IPFS is split in too small blocks, making the data-to-control ratio way too low.

jhunter1016 4y ago

Disclaimer, I am head of product for Pinata I think of IPFS as being an open data platform first. You can connect to it and disconnect as frequently as you like. The underlying p2p capabilities don't have to necessarily be fast. It just has to do the job it was designed to do—get content in a permissionless fashion. Speed, convenience, reliability, and more are not the problems protocol need to solve for. Providers can solve for these problems without infringing on your ability to "take your ball and go home." Take Pinata for example. We provide dedicated IPFS gateways that provide essentially the same experience you would expect from traditional cloud providers. But if you ever want to leave Pinata or back your data up or just inspect your data, you don't need Pinata's permission. IPFS media is public and open. Convenience is a layer on top of that. IPFS also doesn't need or have tokens. Filecoin is a separate entity. IPFS is especially powerful because it is not linked to a specific blockchain or currency.

dmitriid 4y ago
> Speed, convenience, reliability, and more are not the problems protocol need to solve for. Providers can solve for these problems They can't if the protocol doesn't allow it. Otherwise it will be regular centralized storage providers with IPFS bolted on top for one or two geeks who care about it.
- zeroclip 4y ago
  That’s kinda the whole point. A couple of centralized gateways can be the main hosts, like Protocol Labs and Cloudflare, and provide the best performance, and also compete with each other on the same content hash. If one or both of them goes down, any user in the world can re-host the data by pinning it, if they still happen to have the file on disk or if they’ve backed it up.
  - deleted 4y ago
    [deleted]
  - dmitriid 4y ago
    I don't understand the point, really. Especially not from the user's point of view. IPFS doesn't add anything in n this equation except additional complexity and "well, if this already centralized service goes down, you're still screwed because the chances of someone storing your files are asymptotically approaching zero"

xani_ 4y ago

You don't need central central, but the more you distribute the more you lose on performance. It could be interesting idea on smaller scale, say you start a "virtual hosting provider", where each of 10, 100, 1000 people connects into a mesh and store eachother's data so in event of failure it just keeps working.

VonGuard 4y ago

I am told this is very specifically a default setting in the app that makes the minimum connections something like 100 or so. If you crank that down to 5 to 20 it goes much faster. YMMV.

matrix_overload 4y ago

Yeah, but Filecoin raised $250M two years ago. They have enough capital to keep writing these articles until the thermodynamic death of the Universe.

simonebrunozzi 4y ago
In 2017, not in 2020.