8 ms·

I don't see how people consider this and IPFS decentralized. You still need peers to subscribe and pin, and services like Pinata aren't cheap and still centrali

I don't see how people consider this and IPFS decentralized. You still need peers to subscribe and pin, and services like Pinata aren't cheap and still centralized. Is there an example of websites that would continue to run indefinitely if there wasn't a centralization of services with IPFS? I just don't think it is likely unless you start distributing your app that bundles IPFS, so that your users end up hosting it for you. However, if someone new tries it out and only one of your users has your app open you better hope it is being cached by Cloudflare.

aaomidi 6y ago
What do you think decentralized means? It means that hosting it isn't the responsibility of a single party. If I put something online, and people think it's interesting and pin it. Even if I go down, the pinned content remains up. Decentralized doesn't magically solve hosting availability. It just removes responsibility from a single party.
- encryptluks2 6y ago
  I would say there are very few IPFS sites where hosting doesn't still end up being the primary responsibility of a single party. That was the point I was trying to make. If you have to pay someone else to pin it and keep giving them funds then it is still your responsibility. If you have a website that you want to scale and load fast for all your users on IPFS then it will still end up being your responsibility. There may be a few exceptions.
  - aaomidi 6y ago
    There is no technology possible that removes said responsibility. This is how peer to peer systems work. Your criticism is something that no peer to peer system claims to solve.
    - 9000 6y ago
      Actually, I recommend you check out Freenet [0], a project that's 20 years old. In fact, the Wikipedia page explicitly states: > Information flow in Freenet is different from networks like eMule or BitTorrent; in Freenet: > 1. A user wishing to share a file or update a freesite "inserts" the file "to the network" > 2. After "insertion" is finished, the publishing node is free to shut down, because the file is stored in the network. It will remain available for other users whether or not the original publishing node is online. No single node is responsible for the content; instead, it is replicated to many different nodes. [0] https://en.wikipedia.org/wiki/Freenet https://en.wikipedia.org/wiki/Freenet
      - octopoc 6y ago
        Aren’t collaborative systems like that susceptible to abuse? E.g., an attacker could upload 1TB of noise and then shut down, reducing the network size by much more than 1TB.
      - FeepingCreature 6y ago
        Of course, in Freenet, if a file is not requested often it will quickly fall off the network due to user churn.
    - calibas 6y ago
      He said that IPFS hosting is the "primary responsibility of a single party". The whole idea behind peer to peer is to avoid precisely that.
      - FeepingCreature 6y ago
        Not really, if you want something to be perpetually available in any p2p network you need to ensure that you're still hosting it. That goes for torrents, ed2k, freenet, and ipfs too.
        argvargc 6y ago
        This isn't true of most blockchain p2p networks, eg, Bitcoin. Anything uploaded to these lasts as long as the entire network does. The reason this works is it costs some amount to initially print the data to the network, limiting that kind of abuse.
        aaomidi 6y ago
        And because there is money involved in making it last as long as the entire network. Once the monetary reasons drop then so does the data.
        argvargc 6y ago
        It's possible for sure, but with Bitcoin and the overall crypto space steadily growing in value into the trillions over the course of over a decade, it seems unlikely.
        calibas 6y ago
        I'm not claiming the reality of P2P always matches the ideas behind it. And surely you're not saying centralized hosting being a strict requirement was what people had in mind for P2P?
        FeepingCreature 6y ago
    - argvargc 6y ago
      Except for nearly all blockchain-based p2p networks. Eg: https://cryptograffiti.info https://cryptograffiti.info
  - BelenusMordred 6y ago
    Would you call torrents decentralised?
    - encryptluks2 6y ago
      To an extent, although many times you still rely on a centralized tracker and dedicated seeds. Unless you belong to a private tracker where users are forced to maintain certain ratios to continue using a service, then in most cases a torrent will end up like an IPFS site without any pins. If IPFS provided the ability to track pins from users of a site like private trackers, and provide rewards or some incentive then I think IPFS would gain much wider adoption.
      - t3rr0r 6y ago
        you're describing an architecture similar to a project I'm working on using orbitdb/ipfslog - it's not quite ready for sharing/using but there's more information here: https://www.reddit.com/r/musichoarder/comments/lrqx7m/record_a_distributed_audio_file_system/ https://www.reddit.com/r/musichoarder/comments/lrqx7m/record...
        encryptluks2 6y ago
        Thanks. Napster 3.0 here we come ;)
      - pythonaut_16 6y ago
        Most torrent clients also "pin" content a user downloads by default, with configuration options to stop seeding once certain conditions or upload ratios are met. Does IPFS support something similar? e.g. pin this content until I have a 5:1 upload to download ratio --- I 100% agree on the ability to track pins. I want to see organizations be able to easily peer/pin content from other orgs they endorse; and for end users to be able to follow and help peer for organizations they support. I want something like Youtube where I choose who appears as in the related videos on my channel and can be a peer to hosting them; and where my users can help to peer my videos (and I can help peer others). As an end user, my client (or local server/cache) would prefetch the content I follow both to be a peer and to have it ready for me to access. This would also help with locality of data; if 4 people in my household watch the same video or channel they would be able to get the content locally. If I'm an organization with a physical location, users connected to my network can fetch the content on a local link.
  - calibas 6y ago
    You're speaking from actual experience, aren't you? I tried out IPFS and came to the same conclusion myself. Content has to be kept alive or it disappears forever, and keeping it alive requires some kind of investment. I like IPFS, I want it to succeed, but HN has these romantic ideas about IPFS that don't quite match reality, and you get downvoted for being honest sometimes. Not acknowledging the limits of IPFS are part of what holds it back...
    - deleted 6y ago
      [deleted]
    - encryptluks2 6y ago
      Yes. I really enjoy IPFS and fully intend to use it in my next project, but there is still quite a bit to be desired. I think the IPFS cluster project may end up being what I'm looking for, but it currently has a few issues that need to be worked out: https://github.com/ipfs/ipfs-cluster/issues/1226 https://github.com/ipfs/ipfs-cluster/issues/1226 Ideally, I could provide instructions for app users to follow my cluster, set the amount of content they want to share, track the size of content they've pinned and for how long, and provide rewards/incentives through the app for doing so.. similar to how private trackers work.
  - kevincox 6y ago
    Of course. Just using IPFS doesn't make other people interested in your content. But if you have interesting content it does mean that people will be likely to save it. You can imagine that if YouTube hosted its videos on IPFS then popular channels would be widely pinned so that even if YouTube shut down and the channel author left most of the videos would still be accessible. And much unlike current HTTP mirrors you would be able to find out who is pinning it without having to check multiple archive sites. (Of course you need to know the hash, but remembering hashes is much lest costly than mirroring whole sites yourself so I think we would see lots of these) I think that in most cases it is simplest to think of IPFS as torrents. I think the biggest improvement that IPFS made was a single namespace for content instead of each torrent basically being its own swarm. This means that you can find content no matter how it is packaged and you can share blocks between "torrents".
hosh 6y ago
I'm looking at integrating IPFS with sandstorm (sandstorm.io). The sandstorm (and other homelab/homeserver) community have an interest in self-hosting. IPFS allows people that are interested in seeing something stored to pin it themselves. But yes: bundling IPFS with the app is pretty much what would need to happen. As they are now, neither the IPFS ecosystem or sandstorm are really decentralized -- or maybe I should be more precise: not decentralized in a resilient way. Yet, I think both are great technologies that enable some interesting things. I'm not as interested in the blockchain notion of decentralized though. I'm looking for resiliency to severe netsplits. As a thought experiment, if Earth crosses a coronal mass ejection event and our power and communication grids go down, would we be able stitch things back together again when our grid fractures?
- secfirstmd 6y ago
  Very cool idea!
- pm 6y ago
  Curious about the homelab/homeserver community. Are there any sites with further information?
  - hosh 6y ago
    If you reddit, r/selfhosted and r/homelab are a great place to start. I really like the sandstorm.io platform, though it is not currently supported on an rpi. It has the potential as a platform that “grandma” can use. There is a project called HomelabOS. It reminds me a bit of Homebrew.
    - pm 6y ago
      Cheers. I remember hearing SandStorm (the company) went under a couple years ago, and was curious but hadn't explored it in depth. I've been paying attention to CollapseOS as well, though it's a little more apocalyptic in nature. The eventual aim is to be able to provision a set of servers from a thumb drive without Internet (if that's even possible), but I haven't had much time to get far.
markhenderson 6y ago
Swarm persistence is hard, if not close to impossible. Better that multiple parties pin the data than just one though. It's a step in the right direction.
- encryptluks2 6y ago
  Agreed. I started looking into setting up cluster followers, but currently you have to have enough space to pin everything and can't have it only do a percentage.
t3rr0r 6y ago
I share your viewpoint and frustration with "decentralized" apps. But as you said it is indeed possible by bundling your app with IPFS and using something like orbitdb/ipfslog for structuring data. I linked this in my other comment, but take a look at this proof of concept: https://www.reddit.com/r/musichoarder/comments/lrqx7m/record_a_distributed_audio_file_system/ https://www.reddit.com/r/musichoarder/comments/lrqx7m/record... At this point the major challenges are with UX, especially for users who are not accustomed to self-hosting. I think we have the technologies and ideas to overcome the rest.
bawolff 6y ago
I think that's the big problem with the "decentralized" WWW movement. There are about a billion conflicting definitions of decentralized providing very different properties, and it's never clear which definition people mean.
saurik 6y ago
It is decentralized in that anyone can step up and host the content on IPFS and it is considered equivalent and secure. With a normal website, if the canonical host goes down, someone else can host it, but it isn't the same website.
Geee 6y ago
IPFS, torrents and many old-school p2p-networks are decentralized to an extent, but they're also non-sybil-resistant and voluntarist, which makes them unsuitable for many use cases. They're not really a part of what is meant by decentralized web. You should take a look at SkyDB to get an idea what's possible with true decentralization: https://blog.sia.tech/skydb-a-mutable-database-for-the-decentralized-web-7170beeaa985 https://blog.sia.tech/skydb-a-mutable-database-for-the-decen...
amw-zero 6y ago
I feel like you don't understand the definition of "peer to peer." The fact that you require nodes to ultimate do the hosting of data does not make IPFS centralized.

aaomidi 6y ago

What do you think decentralized means? It means that hosting it isn't the responsibility of a single party. If I put something online, and people think it's interesting and pin it. Even if I go down, the pinned content remains up. Decentralized doesn't magically solve hosting availability. It just removes responsibility from a single party.

encryptluks2 6y ago
I would say there are very few IPFS sites where hosting doesn't still end up being the primary responsibility of a single party. That was the point I was trying to make. If you have to pay someone else to pin it and keep giving them funds then it is still your responsibility. If you have a website that you want to scale and load fast for all your users on IPFS then it will still end up being your responsibility. There may be a few exceptions.
- aaomidi 6y ago
  There is no technology possible that removes said responsibility. This is how peer to peer systems work. Your criticism is something that no peer to peer system claims to solve.
  - 9000 6y ago
    Actually, I recommend you check out Freenet [0], a project that's 20 years old. In fact, the Wikipedia page explicitly states: > Information flow in Freenet is different from networks like eMule or BitTorrent; in Freenet: > 1. A user wishing to share a file or update a freesite "inserts" the file "to the network" > 2. After "insertion" is finished, the publishing node is free to shut down, because the file is stored in the network. It will remain available for other users whether or not the original publishing node is online. No single node is responsible for the content; instead, it is replicated to many different nodes. [0] https://en.wikipedia.org/wiki/Freenet https://en.wikipedia.org/wiki/Freenet
    - octopoc 6y ago
      Aren’t collaborative systems like that susceptible to abuse? E.g., an attacker could upload 1TB of noise and then shut down, reducing the network size by much more than 1TB.
    - FeepingCreature 6y ago
      Of course, in Freenet, if a file is not requested often it will quickly fall off the network due to user churn.
  - calibas 6y ago
    He said that IPFS hosting is the "primary responsibility of a single party". The whole idea behind peer to peer is to avoid precisely that.
    - FeepingCreature 6y ago
      Not really, if you want something to be perpetually available in any p2p network you need to ensure that you're still hosting it. That goes for torrents, ed2k, freenet, and ipfs too.
      - argvargc 6y ago
        This isn't true of most blockchain p2p networks, eg, Bitcoin. Anything uploaded to these lasts as long as the entire network does. The reason this works is it costs some amount to initially print the data to the network, limiting that kind of abuse.
        aaomidi 6y ago
        And because there is money involved in making it last as long as the entire network. Once the monetary reasons drop then so does the data.
        argvargc 6y ago
        It's possible for sure, but with Bitcoin and the overall crypto space steadily growing in value into the trillions over the course of over a decade, it seems unlikely.
      - calibas 6y ago
        I'm not claiming the reality of P2P always matches the ideas behind it. And surely you're not saying centralized hosting being a strict requirement was what people had in mind for P2P?
        FeepingCreature 6y ago
  - argvargc 6y ago
    Except for nearly all blockchain-based p2p networks. Eg: https://cryptograffiti.info https://cryptograffiti.info
- BelenusMordred 6y ago
  Would you call torrents decentralised?
  - encryptluks2 6y ago
    To an extent, although many times you still rely on a centralized tracker and dedicated seeds. Unless you belong to a private tracker where users are forced to maintain certain ratios to continue using a service, then in most cases a torrent will end up like an IPFS site without any pins. If IPFS provided the ability to track pins from users of a site like private trackers, and provide rewards or some incentive then I think IPFS would gain much wider adoption.
    - t3rr0r 6y ago
      you're describing an architecture similar to a project I'm working on using orbitdb/ipfslog - it's not quite ready for sharing/using but there's more information here: https://www.reddit.com/r/musichoarder/comments/lrqx7m/record_a_distributed_audio_file_system/ https://www.reddit.com/r/musichoarder/comments/lrqx7m/record...
      - encryptluks2 6y ago
        Thanks. Napster 3.0 here we come ;)
    - pythonaut_16 6y ago
      Most torrent clients also "pin" content a user downloads by default, with configuration options to stop seeding once certain conditions or upload ratios are met. Does IPFS support something similar? e.g. pin this content until I have a 5:1 upload to download ratio --- I 100% agree on the ability to track pins. I want to see organizations be able to easily peer/pin content from other orgs they endorse; and for end users to be able to follow and help peer for organizations they support. I want something like Youtube where I choose who appears as in the related videos on my channel and can be a peer to hosting them; and where my users can help to peer my videos (and I can help peer others). As an end user, my client (or local server/cache) would prefetch the content I follow both to be a peer and to have it ready for me to access. This would also help with locality of data; if 4 people in my household watch the same video or channel they would be able to get the content locally. If I'm an organization with a physical location, users connected to my network can fetch the content on a local link.
- calibas 6y ago
  You're speaking from actual experience, aren't you? I tried out IPFS and came to the same conclusion myself. Content has to be kept alive or it disappears forever, and keeping it alive requires some kind of investment. I like IPFS, I want it to succeed, but HN has these romantic ideas about IPFS that don't quite match reality, and you get downvoted for being honest sometimes. Not acknowledging the limits of IPFS are part of what holds it back...
  - deleted 6y ago
    [deleted]
  - encryptluks2 6y ago
    Yes. I really enjoy IPFS and fully intend to use it in my next project, but there is still quite a bit to be desired. I think the IPFS cluster project may end up being what I'm looking for, but it currently has a few issues that need to be worked out: https://github.com/ipfs/ipfs-cluster/issues/1226 https://github.com/ipfs/ipfs-cluster/issues/1226 Ideally, I could provide instructions for app users to follow my cluster, set the amount of content they want to share, track the size of content they've pinned and for how long, and provide rewards/incentives through the app for doing so.. similar to how private trackers work.
- kevincox 6y ago
  Of course. Just using IPFS doesn't make other people interested in your content. But if you have interesting content it does mean that people will be likely to save it. You can imagine that if YouTube hosted its videos on IPFS then popular channels would be widely pinned so that even if YouTube shut down and the channel author left most of the videos would still be accessible. And much unlike current HTTP mirrors you would be able to find out who is pinning it without having to check multiple archive sites. (Of course you need to know the hash, but remembering hashes is much lest costly than mirroring whole sites yourself so I think we would see lots of these) I think that in most cases it is simplest to think of IPFS as torrents. I think the biggest improvement that IPFS made was a single namespace for content instead of each torrent basically being its own swarm. This means that you can find content no matter how it is packaged and you can share blocks between "torrents".

hosh 6y ago

I'm looking at integrating IPFS with sandstorm (sandstorm.io). The sandstorm (and other homelab/homeserver) community have an interest in self-hosting. IPFS allows people that are interested in seeing something stored to pin it themselves. But yes: bundling IPFS with the app is pretty much what would need to happen. As they are now, neither the IPFS ecosystem or sandstorm are really decentralized -- or maybe I should be more precise: not decentralized in a resilient way. Yet, I think both are great technologies that enable some interesting things. I'm not as interested in the blockchain notion of decentralized though. I'm looking for resiliency to severe netsplits. As a thought experiment, if Earth crosses a coronal mass ejection event and our power and communication grids go down, would we be able stitch things back together again when our grid fractures?

secfirstmd 6y ago
Very cool idea!
pm 6y ago
Curious about the homelab/homeserver community. Are there any sites with further information?
- hosh 6y ago
  If you reddit, r/selfhosted and r/homelab are a great place to start. I really like the sandstorm.io platform, though it is not currently supported on an rpi. It has the potential as a platform that “grandma” can use. There is a project called HomelabOS. It reminds me a bit of Homebrew.
  - pm 6y ago
    Cheers. I remember hearing SandStorm (the company) went under a couple years ago, and was curious but hadn't explored it in depth. I've been paying attention to CollapseOS as well, though it's a little more apocalyptic in nature. The eventual aim is to be able to provision a set of servers from a thumb drive without Internet (if that's even possible), but I haven't had much time to get far.

markhenderson 6y ago

Swarm persistence is hard, if not close to impossible. Better that multiple parties pin the data than just one though. It's a step in the right direction.

encryptluks2 6y ago
Agreed. I started looking into setting up cluster followers, but currently you have to have enough space to pin everything and can't have it only do a percentage.

t3rr0r 6y ago

I share your viewpoint and frustration with "decentralized" apps. But as you said it is indeed possible by bundling your app with IPFS and using something like orbitdb/ipfslog for structuring data. I linked this in my other comment, but take a look at this proof of concept: https://www.reddit.com/r/musichoarder/comments/lrqx7m/record_a_distributed_audio_file_system/ https://www.reddit.com/r/musichoarder/comments/lrqx7m/record... At this point the major challenges are with UX, especially for users who are not accustomed to self-hosting. I think we have the technologies and ideas to overcome the rest.

bawolff 6y ago

I think that's the big problem with the "decentralized" WWW movement. There are about a billion conflicting definitions of decentralized providing very different properties, and it's never clear which definition people mean.

saurik 6y ago

It is decentralized in that anyone can step up and host the content on IPFS and it is considered equivalent and secure. With a normal website, if the canonical host goes down, someone else can host it, but it isn't the same website.

Geee 6y ago

IPFS, torrents and many old-school p2p-networks are decentralized to an extent, but they're also non-sybil-resistant and voluntarist, which makes them unsuitable for many use cases. They're not really a part of what is meant by decentralized web. You should take a look at SkyDB to get an idea what's possible with true decentralization: https://blog.sia.tech/skydb-a-mutable-database-for-the-decentralized-web-7170beeaa985 https://blog.sia.tech/skydb-a-mutable-database-for-the-decen...

amw-zero 6y ago

I feel like you don't understand the definition of "peer to peer." The fact that you require nodes to ultimate do the hosting of data does not make IPFS centralized.