36 ms·

One thing I've failed to find out about IPFS: who pays for hosting? The user? Or is it donated by some peers?

by proaralyst 9y ago

mbrock 9y ago
It's just like BitTorrent.
jstanley 9y ago
You host anything you want to host. If nobody wants to host something it disappears.
- rco8786 9y ago
  When you put it like that, it doesn't sound much different than how website hosting works today.
  - ashark 9y ago
    The content addressing and peer-to-peer architecture are the main differences.
  - jstanley 9y ago
    The difference is that if you decide to stop hosting, it doesn't just disappear. If there is anyone else in the world who wants it, it stays. It also means you don't need to trust the server to send you the correct content, since everything is automatically checked against its content hash. It also means it's automatically a CDN - you don't need to be able to serve every single person who wants to look at your content, because they'll all be serving it to each other.
    - DougN7 9y ago
      I'm having a hard time imagining this at scale. I have a few thousand web pages on my site, blog posts, etc. Most of it won't be seen in any given day. So I have to pin it as it's realistically unlikely anyone else will. Not much different from today. This probably describes 90% of web content out there, where a single node is all that is keeping that content alive. Now imagine something slightly more popular than my website. Maybe a particular Medium article. If it's two years old, it's unlikely to be getting any hits at this point, though it might have been popular at one time. So again, Medium has to pin it (or maybe the author would). Popular stuff would stay floating around while it's being viewed. It's almost a dynamic CDN/torrent which could help with bandwidth, but I don't see any advantage for persistence in most cases.
      - biggerfisch 9y ago
        I think the advantage is that it's easier for _anyone_ to be able to keep a site/file alive without having to implement an archive.org style of rewriting all the URLs/links. If you want, you could pin that medium article you talk about and now even if the medium node(s) decide its no longer worth pinning, you've still got it pinned so it's accessible to anyone at the expected "location". Similarly, this makes take-downs harder. It's no longer one node/machine hosting a file, but as many nodes as who find it valuable. If users never pinned anything, you'd be correct, it would be basically the same as the current internet, with one organization maintaining everything. But IPFS gives you the ability to change that.
        DougN7 9y ago
        Thank you for that perspective - that makes sense. I hope there is an easy way to pin an entire site and not page-by-page so, for example, a full documentation set could be kept.
        wolfgang42 9y ago
        Yes, content on IPFS is represented as a Merkle tree so if you pin the top of the tree it will traverse down and recursively pin all of the subtrees as well. (You can, of course, pin a subtree explicitly if there's a only certain part you want.)
      - StavrosK 9y ago
        Maybe you'll find an article interesting and want to pin it yourself. Maybe you yourself will link to it and want to pin it for that purpose. Maybe someone else will link to a very influential article you wrote and that will make it stay. Overall, it's a subset of the availability of a single server, plus the page always stays at the same "place" and never moves.
        ryukafalz 9y ago
        >Maybe you yourself will link to it and want to pin it for that purpose. This is an important point. If you pin all the content you link to, the links on your site will never break so long as your node stays up.
    - Paul-ish 9y ago
      > It also means you don't need to trust the server to send you the correct content, since everything is automatically checked against its content hash. This is the confusing part to me. How does updating content work? Lets say I want to visit the jstanley blog. Does this mean you can only ever publish once, because adding an entry would change the content hash? Does everyone have to contact you for the new info hash? Does ipfs work with existing DNS?
      - wolfgang42 9y ago
        On top of the content-addressed IPFS is an IPNS layer which maps addresses to content hashes. The address of a piece of content is a public key, which is used to sign the content hash. This means only nodes in possession of the private key can change where the IPNS address points to. In addition, IPFS gateways also support a special SRV record in DNS, which can point either directly to a hash or to an address which points to a hash. (This is how you can associate a domain name to content served by IPFS. If you also point the A record of the domain to a public gateway, the gateway will automatically look up the SRV record and return the content when someone retrieves it by HTTP.) At the moment, go-ipfs (the reference implementation) only supports pinning content hashes, so if you have someone else pinning your content (like a pinning service) you have to notify them of the change. There are plans to add support for mutable pins in the future. Put in the vocabulary of git: any given commit is referenced by hash[1] and is therefore immutable; IPNS addresses are like references (branches, etc) which can be updated at any time to point to a different commit. [1]: Since the content is a merkle tree, if they've only changed a few files most of the pinned content will retain the same hash.
        Paul-ish 9y ago
        That's pretty neat. Do you know if there is protection against rollback?
        wolfgang42 9y ago
        IPNS records have a fairly short expiry, so you have to keep republishing them. This also keeps the DHT from getting overly full of records that nobody is using anymore, but does mean that a node with the private key must be kept online. They're working on improving this situation; here's a discussion on the topic: https://discuss.ipfs.io/t/confusion-about-ipns/1414/4 https://discuss.ipfs.io/t/confusion-about-ipns/1414/4
  - hinkley 9y ago
    I am still learning about IPFS but I participated as a minor party in the design of one of its intellectual predecessors, Freenet. IPFS is not just going to act as an ad-hoc CDN, it's also going to work as a regional CDN. If someone does a report on your funny video on WGN-TV and ten thousand Chicagoans hit your tiny little website in the next ten minutes, your server will get a few hits because nobody else has it yet. After that the network effect kicks in, and eventually everyone is loading it from an IPFS server somewhere in Chicago. When they start 'retweeting' it and everyone is looking for it, it'll end up scattered all over the country. Until people lose interest, and then the copies will begin to disappear as the next fad automatically pushes it out.
diggan 9y ago
Similar to with HTTP, the ones hosting the content. You'll have to access the content or explicitly pin it to share it with other peers.
- samschooler 9y ago
  So in the case of the demo, is their server is hosting the content for 4 days?
  - reconbot 9y ago
    They seem to be using https://www.eternum.io https://www.eternum.io to "pin" the file (keep a copy of the file alive). Since eternum is a paid service they might unpin after a while but it's up to them. If they didn't do that, the file might only be available as long as your browser is open.
  - kpcyrd 9y ago
    In this case the file is served by a javascript implementation of ipfs running inside the browser over websockets. I'm not sure if the server is pinning the files as well.
  - fiatjaf 9y ago
    Yes.
kakarot 9y ago
You can't have dug too deep into IPFS without knowing that because it's one of the fundamental aspects of the design. Try reading their documentation. https://ipfs.io/docs/ https://ipfs.io/docs/
wolfgang42 9y ago
This is the #1 thing I see people confused about with IPFS. Basically, there's 3 ways for content to become available: 1. You add the files to your own node. This is how content gets added, but obviously it only lasts as long as your node is connected to the Internet, just like an ordinary HTTP server. 2. Someone views your content using their node. This causes their node to cache the content temporarily (IIRC for 30 minutes by default) and publish it to other nodes. In theory, if your content got at least one view every half hour it could live on in the users' caches forever. 3. Someone tells their node to pin your content. The node will then keep it permanently (until they unpin it) and serve it whenever it's connected to the Internet. Generally they would do this if they believe it's valuable--either because they want to keep it themselves, or as a public service to make it available to others (for example, pinning the Turkish Wikipedia to help evade censorship). There are also several pinning services, which you can pay to have them pin your content on their node, in much the same way as you'd pay a hosting provider to serve your content.
- robto 9y ago
  In this case, it appears the node is in your browser, right? Does it go away when you close your tab? In which case your link will survive for 30 minutes, or is it just gone?
  - wolfgang42 9y ago
    It looks like ipfs-dropzone is running an in-browser node, which would go away as soon as you close the tab (since it's just a web page like any other). Content on IPFS stays available for as long as there's at least one node which has it and is connected to the network, so how long the link would survive for depends on whether other nodes have it and how long they will keep it for. If nothing else has downloaded the content then it will go away; on the other hand, if someone has pinned it then it will stay forever (or until they unpin it or go away). The 30 minute interval would apply to the IPFS gateway that someone else would use to download your content (generally a standalone process though I believe there are efforts to make a browser with an embedded gateway). IPFS nodes automatically publish their cache, effectively making an autoscaling distributed CDN. However they will only cache content that they have gotten on behalf of their user; they won't go out and pull random content from the network to host it.
prophesi 9y ago
This hasn't been launched yet, but IPFS is also planning to introduce their cryptocurrency, Filecoin, to offer cheap hosting. People will then host your site (aka pin it on their IPFS node) for Filecoins, which should be cheaper than traditional file storage services.

One thing I've failed to find out about IPFS: who pays for hosting? The user? Or is it donated by some peers?

mbrock 9y ago

It's just like BitTorrent.

jstanley 9y ago

You host anything you want to host. If nobody wants to host something it disappears.

rco8786 9y ago
When you put it like that, it doesn't sound much different than how website hosting works today.
- ashark 9y ago
  The content addressing and peer-to-peer architecture are the main differences.
- jstanley 9y ago
  The difference is that if you decide to stop hosting, it doesn't just disappear. If there is anyone else in the world who wants it, it stays. It also means you don't need to trust the server to send you the correct content, since everything is automatically checked against its content hash. It also means it's automatically a CDN - you don't need to be able to serve every single person who wants to look at your content, because they'll all be serving it to each other.
  - DougN7 9y ago
    I'm having a hard time imagining this at scale. I have a few thousand web pages on my site, blog posts, etc. Most of it won't be seen in any given day. So I have to pin it as it's realistically unlikely anyone else will. Not much different from today. This probably describes 90% of web content out there, where a single node is all that is keeping that content alive. Now imagine something slightly more popular than my website. Maybe a particular Medium article. If it's two years old, it's unlikely to be getting any hits at this point, though it might have been popular at one time. So again, Medium has to pin it (or maybe the author would). Popular stuff would stay floating around while it's being viewed. It's almost a dynamic CDN/torrent which could help with bandwidth, but I don't see any advantage for persistence in most cases.
    - biggerfisch 9y ago
      I think the advantage is that it's easier for _anyone_ to be able to keep a site/file alive without having to implement an archive.org style of rewriting all the URLs/links. If you want, you could pin that medium article you talk about and now even if the medium node(s) decide its no longer worth pinning, you've still got it pinned so it's accessible to anyone at the expected "location". Similarly, this makes take-downs harder. It's no longer one node/machine hosting a file, but as many nodes as who find it valuable. If users never pinned anything, you'd be correct, it would be basically the same as the current internet, with one organization maintaining everything. But IPFS gives you the ability to change that.
      - DougN7 9y ago
        Thank you for that perspective - that makes sense. I hope there is an easy way to pin an entire site and not page-by-page so, for example, a full documentation set could be kept.
        wolfgang42 9y ago
        Yes, content on IPFS is represented as a Merkle tree so if you pin the top of the tree it will traverse down and recursively pin all of the subtrees as well. (You can, of course, pin a subtree explicitly if there's a only certain part you want.)
    - StavrosK 9y ago
      Maybe you'll find an article interesting and want to pin it yourself. Maybe you yourself will link to it and want to pin it for that purpose. Maybe someone else will link to a very influential article you wrote and that will make it stay. Overall, it's a subset of the availability of a single server, plus the page always stays at the same "place" and never moves.
      - ryukafalz 9y ago
        >Maybe you yourself will link to it and want to pin it for that purpose. This is an important point. If you pin all the content you link to, the links on your site will never break so long as your node stays up.
  - Paul-ish 9y ago
    > It also means you don't need to trust the server to send you the correct content, since everything is automatically checked against its content hash. This is the confusing part to me. How does updating content work? Lets say I want to visit the jstanley blog. Does this mean you can only ever publish once, because adding an entry would change the content hash? Does everyone have to contact you for the new info hash? Does ipfs work with existing DNS?
    - wolfgang42 9y ago
      On top of the content-addressed IPFS is an IPNS layer which maps addresses to content hashes. The address of a piece of content is a public key, which is used to sign the content hash. This means only nodes in possession of the private key can change where the IPNS address points to. In addition, IPFS gateways also support a special SRV record in DNS, which can point either directly to a hash or to an address which points to a hash. (This is how you can associate a domain name to content served by IPFS. If you also point the A record of the domain to a public gateway, the gateway will automatically look up the SRV record and return the content when someone retrieves it by HTTP.) At the moment, go-ipfs (the reference implementation) only supports pinning content hashes, so if you have someone else pinning your content (like a pinning service) you have to notify them of the change. There are plans to add support for mutable pins in the future. Put in the vocabulary of git: any given commit is referenced by hash[1] and is therefore immutable; IPNS addresses are like references (branches, etc) which can be updated at any time to point to a different commit. [1]: Since the content is a merkle tree, if they've only changed a few files most of the pinned content will retain the same hash.
      - Paul-ish 9y ago
        That's pretty neat. Do you know if there is protection against rollback?
        wolfgang42 9y ago
        IPNS records have a fairly short expiry, so you have to keep republishing them. This also keeps the DHT from getting overly full of records that nobody is using anymore, but does mean that a node with the private key must be kept online. They're working on improving this situation; here's a discussion on the topic: https://discuss.ipfs.io/t/confusion-about-ipns/1414/4 https://discuss.ipfs.io/t/confusion-about-ipns/1414/4
- hinkley 9y ago
  I am still learning about IPFS but I participated as a minor party in the design of one of its intellectual predecessors, Freenet. IPFS is not just going to act as an ad-hoc CDN, it's also going to work as a regional CDN. If someone does a report on your funny video on WGN-TV and ten thousand Chicagoans hit your tiny little website in the next ten minutes, your server will get a few hits because nobody else has it yet. After that the network effect kicks in, and eventually everyone is loading it from an IPFS server somewhere in Chicago. When they start 'retweeting' it and everyone is looking for it, it'll end up scattered all over the country. Until people lose interest, and then the copies will begin to disappear as the next fad automatically pushes it out.

diggan 9y ago

Similar to with HTTP, the ones hosting the content. You'll have to access the content or explicitly pin it to share it with other peers.

samschooler 9y ago
So in the case of the demo, is their server is hosting the content for 4 days?
- reconbot 9y ago
  They seem to be using https://www.eternum.io https://www.eternum.io to "pin" the file (keep a copy of the file alive). Since eternum is a paid service they might unpin after a while but it's up to them. If they didn't do that, the file might only be available as long as your browser is open.
- kpcyrd 9y ago
  In this case the file is served by a javascript implementation of ipfs running inside the browser over websockets. I'm not sure if the server is pinning the files as well.
- fiatjaf 9y ago
  Yes.

kakarot 9y ago

You can't have dug too deep into IPFS without knowing that because it's one of the fundamental aspects of the design. Try reading their documentation. https://ipfs.io/docs/ https://ipfs.io/docs/

wolfgang42 9y ago

This is the #1 thing I see people confused about with IPFS. Basically, there's 3 ways for content to become available: 1. You add the files to your own node. This is how content gets added, but obviously it only lasts as long as your node is connected to the Internet, just like an ordinary HTTP server. 2. Someone views your content using their node. This causes their node to cache the content temporarily (IIRC for 30 minutes by default) and publish it to other nodes. In theory, if your content got at least one view every half hour it could live on in the users' caches forever. 3. Someone tells their node to pin your content. The node will then keep it permanently (until they unpin it) and serve it whenever it's connected to the Internet. Generally they would do this if they believe it's valuable--either because they want to keep it themselves, or as a public service to make it available to others (for example, pinning the Turkish Wikipedia to help evade censorship). There are also several pinning services, which you can pay to have them pin your content on their node, in much the same way as you'd pay a hosting provider to serve your content.

robto 9y ago
In this case, it appears the node is in your browser, right? Does it go away when you close your tab? In which case your link will survive for 30 minutes, or is it just gone?
- wolfgang42 9y ago
  It looks like ipfs-dropzone is running an in-browser node, which would go away as soon as you close the tab (since it's just a web page like any other). Content on IPFS stays available for as long as there's at least one node which has it and is connected to the network, so how long the link would survive for depends on whether other nodes have it and how long they will keep it for. If nothing else has downloaded the content then it will go away; on the other hand, if someone has pinned it then it will stay forever (or until they unpin it or go away). The 30 minute interval would apply to the IPFS gateway that someone else would use to download your content (generally a standalone process though I believe there are efforts to make a browser with an embedded gateway). IPFS nodes automatically publish their cache, effectively making an autoscaling distributed CDN. However they will only cache content that they have gotten on behalf of their user; they won't go out and pull random content from the network to host it.

prophesi 9y ago

This hasn't been launched yet, but IPFS is also planning to introduce their cryptocurrency, Filecoin, to offer cheap hosting. People will then host your site (aka pin it on their IPFS node) for Filecoins, which should be cheaper than traditional file storage services.