4 ms·

Total noob questions, would love some answers if anyone has them. Here it says: "IPFS is a peer-to-peer file system composed of thousands of computers around

Total noob questions, would love some answers if anyone has them. Here it says: "IPFS is a peer-to-peer file system composed of thousands of computers around the world, each of which stores files on behalf of the network. These files can be anything: cat pictures, 3D models, or even entire websites" - If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff? Then it says: "The content with a hash of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy could be stored on dozens of nodes, so if one node that was caching that content goes down, the network will just look for the content on another node." - How long does it take for 1 specific file to be "re-distributed" on another node? and how many times is a file rehosted/distributed on a different node? It would be stupid to have the same file hosted 1.000.000 times - what's the break off point? Is there like a healthcheck so to speak which ensures that a certain file is in danger of disappearing from the network and therefore is automatically pushed for re-distribution And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them? But hold up, isn't p2p file system like this all about trusting the network and not 1 server?

diggan 8y ago
The IPFS network works in a pull-model, not push. So when nodes add content, they only add it locally. It's not until some other nodes request it, as it actually transfers. So if you start a IPFS node locally, it won't get content pushed to it (well, some DHT data which is basically routing information, but not the content itself). > ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently (maybe in the future?). So, download file A and add it to IPFS from your end. If you end up with the same hash, the content was not modified. > isn't p2p file system like this all about trusting the network and not 1 server Yes, IPFS <> IPFS is trustless as this verification happens automatically and the clients can verify content locally when it gets fetched. IPFS <> HTTP is harder as there is no functionality for doing this verification.
- deckar01 8y ago
  IPFS is a protocol that dictates how it's clients are supposed to validate the data. If you use any of the client libraries they will do this for you (go-ipfs/js-ipfs). If you want to use cloudflare to distribute your index, you need to trust them to use a valid client implementation. Otherwise you can serve an index file off of your own server/network and distribute the rest of the assets via IPFS.
- the8472 8y ago
  > You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently Browser extensions could by verifying content as it arrives. https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/filterResponseData https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...
  - AgentME 8y ago
    A browser extension could be used to request the data straight from ipfs instead of Cloudflare.
    - akavel 8y ago
      The extension exists: https://github.com/ipfs-shipyard/ipfs-companion https://github.com/ipfs-shipyard/ipfs-companion
    - bren2013 8y ago
      I believe the IPFS team has already built a browser extension that runs a js-ipfs node. Running an IPFS node is not ideal for power and bandwidth constrained devices like a mobile phone. This is where I see gateways fitting in, long-term. Not having to trust a gateway unconditionally is necessary to make this viable.
acover 8y ago
It's like a torrent. You don't host data that you haven't accessed. For content to stay hosted on your computer forever you have to explicitly pin it. The number of people hosting it is the number of people accessing it. CloudFlare can't modify the files. On IPFS you request files by a hash. If they give you a file with a different hash you know they gave you the wrong file.
- niij 8y ago
  I think the point is that they're hosting an HTTP proxy to IPFS. Since you're accessing IPFS files over their proxy they can modify the response you receive, since at that point it's just like any other website.
  - asdkhadsj 8y ago
    > Since you're accessing IPFS files over their proxy they can modify the response you receive, since at that point it's just like any other website. Correct, but gateways aren't intended to be final solutions. They're compromises targeted at increasing adoption. You wouldn't try and reinvent the internet - you'd make the interoperable where possible, such that adopters can still send normal links, etc.
furi 8y ago
> And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them? The hash serves that purpose, the file is identified by the hash for the purposes of retrieval and coming up with another file with the same hash (especially one that's usefully different, like changing the overall message in somebody's blog post) is computationally infeasible. Now you might have a problem there in that in a web only implementation like this the code that checks the hash is probably also provided by Cloudflare, but the functionality is there, you can download the file, compute the hash and compare it against the one you retrieved it from. > But hold up, isn't p2p file system like this all about trusting the network and not 1 server? Ideally yes but 1. it's very difficult to get more than a handful of users to use anything that doesn't run in a web browser, 2. IPFS' implementation for Windows is effectively unusable (it's a command line interface like the Linux one, which is a non-starter on Windows), 3. attempts to built a complete IPFS node in the browser are currently incomplete, I believe because DHT discovery isn't possible in the browser presently. These gateways should probably be viewed as a stop-gap solution until such a time as we can have full IPFS nodes in the browser.
- grittygrease 8y ago
  This browser extension can help you guarantee e2e integrity in the meantime: https://blog.cloudflare.com/e2e-integrity/ https://blog.cloudflare.com/e2e-integrity/
oscargrouch 8y ago
> - If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff? Thats some of the reasons why i prefer the 'Dat' and torrent(be it classic or DHT) model of distribution per content, instead of IPFS which is per block. The IPFS model make a lot of sense for cloud storage.. You might create a whole CDN based on its model where the clients dont even need to care about your backend storage layer. But in the per-content p2p distribution model, we have the 'page-rank' effect, where popular or important stuff, curated by human reasoning of whats popular and important. This model has worked pretty well with torrent, and public interest should drive the p2p storage model, where by using blocks, you lose some of the control and the context. And to see how this is a problem, the IPFS folks created the Filecoin to deal with the problem of the incentives, which on the 'per-content' model of Dat and Torrent, its a non-issue because of the natural page ranking indexing of content drive by public interest. If something doesnt matter much, it will go away with time, just like our human brain works. (Of course, if IPFS add some form of indexing of content by public interest on top of the block system, and even let you control what you serve. That problem would be solved)
- diggan 8y ago
  (Disclaimer: I work for Protocol Labs on IPFS and other projects [not Filecoin though]) I'm not sure I understand what's different in the way "seeding" or "hosting" of data in Dat and Torrents compared to IPFS. All those requires you to initiate the transfer, nothing gets transferred by itself (unlike FreeNet for example). Blocks in IPFS is just a detail that a file bigger than X MB gets split up into many smaller "files" so they can be reused between files and simpler to transfer. It has nothing to do with which peers are hosting the content. In the end, in IPFS, Dat and Torrents, you download and share content based on a ID, and that ID can mean an entire archive of content, or just one picture.
  - oscargrouch 8y ago
    Its not the technical part. Overall i even think that technically, IPFS has a superior model. I understand it works, and how it works. What im saying is the control of whats important in a organic form is no there yet. The protocol controls what blocks the peers serve and people dont have much of a saying into it. But given IPFS its layered in a good design, this could be changed and the block scheduling/routing could be binded more to this organic page-rank by letting people control more which blocks they serve, being more content aware. This could be even done in a topic level, say you are ok to serve "free software". I guess that if you guys implemented something like that, giving some control to the peers of what they serve, it will be much more easy to sell the concept to everybody.
bren2013 8y ago
> And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? https://blog.cloudflare.com/e2e-integrity/ https://blog.cloudflare.com/e2e-integrity/

diggan 8y ago

The IPFS network works in a pull-model, not push. So when nodes add content, they only add it locally. It's not until some other nodes request it, as it actually transfers. So if you start a IPFS node locally, it won't get content pushed to it (well, some DHT data which is basically routing information, but not the content itself). > ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently (maybe in the future?). So, download file A and add it to IPFS from your end. If you end up with the same hash, the content was not modified. > isn't p2p file system like this all about trusting the network and not 1 server Yes, IPFS <> IPFS is trustless as this verification happens automatically and the clients can verify content locally when it gets fetched. IPFS <> HTTP is harder as there is no functionality for doing this verification.

deckar01 8y ago
IPFS is a protocol that dictates how it's clients are supposed to validate the data. If you use any of the client libraries they will do this for you (go-ipfs/js-ipfs). If you want to use cloudflare to distribute your index, you need to trust them to use a valid client implementation. Otherwise you can serve an index file off of your own server/network and distribute the rest of the assets via IPFS.
the8472 8y ago
> You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently Browser extensions could by verifying content as it arrives. https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/webRequest/filterResponseData https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/Web...
- AgentME 8y ago
  A browser extension could be used to request the data straight from ipfs instead of Cloudflare.
  - akavel 8y ago
    The extension exists: https://github.com/ipfs-shipyard/ipfs-companion https://github.com/ipfs-shipyard/ipfs-companion
  - bren2013 8y ago
    I believe the IPFS team has already built a browser extension that runs a js-ipfs node. Running an IPFS node is not ideal for power and bandwidth constrained devices like a mobile phone. This is where I see gateways fitting in, long-term. Not having to trust a gateway unconditionally is necessary to make this viable.

acover 8y ago

It's like a torrent. You don't host data that you haven't accessed. For content to stay hosted on your computer forever you have to explicitly pin it. The number of people hosting it is the number of people accessing it. CloudFlare can't modify the files. On IPFS you request files by a hash. If they give you a file with a different hash you know they gave you the wrong file.

niij 8y ago
I think the point is that they're hosting an HTTP proxy to IPFS. Since you're accessing IPFS files over their proxy they can modify the response you receive, since at that point it's just like any other website.
- asdkhadsj 8y ago
  > Since you're accessing IPFS files over their proxy they can modify the response you receive, since at that point it's just like any other website. Correct, but gateways aren't intended to be final solutions. They're compromises targeted at increasing adoption. You wouldn't try and reinvent the internet - you'd make the interoperable where possible, such that adopters can still send normal links, etc.

furi 8y ago

> And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them? The hash serves that purpose, the file is identified by the hash for the purposes of retrieval and coming up with another file with the same hash (especially one that's usefully different, like changing the overall message in somebody's blog post) is computationally infeasible. Now you might have a problem there in that in a web only implementation like this the code that checks the hash is probably also provided by Cloudflare, but the functionality is there, you can download the file, compute the hash and compare it against the one you retrieved it from. > But hold up, isn't p2p file system like this all about trusting the network and not 1 server? Ideally yes but 1. it's very difficult to get more than a handful of users to use anything that doesn't run in a web browser, 2. IPFS' implementation for Windows is effectively unusable (it's a command line interface like the Linux one, which is a non-starter on Windows), 3. attempts to built a complete IPFS node in the browser are currently incomplete, I believe because DHT discovery isn't possible in the browser presently. These gateways should probably be viewed as a stop-gap solution until such a time as we can have full IPFS nodes in the browser.

grittygrease 8y ago
This browser extension can help you guarantee e2e integrity in the meantime: https://blog.cloudflare.com/e2e-integrity/ https://blog.cloudflare.com/e2e-integrity/

oscargrouch 8y ago

> - If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff? Thats some of the reasons why i prefer the 'Dat' and torrent(be it classic or DHT) model of distribution per content, instead of IPFS which is per block. The IPFS model make a lot of sense for cloud storage.. You might create a whole CDN based on its model where the clients dont even need to care about your backend storage layer. But in the per-content p2p distribution model, we have the 'page-rank' effect, where popular or important stuff, curated by human reasoning of whats popular and important. This model has worked pretty well with torrent, and public interest should drive the p2p storage model, where by using blocks, you lose some of the control and the context. And to see how this is a problem, the IPFS folks created the Filecoin to deal with the problem of the incentives, which on the 'per-content' model of Dat and Torrent, its a non-issue because of the natural page ranking indexing of content drive by public interest. If something doesnt matter much, it will go away with time, just like our human brain works. (Of course, if IPFS add some form of indexing of content by public interest on top of the block system, and even let you control what you serve. That problem would be solved)

diggan 8y ago
(Disclaimer: I work for Protocol Labs on IPFS and other projects [not Filecoin though]) I'm not sure I understand what's different in the way "seeding" or "hosting" of data in Dat and Torrents compared to IPFS. All those requires you to initiate the transfer, nothing gets transferred by itself (unlike FreeNet for example). Blocks in IPFS is just a detail that a file bigger than X MB gets split up into many smaller "files" so they can be reused between files and simpler to transfer. It has nothing to do with which peers are hosting the content. In the end, in IPFS, Dat and Torrents, you download and share content based on a ID, and that ID can mean an entire archive of content, or just one picture.
- oscargrouch 8y ago
  Its not the technical part. Overall i even think that technically, IPFS has a superior model. I understand it works, and how it works. What im saying is the control of whats important in a organic form is no there yet. The protocol controls what blocks the peers serve and people dont have much of a saying into it. But given IPFS its layered in a good design, this could be changed and the block scheduling/routing could be binded more to this organic page-rank by letting people control more which blocks they serve, being more content aware. This could be even done in a topic level, say you are ok to serve "free software". I guess that if you guys implemented something like that, giving some control to the peers of what they serve, it will be much more easy to sell the concept to everybody.

bren2013 8y ago

> And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? https://blog.cloudflare.com/e2e-integrity/ https://blog.cloudflare.com/e2e-integrity/