15 ms·

Introducing Cloudflare’s IPFS Gateway

621 points by jgrahamc 8y ago

styfle 8y ago
Here is one really important part that I almost missed when reading this announcement: > Using Cloudflare's gateway, you can also build a website that’s hosted entirely on IPFS, but still available to your users at a custom domain name. This is huge step forward for the distributed web!
- koalalorenzo 8y ago
  You can use any gateway to do that :)
  - toomuchtodo 8y ago
    Cloudflare offering it might legitimize IPFS in ways that non-corporate entities championing it doesn't. Not a huge Cloudflare fan, but I will admit this is cool of them to spend engineering time and money on. EDIT: Thanks for the efforts to promote the Distributed Web. A surprise, to be sure, but a welcome one.
    - jgrahamc 8y ago
      That's the whole point. We think the Distributed Web is cool and coming and we want to accelerate that happening. Hence the time and effort to make this happen.
      - deleted 8y ago
        [deleted]
      - fold_left 8y ago
        Bravo
      - carapace 8y ago
        I was just wondering the other day if IPFS would get enough traction to reach critical mass. This is a very hopeful sign I think. Cheers!
        steven_pack 8y ago
        That was one of the drivers. We liked the core technology, but could see there was a significant barrier to actually using it. Being able to point your domain at IPFS content and then providing that (ssl secured) site to the masses is a big step forward.
      - mtgx 8y ago
        A most welcome feature.
      - nil_pointer 8y ago
        That is wonderful to hear
  - styfle 8y ago
    You can use any gateway with your custom domain name? Potentially you could but I don't know of any that let you do this. My impression of the public gateways was that the end user would need to know the content hash of your website which makes for a poor user experience. This announcement claims the end user just uses the website as they normally would by visiting example.com but behind the scenes there is no central web server because the content is pulled from IPFS. DNS was the missing piece :) https://developers.cloudflare.com/distributed-web/ipfs-gateway/connecting-website/ https://developers.cloudflare.com/distributed-web/ipfs-gatew...
    - lgierth 8y ago
      You can use custom domains with any go-ipfs gateway, it'll transparently turn `Host: example.com` into `/ipns/example.com`. What Cloudflare adds to the mix is their infrastructure for automatic TLS certs.
    - marius_k 8y ago
      I was running my own website with my own domain on ipfs quite successfully for few months. And I have configured cloudflare just to have a green https icon. The only problem is that I have to visit my website once in a while, otherwise the content is deleted on the gateway.
  - andrewxhill 8y ago
    We wrote a tutorial on setting it up on your own (secure nginx server for ipfs gateway), https://medium.com/textileio/tutorial-setting-up-an-ipfs-peer-part-ii-67a99cd2c5 https://medium.com/textileio/tutorial-setting-up-an-ipfs-pee.... But for sure, for most users, going right through cloudflare is an awesome simplification. Exciting stuff.
- fiatjaf 8y ago
  The killer app for IPFS in this early days should be people who are paying $10/mo to host their half-dozen HTML files on WhateverHost companies everywhere. Bonus points if they are hosting heavy files, like podcast files, for example. Is there a Soundcloud clone on IPFS anywhere? (One that doesn't provide you with cloud storage, only the public index)
  - magnetic 8y ago
    Is there a way (even not free) to ensure that my data in IPFS is always available (at least as available as with "traditional hosting services)? That's a bit of a requirement when building a serious service/website.
    - 0x8BADF00D 8y ago
      CAP theorem.
      - magnetic 8y ago
        Perhaps you can elaborate?
      - ric2b 8y ago
        What we're talking about only requires the A and the P.
    - wiml 8y ago
      As I understand it, the way is: run an IPFS node (perhaps on a cheap VPS); use `ipfs pin ` to force the node to retain the file even if it isn't being requested. Presumably someone could sell "pinning as a service" for fairly cheap: a popular file will be widely cached, so the service will rarely need to provide it; and an unpopular file will be rarely requested.
      - magnetic 8y ago
        Isn't pinning as a service essentially a CDN? Is there a reason to believe that such service would be cheaper than a "traditional" CDN?
- str33t_punk 8y ago
  Can you? You will still need to run a node to host your own content on IPFS. Unless you just simply plan on using data that already exists in the wild
jgrahamc 8y ago
This is my current favourite use case (from the blog https://blog.cloudflare.com/e2e-integrity/ https://blog.cloudflare.com/e2e-integrity/) I wanted to provide an example of the kinds of secure, performant applications that are possible with IPFS, and this made building a search engine seem like a prime candidate. Rather than steal Protocol Labs' idea of 'Wikipedia on IPFS', we decided to take the Kiwix archives of all the different StackExchange websites and build a distributed search engine on top of that. You can play with the finished product here: https://ipfs-sec.stackexchange.cloudflare-ipfs.com https://ipfs-sec.stackexchange.cloudflare-ipfs.com
- nathcd 8y ago
  Neat! Looks like this generally works by building a static index and publishing it to IPFS [1], then having some client-side JS do a lookup in the index, and getting a metadata file for each result [2]? I made something kinda similar a while ago [3], where I gathered a bunch of metadata about files on IPFS at the time by scraping a Searx instance, with a query like "site:ipfs.io/ipfs" or something like that. It was a quick hack job but was fun, and it's cool to see something similar on a bigger scale! [1] like https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index/tokens https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index... [2] like https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index/d/186 https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index... [3] https://ipfs.io/ipfs/QmYo5ZWqNW4ib1Ck4zdm6EKteX3zZWw1j4CVfKtnAzNdvu/ https://ipfs.io/ipfs/QmYo5ZWqNW4ib1Ck4zdm6EKteX3zZWw1j4CVfKt..., https://github.com/doesntgolf/ipfs-search https://github.com/doesntgolf/ipfs-search, https://github.com/doesntgolf/ipfs-searx-scraper https://github.com/doesntgolf/ipfs-searx-scraper
  - lgierth 8y ago
    Yeah! Every question of "how would X work on IPFS?" boils down to data structures. An example of slightly different approach (a sharded index) is the wikipedia-on-ipfs experiment from last year: https://github.com/magik6k/distributed-wiki-search https://github.com/magik6k/distributed-wiki-search && https://github.com/ipfs/distributed-wikipedia-mirror https://github.com/ipfs/distributed-wikipedia-mirror
  - 8y ago
kpcyrd 8y ago
I really didn't expect this to happen, congrats to the ipfs team! A demo link over here: https://cloudflare-ipfs.com/ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv https://cloudflare-ipfs.com/ipfs/QmS4ustL54uo8FzR9455qaxZwuM...
dark_glass 8y ago
This seems very similar to Freenet. The only difference I see at a glance is that Cloudflare is running gateways so that everyone does not have to run a node.
- kpcyrd 8y ago
  On freenet other people are sending you files to store, on ipfs you only store your own files, files pinned (basically seeding) and files from your cache. Freenet is built for anonymity, while ipfs is built for performance.
  - Dylan16807 8y ago
    You can simplify freenet's behavior to storing only cached files, though.
souterrain 8y ago
This is cool from an implementation standpoint. However, doesn't this defeat the purpose of using IPFS? Using Cloudflare to front-end content stored within IPFS makes Cloudflare the choke point for all traffic, effectively re-centralizing the distributed content.
- detaro 8y ago
  You'll need a "centralized" version of a web site for those that don't use IPFS today. That's what a gateway provides. Clients using the IPFS protocol will ignore them and fetch the distributed version.
  - dboreham 8y ago
    >Clients using the IPFS protocol will ignore them and fetch the distributed version Except they won't know the hash for the content they're looking for..
    - detaro 8y ago
      Clients and gateways use the same DNS entries to know that.
- jgrahamc 8y ago
  It doesn't change IPFS fundamentally, it just provides a simple way to get to IPFS content and a place where it gets cached. We hope it help legitimize IPFS.
  - marknadal 8y ago
    Hey, thanks for your work! This is an exciting start to the dWeb! Any chance you guys would also support DAT, GUN, WebTorrent, and SSB? I know Feross, Dominic, Mathias, etc. would love to introduce you to them. Ping me? mark@gun.eco
- patrickg_zill 8y ago
  Yes and cloudflare has already demonstrated both their ability and willingness to de-platform. See https://news.ycombinator.com/item?id=15031922 https://news.ycombinator.com/item?id=15031922
r1ch 8y ago
I'm a bit worried about the abuse potential here. It seems like a great way to distribute movies, warez, etc since not only is Cloudflare paying for the bandwidth but they're also caching the content so you won't be bottlenecked by IPFS itself.
- jgrahamc 8y ago
  From the article: IPFS is a peer-to-peer network, so there is the possibility of users sharing abusive content. This is not something we support or condone. However, just like how Cloudflare works with more traditional customers, Cloudflare’s IPFS gateway is simply a cache in front of IPFS. Cloudflare does not have the ability the modify or remove content from the IPFS network. If any abusive content is found that is served by the Cloudflare IPFS gateway, you can use the standard abuse reporting mechanism described here.
  - r1ch 8y ago
    "Cloudflare will forward abuse reports that appear to be substantially complete to the responsible website hosting provider and to the website owner. In response to a substantially complete abuse report, Cloudflare will provide the complainant with the contact information for the responsible website hosting provider so they can be contacted directly." So where does this abuse report end up?
    - yarrel 8y ago
      Where do abuse reports usually end up? In my experience they end up with the DMCA officer for the site.
    - jstanley 8y ago
      The best you can do as an IPFS gateway operator is block particular hashes. I have had to do this when a botnet was fetching its code from my public gateway and accidentally DDoS'ing my server :).
    - xxdesmus 8y ago
      For non-IPFS -- It depends on the notification settings selected by the complainant, but generally speaking we notify the website owner based on the Cloudflare account email address, and notify the hosting provider of the website. For IPFS -- there isn't an "owner" of the content in IPFS per se, and as such we will review the specifics of the report and then determine the appropriate next steps.
aclelland 8y ago
This looks great and will hopefully help IPFS grow! I do wonder if there are any limits which CF are going to impose on these files? A few years ago there was an image hosting site [1] who was told by CF that they were using too much bandwidth on the free plan. With this gateway, can't anyone start doing exactly the same and not even have to be a CF customer? [1] https://news.ycombinator.com/item?id=12825719 https://news.ycombinator.com/item?id=12825719
javagram 8y ago
IPFS seems like torrents. The post doesn’t really address how abuse will be handled beyond blocking access to a specific hash from cloudflare’s gateway. Illegal files can and will be uploaded to IPFS, why would anyone run a node knowing they might be hosting child porn or other illegal content that could land them in jail?
- deleted 8y ago
  [deleted]
- freeone3000 8y ago
  You run a node because you have data to share. You're only sharing stuff you've already downloaded. Don't want to share illegal stuff? Don't download it.
- ABS 8y ago
  that's not how IPFS works: if you haven't downloaded anything illegal via IPFS you won't have anything illegal in your node
  - javagram 8y ago
    I see. Thanks. I found a faq on this, looks like using additional stuff on top of IPFS would require deny lists, but IPFS itself doesn’t. “Q: Will i store other people's stuff? A: No, by default IPFS will not download anything your node doesn't explicitly ask for. This is a strict design constraint. In order to build group archiving, and faster distribution, protocols are layered on top that may download content for the network, but these are optional and built on top of basic IPFS. Examples include bitswap agents, ipfs-cluster, and Filecoin. Q: but bitswap says it may download stuff for others, to do better? A: yes, this is an extension of bitswap, not implemented yet, and will be either opt-in, or easy to opt-out and following the denylists (to avoid downloading bad bits).”
  - crankylinuxuser 8y ago
    Technically, the original paper does call for ahead-of-time sharing of popular datablocks. SO for example, if piracy caught on quick here, then this functionality could push pirated data through. Even though this was discussed in the initial paper, it was never implemented.
ClassAndBurn 8y ago
Cloudflare is great. I love that they spend resources on projects like this. This would normally be a Moonshot or someone's side projects that would get limited/no release.
robotmay 8y ago
Any plans to support Dat in the future too? I've had a bunch of distributed projects in mind recently, and I've still not decided which platform to start experimenting with properly, but Dat is definitely one of the other interesting ones alongside IPFS.
- bren2013 8y ago
  No plans currently. I'm not really familiar with Dat. Can it do something that IPFS can't?
  - marknadal 8y ago
    Yes, IPFS is purely hash based which is great for static images/movies. DAT is also hash based, but at least it has support for top-level asymmetric keys that you can put files into, and ADD files to without the root directory changing its hash. IPNS works around this, but isn't ideal. Neither systems can handle high frequency P2P data that changes - for instance Reddit-like sites etc. those ( http://notabug.io http://notabug.io ) and other sites like the Internet Archive (which has IPFS, WebTorrent, and GUN versions decentralized) are built on our system (https://github.com/amark/gun https://github.com/amark/gun), are already pushing terabytes of P2P traffic. And don't forget about WebTorrent and Secure Scuttlebutt!!!
    - bren2013 8y ago
      Not sure if this fully addresses your use-case, but I like the idea of serving a static bootloader from IPFS. The bootloader would contain all of a website's assets, and code for getting dynamic content from a backend. The backend could be: - A central API where the bootloader can do arbitrary validation on the API responses. - WebTorrent, Scuttlebutt, IPFS PubSub, etc.
      - marknadal 8y ago
        Yes, that is already what the P2P Reddit does, but without IPFS (although this is a good idea!), and using GUN as the "backend" (fully P2P/decentralized though), SEA for validation/security (no need for a central one), and DAM for pubsub (no flooding problems like in libp2p) which can do WebRTC. I'm sure people would love to see an IPFS version of a bootloader, instead of HTTP, that is a cool idea. Have a repo for it?
komali2 8y ago
Somehow, this is the first time I've heard of IPFS. Seems really awesome! From a total novice standpoint, can someone help me understand: >With IPFS, every single block of data stored in the system is addressed by a cryptographic hash of its contents, i.e., a long string of letters and numbers that is unique to that block. When you want a piece of data in IPFS, you request it by its hash. So rather than asking the network “get me the content stored at 93.184.216.34,” you ask “get me the content that has a hash value of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy.” It seems like you must know the "URL" of the "website" you want to visit (files in IPFS) beforehand? But in the case of IPFS, there's no like, DNS service, so you can't type "www.google.com." Basically, it'd be like if to navigate the modern internet, you'd need to know the IP address of whatever site you visit? Is that true of IPFS? Is there any way around that? It seems like a strong limitation, unless someone can make some sort of IPFS search engine that happens to hash out at QN000000000000000000000 or some really memorizable hash... which seems extremely unlikely!
- ComNik 8y ago
  Check out the white paper at https://github.com/ipfs/papers/raw/master/ipfs-cap2pfs/ipfs-p2p-file-system.pdf https://github.com/ipfs/papers/raw/master/ipfs-cap2pfs/ipfs-..., section 3.7. There is a name system called IPNS.
- wongarsu 8y ago
  I don't know how IPFS handles this currently, but TOR has the same problem (there addresses are basically a hash of the server certificate). The TOR community handles this by widely advertising the addresses of a few entry points. From there you use classic link traversal and bookmarks. In general, bookmarks become extremely important once you can't memorize addresses.
- hanklazard 8y ago
  This FAQ from github is a helpful summary of human-readable IPFS via IPNS: https://github.com/ipfs/faq/issues/16#issuecomment-232497229 https://github.com/ipfs/faq/issues/16#issuecomment-232497229
- Communitivity 8y ago
yarrel 8y ago
Censorious centralization platform embraces censorship-resistant decentralized platform.
- snaky 8y ago
  That's the way we've got modern web, where decentralized web sites need the Google Search as entrypoint, decentralized blogs morphed into Facebook, and Cloudflare is going to monopolize the transport layer of all that.
th0ma5 8y ago
How does IPFS handle Denial of Service?
- diggan 8y ago
  Can you expand on what kind of DoS you're thinking about? Files added to IPFS are stored locally (not pushed to other peers), up until some other node asks about it and/or tries to actually fetch it.
  - th0ma5 8y ago
    Yeah I guess are these discoverable or guessable? Could someone request my files at a rate I couldn't supply?
joshfraser 8y ago
This is awesome. We've been running an IPFS gateway at Origin for about a year and are huge fans of the technology. It's nice to see Cloudflare investing in decentralized technology despite contributing to the over-centralization of the web with their core business. There are quite a few public IPFS gateways available, but not all of them are broadly advertised. A few others that I know of: ipfs.io ipfs.infura.io siderus.io ipfs.jes.xxx gateway.originprotocol.com
- diggan 8y ago
  I just went through our "Public IPFS Gateway Checker" and added a bunch of new ones (include the Cloudflare) one. You can see it running here: https://ipfs.github.io/public-gateway-checker/ https://ipfs.github.io/public-gateway-checker/ And of course, the source code: https://github.com/ipfs/public-gateway-checker/ https://github.com/ipfs/public-gateway-checker/
  - joshfraser 8y ago
    Oh neat. Should I send a PR to add the others?
    - diggan 8y ago
      Please do :)
- burdakovd 8y ago
  Just want to add an alternative gateway that I've been working on recently: dapps.earth It is somewhat different in the sense that it redirects from `gateway/ipfs/hash` to `hash.ipfs.gateway` to ensure different ipfs websites are hosted at different origins (Cloudflare also talked about origin issues in their announcement). Now with what Cloudflare release, they do seem to solve some of the similar problems, but they still ask users to register the domains by themselves (or access in unsafe way with `/ipfs/hash` urls), which means those domains become a centralization point, and also most of the content is served from the same domain. It would be fantastic if they allowed access in form `hash.cloudflare-ipfs.com` or `ens-name.cloudflare-ipfs.com`. In the meantime I may just abandon the idea of serving ipfs content myself, and instead can just operate a small DNS-server that will return CNAME record pointing to Cloudflare and TXT record pointing to ipfs hash for each subdomain.
yoava 8y ago
Being from wix and managing petabytes of media files, we always wondered if a distributed web solution for websites can actually work. Even with the best of clouds (more then one of the big names) we faced downtimes caused by multiple cloud failures. Just imagine, a store that is Geo distributed, resilient and just works. This is super exciting for the web and for customers of websites!!! Big thumb up.
- dgreensp 8y ago
  Is IPFS really a store, let alone a store with any particular performance or availability properties? If I want to be sure a file is available, don’t I have to host it or pay someone to host it? And if I want to make sure it is geo distributed, I have to geo distribute it or pay a service to? What is the big change?
  - wongarsu 8y ago
    Wit IPFS you could rent a few cheap storage servers at different providers around the world and have them all pin your content. That should be resilient against most classes of failures (unlike a cloud provider where bad code pushed to production by the cloud hoster can cause failures in all availability regions). It's not exactly the first protocol enabling that, but IPFS also has the benefit of distributing bandwidth if your clients all access the files via IPFS (since they will serve them from their cache).
    - dgreensp 8y ago
      I see, so geo distribution could be as simple as making the file available in multiple locations, with IPFS doing the rest.
  - progval 8y ago
    IPFS is not a store, it's a protocol. You still have to pay someone to host it.
orliesaurus 8y ago
Total noob questions, would love some answers if anyone has them. Here it says: "IPFS is a peer-to-peer file system composed of thousands of computers around the world, each of which stores files on behalf of the network. These files can be anything: cat pictures, 3D models, or even entire websites" - If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff? Then it says: "The content with a hash of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy could be stored on dozens of nodes, so if one node that was caching that content goes down, the network will just look for the content on another node." - How long does it take for 1 specific file to be "re-distributed" on another node? and how many times is a file rehosted/distributed on a different node? It would be stupid to have the same file hosted 1.000.000 times - what's the break off point? Is there like a healthcheck so to speak which ensures that a certain file is in danger of disappearing from the network and therefore is automatically pushed for re-distribution And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them? But hold up, isn't p2p file system like this all about trusting the network and not 1 server?
- diggan 8y ago
  The IPFS network works in a pull-model, not push. So when nodes add content, they only add it locally. It's not until some other nodes request it, as it actually transfers. So if you start a IPFS node locally, it won't get content pushed to it (well, some DHT data which is basically routing information, but not the content itself). > ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently (maybe in the future?). So, download file A and add it to IPFS from your end. If you end up with the same hash, the content was not modified. > isn't p2p file system like this all about trusting the network and not 1 server Yes, IPFS <> IPFS is trustless as this verification happens automatically and the clients can verify content locally when it gets fetched. IPFS <> HTTP is harder as there is no functionality for doing this verification.
antpls 8y ago
That's a nice initiative. BitTorrent files are also content addressable, and the DHT allows for routing to several provider nodes. Only files that are downloaded are shared too. What's the plus-value of IPFS over BitTorrent in this scenario?
- lgierth 8y ago
  Torrents don't have the concept of links at all, they're not a web. Also: Being able to inter-link any content-addressed data structures. With IPLD [1], you can link from an IPFS directory to a Bitcoin transaction, to a Git repository, to a Torrent. [1] https://ipld.io https://ipld.io && https://github.com/ipld/ipld#implementations https://github.com/ipld/ipld#implementations
  - antpls 8y ago
    From what I understand from the ipld.io website, IPLD allows to add links to any content-adressable systems. BitTorrent + IPLD could be an alternative to IPFS, I guess? (BTW, notice that BitTorrent isnt used as an example on the front page). When I looked at IPFS some months ago, it looked like a lot of on-going work. It wasn't clear to me what was new and what were the reusable components of the project, and what was reused from the past. In BitTorrent, the DHT for example is a reusable block emerged from previous P2P systems, but it didn't try to reinvent the wheel, and it didn't try to add a FS on top of it, so it solves a specific problem. "IPFS" does a lot of things at once, it would be clearer if the project was cut into sub-specifications that could be reused with legacy protocols. Tho I believe the project is going into that direction, and for example IPLD emerged from it. IPLD looks like a real step-forward. It looks like a "federation" of previous content-addressable protocols, to make them work together, rather than trying to compete with them. BitTorrent + IPLD makes more sense than IPFS to me, more developers have a heard of BitTorrent and they knows what are BitTorrent features, so the introduction of a concept on top of it is easier to grasp.
  - imhoguy 8y ago
    > Torrents don't have the concept of links at all, they're not a web. BitTorrent has content-addressable links: https://en.wikipedia.org/wiki/Magnet_URI_scheme https://en.wikipedia.org/wiki/Magnet_URI_scheme You can perfectly form a web with BitTorrent network by sharing e.g. HTML file which links to further resources.
zapita 8y ago
Is Cloudflare doing this in partnership with Protocol Labs, the company backing IPFS? Or are they doing it on their own? I remember that the way Cloudflare communicated around Nginx rubbed a lot of people the wrong way, because they aggressively positioned themselves as "the de-facto Nginx company", to the point where they would sometimes announce a new Nginx feature on their blog before the Nginx developers themselves had a chance to announce it... And of course nobody was more pissed off than Nginx.com, the actual official sponsor of Nginx. I'm wondering if they will behave the same way with Protocol Labs and the existing IPFS community? I sure hope not, I like Cloudflare and love IPFS, and would like to see everyone collaboratig in good spirits, and not competing for cheap marketing points.
headcrack 8y ago
...and the devil introduces an own church. IPFS is about decentralisation! And Cloudflare is the devil of centralization!
- dboreham 8y ago
  If you analyze any decentralized system long enough you'll discover that it can't be used in practice by regular end-users without some element of centralization. (Dboreham's Law)
  - headcrack 8y ago
    Central system Google does not know anything about any Dboreham's "Law".
- auslander 8y ago
  ... and with access to plaintext, SSL stripped data of all traffic
  - headcrack 8y ago
    Exactly, it becomes more and more obvious, cloudflare's role in intelligence apparatus becomes bigger.
    - bren2013 8y ago
      IPFS offers content integrity, which you can use to bootstrap confidentiality.
      - headcrack 8y ago
        Confidentiality without central gateways. The topic was, "with stripped down SSL" and evey user should be aware of, cloudflare can read and store every content of every connection made to a domain connected to cloudflare gateways even if they seem to be end-to-end-encrypted.
- 8y ago
llama052 8y ago
This kind of reminds me of usenet, in a good way. It will be interesting to see what kind of uses this has over time, or if it will be something sustainable if it gains a lot of traction. Good stuff!
brtknr 8y ago
Does anyone know the answer to whether large content that is widely available becomes quicker to download similar to peers in BitTorrent?
- lgierth 8y ago
  Yes it does -- IPFS will fetch from multiple peers at the same time. It's not terribly optimized yet for finding even more peers for content if it's already fetching from some, but it already does generally fetch from multiple at the same time.
frio 8y ago
This is excellent. I've currently got a hacked together static page for my personal site, which is fronted by CloudFlare, but forwards the actual request onto ipfs.io (which, in turn, serve the files -- ultimately -- from where I've pinned them on eternum). This will let me take a step out of that chain.
timwis 8y ago
Nicely written for more general audiences! Big news :) Congrats Juan, Matt, Michelle, and team if you see this!
loeg 8y ago
> Cloudflare’s IPFS gateway is simply a cache in front of IPFS. Cloudflare does not have the ability to modify or remove content from the IPFS network. If any abusive content is found that is served by the Cloudflare IPFS gateway, you can use the standard abuse reporting mechanism described here. That's an interesting legal hypothesis for when they are prosecuted for serving kiddie porn, isn't it. Seems like they're taking on all of same types of likely abuse as running a Tor exit node. At a minimum it seems like they will have to blacklist serving known abuse.
- X-Cubed 8y ago
  I think what they're trying to say here is that they may be able to hide/block it, but they can't remove it outright. The same applies for most of their proxying services, they don't have access to modify the origin.
- deleted 8y ago
  [deleted]
- mirimir 8y ago
  I expect that they'll blacklist anything iffy. The ipfs.io does, so hey. But you can run your own ipfs demon, and gateway. Even as a tor .onion service.
andrewchambers 8y ago
semi self hosted pastebin alternative with annotations :) (You may need to disable adblock) https://genius.it/cloudflare-ipfs.com/ipfs/Qmc8qNo5adHBnLGSJUCw1WL2gBDyzjQ5NBDWiSHTetbDJQ https://genius.it/cloudflare-ipfs.com/ipfs/Qmc8qNo5adHBnLGSJ...
ezoe 8y ago
After quickly glancing the IPFS document, I can't find the superiority over... say, BitTorrent with DHT and Magnet link(no tracker server or torrent file required.) Save for data deduplication and IPNS maybe. And isn't it rather contradictory? A gateway for P2P-based filesystem.
- lgierth 8y ago
  You can't easily interlink torrents in way that torrent clients could understand -- torrents are mostly just packages of data, not made to be a web. On your point about the gateway -- I've co-maintained the ipfs.io gateway for the last 3 years and my perspective on its importance is the upgrade path from the existing systems. When developing and deploying new protocols, you need to onboard both users and applications/websites, and the most effective way to do so is seamless interoperability between old-web and new-web. I would imagine it tricky to get any critical mass of adoption if everyone has to choose either old-web or new-web.
  - ezoe 8y ago
    But as I understand, BitTorrent can express directory structure. So the IPFS features like mounting by FUSE or browsing through web browser via local server can be implemented as the frontend of BitTorrent too. Currently, there are tons of independently developed BitTorrent protocol implementations while IPFS has very few implementations from the same body. You can't beat the existing software by slightly better implementations. Apart from probably IPNS, most IPFS features can be achieved by BitTorrent frontend too. As for the gateway. If somebody want a gateway for P2P network, that's because the same P2P network is inefficient or difficult to use. If the P2P network is difficult to use, not much people run their own node. Rather, they relies on somebody else's node. We see the bad consequence of this number of times in the crypto-currency. In order to trust the P2P crypto-currenncy, It's essential that you run your own node for your wallet from the computer you own and trust. But the reality is, it's pretty inconvenient or difficult to do for most of the users. So what happened was, very few big central node which manage the wallet for you, exchange between different currencies for you. At this point, this is not a P2P crypto-currency. The big central node become the single point of failure. If it goes down or breached, you're doomed. For the healthy P2P network that can replace the existing central-authority network of today, all users must run their own node and it must be very very easy to do so. So there should no demand for the gateway.
blacksoil 8y ago
The part I'm still confused is: how does the cache being updated? For example in the traditional web architecture if I go to www.example.com/index.htm, the host server of example.com tells the hash of index.htm and depending on that my web browser decides to use its cache or do a fresh request. How would this work in IPFS? How about dynamic pages? Does that mean my browser still has to contact www.example.com to get the latest version's hash but then has the option to request the file from IPFS instead of www.example.com? What about if example.com goes down?
- e12e 8y ago
  In basic ipfs the address is content-dependent hash - same url should always return same content (excepting hash collisions which should be so rare and improbable as to never occur in practice). So the short answer is that there's nothing to update: and url will return the same content (or no content if no one has it pinned anymore). AFAIK the ipfs approach to this conundrum is a name/resolver system - which essentially adds indirection. This is Thomas he'll always be Thomas. But for now, he's also class valedictorian. Some time in the future, Thomas will still be Thomas, but someone else might be class valedictorian.
Tetris1 8y ago
File sharing community should adopt and thrive on this technology. Viva!
shalabhc 8y ago
Is there a way to uniformly represent links and relationships between IPFS files in IPFS itself?
- diggan 8y ago
  Yes, that's what IPLD (https://ipld.io/ https://ipld.io/) is for and more. It's used in IPFS to do links between files in IPFS and also links between files and other formats like Git or Ethereum transactions. You can see the data structure itself if you use the `ipfs object` commands: $ tree test-directory test-directory ├── directory-a │ └── file-b └── file-a 1 directory, 2 files $ ipfs add -r test-directory added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH test-directory/directory-a/file-b added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH test-directory/file-a added QmVSi16sFvaKxJXGHWj9yCVcbxJBLRePBARKt1KUyWMGAK test-directory/directory-a added Qmbm9JEWeD8EX4pQUYpx4eDsjqyzGjQMdGzdJEdwJvcHb4 test-directory $ ipfs object get Qmbm9JEWeD8EX4pQUYpx4eDsjqyzGjQMdGzdJEdwJvcHb4 {"Links":[{"Name":"directory-a","Hash":"QmVSi16sFvaKxJXGHWj9yCVcbxJBLRePBARKt1KUyWMGAK","Size":58},{"Name":"file-a","Hash":"QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH","Size":6}],"Data":"\u0008\u0001"} $ ipfs object get QmVSi16sFvaKxJXGHWj9yCVcbxJBLRePBARKt1KUyWMGAK {"Links":[{"Name":"file-b","Hash":"QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH","Size":6}],"Data":"\u0008\u0001"}

621 points by jgrahamc 8y ago

Here is one really important part that I almost missed when reading this announcement: > Using Cloudflare's gateway, you can also build a website that’s hosted entirely on IPFS, but still available to your users at a custom domain name. This is huge step forward for the distributed web!

koalalorenzo 8y ago
You can use any gateway to do that :)
- toomuchtodo 8y ago
  Cloudflare offering it might legitimize IPFS in ways that non-corporate entities championing it doesn't. Not a huge Cloudflare fan, but I will admit this is cool of them to spend engineering time and money on. EDIT: Thanks for the efforts to promote the Distributed Web. A surprise, to be sure, but a welcome one.
  - jgrahamc 8y ago
    That's the whole point. We think the Distributed Web is cool and coming and we want to accelerate that happening. Hence the time and effort to make this happen.
    - deleted 8y ago
      [deleted]
    - fold_left 8y ago
      Bravo
    - carapace 8y ago
      I was just wondering the other day if IPFS would get enough traction to reach critical mass. This is a very hopeful sign I think. Cheers!
      - steven_pack 8y ago
        That was one of the drivers. We liked the core technology, but could see there was a significant barrier to actually using it. Being able to point your domain at IPFS content and then providing that (ssl secured) site to the masses is a big step forward.
    - mtgx 8y ago
      A most welcome feature.
    - nil_pointer 8y ago
      That is wonderful to hear
- styfle 8y ago
  You can use any gateway with your custom domain name? Potentially you could but I don't know of any that let you do this. My impression of the public gateways was that the end user would need to know the content hash of your website which makes for a poor user experience. This announcement claims the end user just uses the website as they normally would by visiting example.com but behind the scenes there is no central web server because the content is pulled from IPFS. DNS was the missing piece :) https://developers.cloudflare.com/distributed-web/ipfs-gateway/connecting-website/ https://developers.cloudflare.com/distributed-web/ipfs-gatew...
  - lgierth 8y ago
    You can use custom domains with any go-ipfs gateway, it'll transparently turn `Host: example.com` into `/ipns/example.com`. What Cloudflare adds to the mix is their infrastructure for automatic TLS certs.
  - marius_k 8y ago
    I was running my own website with my own domain on ipfs quite successfully for few months. And I have configured cloudflare just to have a green https icon. The only problem is that I have to visit my website once in a while, otherwise the content is deleted on the gateway.
- andrewxhill 8y ago
  We wrote a tutorial on setting it up on your own (secure nginx server for ipfs gateway), https://medium.com/textileio/tutorial-setting-up-an-ipfs-peer-part-ii-67a99cd2c5 https://medium.com/textileio/tutorial-setting-up-an-ipfs-pee.... But for sure, for most users, going right through cloudflare is an awesome simplification. Exciting stuff.
fiatjaf 8y ago
The killer app for IPFS in this early days should be people who are paying $10/mo to host their half-dozen HTML files on WhateverHost companies everywhere. Bonus points if they are hosting heavy files, like podcast files, for example. Is there a Soundcloud clone on IPFS anywhere? (One that doesn't provide you with cloud storage, only the public index)
- magnetic 8y ago
  Is there a way (even not free) to ensure that my data in IPFS is always available (at least as available as with "traditional hosting services)? That's a bit of a requirement when building a serious service/website.
  - 0x8BADF00D 8y ago
    CAP theorem.
    - magnetic 8y ago
      Perhaps you can elaborate?
    - ric2b 8y ago
      What we're talking about only requires the A and the P.
  - wiml 8y ago
    As I understand it, the way is: run an IPFS node (perhaps on a cheap VPS); use `ipfs pin ` to force the node to retain the file even if it isn't being requested. Presumably someone could sell "pinning as a service" for fairly cheap: a popular file will be widely cached, so the service will rarely need to provide it; and an unpopular file will be rarely requested.
    - magnetic 8y ago
      Isn't pinning as a service essentially a CDN? Is there a reason to believe that such service would be cheaper than a "traditional" CDN?
str33t_punk 8y ago
Can you? You will still need to run a node to host your own content on IPFS. Unless you just simply plan on using data that already exists in the wild

jgrahamc 8y ago

This is my current favourite use case (from the blog https://blog.cloudflare.com/e2e-integrity/ https://blog.cloudflare.com/e2e-integrity/) I wanted to provide an example of the kinds of secure, performant applications that are possible with IPFS, and this made building a search engine seem like a prime candidate. Rather than steal Protocol Labs' idea of 'Wikipedia on IPFS', we decided to take the Kiwix archives of all the different StackExchange websites and build a distributed search engine on top of that. You can play with the finished product here: https://ipfs-sec.stackexchange.cloudflare-ipfs.com https://ipfs-sec.stackexchange.cloudflare-ipfs.com

nathcd 8y ago
Neat! Looks like this generally works by building a static index and publishing it to IPFS [1], then having some client-side JS do a lookup in the index, and getting a metadata file for each result [2]? I made something kinda similar a while ago [3], where I gathered a bunch of metadata about files on IPFS at the time by scraping a Searx instance, with a query like "site:ipfs.io/ipfs" or something like that. It was a quick hack job but was fun, and it's cool to see something similar on a bigger scale! [1] like https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index/tokens https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index... [2] like https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index/d/186 https://ipfs-sec.stackexchange.cloudflare-ipfs.com/ai/_index... [3] https://ipfs.io/ipfs/QmYo5ZWqNW4ib1Ck4zdm6EKteX3zZWw1j4CVfKtnAzNdvu/ https://ipfs.io/ipfs/QmYo5ZWqNW4ib1Ck4zdm6EKteX3zZWw1j4CVfKt..., https://github.com/doesntgolf/ipfs-search https://github.com/doesntgolf/ipfs-search, https://github.com/doesntgolf/ipfs-searx-scraper https://github.com/doesntgolf/ipfs-searx-scraper
- lgierth 8y ago
  Yeah! Every question of "how would X work on IPFS?" boils down to data structures. An example of slightly different approach (a sharded index) is the wikipedia-on-ipfs experiment from last year: https://github.com/magik6k/distributed-wiki-search https://github.com/magik6k/distributed-wiki-search && https://github.com/ipfs/distributed-wikipedia-mirror https://github.com/ipfs/distributed-wikipedia-mirror
- 8y ago

kpcyrd 8y ago

I really didn't expect this to happen, congrats to the ipfs team! A demo link over here: https://cloudflare-ipfs.com/ipfs/QmS4ustL54uo8FzR9455qaxZwuMiUhyvMcX9Ba8nUH4uVv https://cloudflare-ipfs.com/ipfs/QmS4ustL54uo8FzR9455qaxZwuM...

dark_glass 8y ago

This seems very similar to Freenet. The only difference I see at a glance is that Cloudflare is running gateways so that everyone does not have to run a node.

kpcyrd 8y ago
On freenet other people are sending you files to store, on ipfs you only store your own files, files pinned (basically seeding) and files from your cache. Freenet is built for anonymity, while ipfs is built for performance.
- Dylan16807 8y ago
  You can simplify freenet's behavior to storing only cached files, though.

souterrain 8y ago

This is cool from an implementation standpoint. However, doesn't this defeat the purpose of using IPFS? Using Cloudflare to front-end content stored within IPFS makes Cloudflare the choke point for all traffic, effectively re-centralizing the distributed content.

detaro 8y ago
You'll need a "centralized" version of a web site for those that don't use IPFS today. That's what a gateway provides. Clients using the IPFS protocol will ignore them and fetch the distributed version.
- dboreham 8y ago
  >Clients using the IPFS protocol will ignore them and fetch the distributed version Except they won't know the hash for the content they're looking for..
  - detaro 8y ago
    Clients and gateways use the same DNS entries to know that.
jgrahamc 8y ago
It doesn't change IPFS fundamentally, it just provides a simple way to get to IPFS content and a place where it gets cached. We hope it help legitimize IPFS.
- marknadal 8y ago
  Hey, thanks for your work! This is an exciting start to the dWeb! Any chance you guys would also support DAT, GUN, WebTorrent, and SSB? I know Feross, Dominic, Mathias, etc. would love to introduce you to them. Ping me? mark@gun.eco
patrickg_zill 8y ago
Yes and cloudflare has already demonstrated both their ability and willingness to de-platform. See https://news.ycombinator.com/item?id=15031922 https://news.ycombinator.com/item?id=15031922

r1ch 8y ago

I'm a bit worried about the abuse potential here. It seems like a great way to distribute movies, warez, etc since not only is Cloudflare paying for the bandwidth but they're also caching the content so you won't be bottlenecked by IPFS itself.

jgrahamc 8y ago
From the article: IPFS is a peer-to-peer network, so there is the possibility of users sharing abusive content. This is not something we support or condone. However, just like how Cloudflare works with more traditional customers, Cloudflare’s IPFS gateway is simply a cache in front of IPFS. Cloudflare does not have the ability the modify or remove content from the IPFS network. If any abusive content is found that is served by the Cloudflare IPFS gateway, you can use the standard abuse reporting mechanism described here.
- r1ch 8y ago
  "Cloudflare will forward abuse reports that appear to be substantially complete to the responsible website hosting provider and to the website owner. In response to a substantially complete abuse report, Cloudflare will provide the complainant with the contact information for the responsible website hosting provider so they can be contacted directly." So where does this abuse report end up?
  - yarrel 8y ago
    Where do abuse reports usually end up? In my experience they end up with the DMCA officer for the site.
  - jstanley 8y ago
    The best you can do as an IPFS gateway operator is block particular hashes. I have had to do this when a botnet was fetching its code from my public gateway and accidentally DDoS'ing my server :).
  - xxdesmus 8y ago
    For non-IPFS -- It depends on the notification settings selected by the complainant, but generally speaking we notify the website owner based on the Cloudflare account email address, and notify the hosting provider of the website. For IPFS -- there isn't an "owner" of the content in IPFS per se, and as such we will review the specifics of the report and then determine the appropriate next steps.

aclelland 8y ago

This looks great and will hopefully help IPFS grow! I do wonder if there are any limits which CF are going to impose on these files? A few years ago there was an image hosting site [1] who was told by CF that they were using too much bandwidth on the free plan. With this gateway, can't anyone start doing exactly the same and not even have to be a CF customer? [1] https://news.ycombinator.com/item?id=12825719 https://news.ycombinator.com/item?id=12825719

javagram 8y ago

IPFS seems like torrents. The post doesn’t really address how abuse will be handled beyond blocking access to a specific hash from cloudflare’s gateway. Illegal files can and will be uploaded to IPFS, why would anyone run a node knowing they might be hosting child porn or other illegal content that could land them in jail?

deleted 8y ago
[deleted]
freeone3000 8y ago
You run a node because you have data to share. You're only sharing stuff you've already downloaded. Don't want to share illegal stuff? Don't download it.
ABS 8y ago
that's not how IPFS works: if you haven't downloaded anything illegal via IPFS you won't have anything illegal in your node
- javagram 8y ago
  I see. Thanks. I found a faq on this, looks like using additional stuff on top of IPFS would require deny lists, but IPFS itself doesn’t. “Q: Will i store other people's stuff? A: No, by default IPFS will not download anything your node doesn't explicitly ask for. This is a strict design constraint. In order to build group archiving, and faster distribution, protocols are layered on top that may download content for the network, but these are optional and built on top of basic IPFS. Examples include bitswap agents, ipfs-cluster, and Filecoin. Q: but bitswap says it may download stuff for others, to do better? A: yes, this is an extension of bitswap, not implemented yet, and will be either opt-in, or easy to opt-out and following the denylists (to avoid downloading bad bits).”
- crankylinuxuser 8y ago
  Technically, the original paper does call for ahead-of-time sharing of popular datablocks. SO for example, if piracy caught on quick here, then this functionality could push pirated data through. Even though this was discussed in the initial paper, it was never implemented.

ClassAndBurn 8y ago

Cloudflare is great. I love that they spend resources on projects like this. This would normally be a Moonshot or someone's side projects that would get limited/no release.

robotmay 8y ago

Any plans to support Dat in the future too? I've had a bunch of distributed projects in mind recently, and I've still not decided which platform to start experimenting with properly, but Dat is definitely one of the other interesting ones alongside IPFS.

bren2013 8y ago
No plans currently. I'm not really familiar with Dat. Can it do something that IPFS can't?
- marknadal 8y ago
  Yes, IPFS is purely hash based which is great for static images/movies. DAT is also hash based, but at least it has support for top-level asymmetric keys that you can put files into, and ADD files to without the root directory changing its hash. IPNS works around this, but isn't ideal. Neither systems can handle high frequency P2P data that changes - for instance Reddit-like sites etc. those ( http://notabug.io http://notabug.io ) and other sites like the Internet Archive (which has IPFS, WebTorrent, and GUN versions decentralized) are built on our system (https://github.com/amark/gun https://github.com/amark/gun), are already pushing terabytes of P2P traffic. And don't forget about WebTorrent and Secure Scuttlebutt!!!
  - bren2013 8y ago
    Not sure if this fully addresses your use-case, but I like the idea of serving a static bootloader from IPFS. The bootloader would contain all of a website's assets, and code for getting dynamic content from a backend. The backend could be: - A central API where the bootloader can do arbitrary validation on the API responses. - WebTorrent, Scuttlebutt, IPFS PubSub, etc.
    - marknadal 8y ago
      Yes, that is already what the P2P Reddit does, but without IPFS (although this is a good idea!), and using GUN as the "backend" (fully P2P/decentralized though), SEA for validation/security (no need for a central one), and DAM for pubsub (no flooding problems like in libp2p) which can do WebRTC. I'm sure people would love to see an IPFS version of a bootloader, instead of HTTP, that is a cool idea. Have a repo for it?

komali2 8y ago

Somehow, this is the first time I've heard of IPFS. Seems really awesome! From a total novice standpoint, can someone help me understand: >With IPFS, every single block of data stored in the system is addressed by a cryptographic hash of its contents, i.e., a long string of letters and numbers that is unique to that block. When you want a piece of data in IPFS, you request it by its hash. So rather than asking the network “get me the content stored at 93.184.216.34,” you ask “get me the content that has a hash value of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy.” It seems like you must know the "URL" of the "website" you want to visit (files in IPFS) beforehand? But in the case of IPFS, there's no like, DNS service, so you can't type "www.google.com." Basically, it'd be like if to navigate the modern internet, you'd need to know the IP address of whatever site you visit? Is that true of IPFS? Is there any way around that? It seems like a strong limitation, unless someone can make some sort of IPFS search engine that happens to hash out at QN000000000000000000000 or some really memorizable hash... which seems extremely unlikely!

ComNik 8y ago
Check out the white paper at https://github.com/ipfs/papers/raw/master/ipfs-cap2pfs/ipfs-p2p-file-system.pdf https://github.com/ipfs/papers/raw/master/ipfs-cap2pfs/ipfs-..., section 3.7. There is a name system called IPNS.
wongarsu 8y ago
I don't know how IPFS handles this currently, but TOR has the same problem (there addresses are basically a hash of the server certificate). The TOR community handles this by widely advertising the addresses of a few entry points. From there you use classic link traversal and bookmarks. In general, bookmarks become extremely important once you can't memorize addresses.
hanklazard 8y ago
This FAQ from github is a helpful summary of human-readable IPFS via IPNS: https://github.com/ipfs/faq/issues/16#issuecomment-232497229 https://github.com/ipfs/faq/issues/16#issuecomment-232497229
Communitivity 8y ago

yarrel 8y ago

Censorious centralization platform embraces censorship-resistant decentralized platform.

snaky 8y ago
That's the way we've got modern web, where decentralized web sites need the Google Search as entrypoint, decentralized blogs morphed into Facebook, and Cloudflare is going to monopolize the transport layer of all that.

th0ma5 8y ago

How does IPFS handle Denial of Service?

diggan 8y ago
Can you expand on what kind of DoS you're thinking about? Files added to IPFS are stored locally (not pushed to other peers), up until some other node asks about it and/or tries to actually fetch it.
- th0ma5 8y ago
  Yeah I guess are these discoverable or guessable? Could someone request my files at a rate I couldn't supply?

joshfraser 8y ago

This is awesome. We've been running an IPFS gateway at Origin for about a year and are huge fans of the technology. It's nice to see Cloudflare investing in decentralized technology despite contributing to the over-centralization of the web with their core business. There are quite a few public IPFS gateways available, but not all of them are broadly advertised. A few others that I know of: ipfs.io ipfs.infura.io siderus.io ipfs.jes.xxx gateway.originprotocol.com

diggan 8y ago
I just went through our "Public IPFS Gateway Checker" and added a bunch of new ones (include the Cloudflare) one. You can see it running here: https://ipfs.github.io/public-gateway-checker/ https://ipfs.github.io/public-gateway-checker/ And of course, the source code: https://github.com/ipfs/public-gateway-checker/ https://github.com/ipfs/public-gateway-checker/
- joshfraser 8y ago
  Oh neat. Should I send a PR to add the others?
  - diggan 8y ago
    Please do :)
burdakovd 8y ago
Just want to add an alternative gateway that I've been working on recently: dapps.earth It is somewhat different in the sense that it redirects from `gateway/ipfs/hash` to `hash.ipfs.gateway` to ensure different ipfs websites are hosted at different origins (Cloudflare also talked about origin issues in their announcement). Now with what Cloudflare release, they do seem to solve some of the similar problems, but they still ask users to register the domains by themselves (or access in unsafe way with `/ipfs/hash` urls), which means those domains become a centralization point, and also most of the content is served from the same domain. It would be fantastic if they allowed access in form `hash.cloudflare-ipfs.com` or `ens-name.cloudflare-ipfs.com`. In the meantime I may just abandon the idea of serving ipfs content myself, and instead can just operate a small DNS-server that will return CNAME record pointing to Cloudflare and TXT record pointing to ipfs hash for each subdomain.

yoava 8y ago

Being from wix and managing petabytes of media files, we always wondered if a distributed web solution for websites can actually work. Even with the best of clouds (more then one of the big names) we faced downtimes caused by multiple cloud failures. Just imagine, a store that is Geo distributed, resilient and just works. This is super exciting for the web and for customers of websites!!! Big thumb up.

dgreensp 8y ago
Is IPFS really a store, let alone a store with any particular performance or availability properties? If I want to be sure a file is available, don’t I have to host it or pay someone to host it? And if I want to make sure it is geo distributed, I have to geo distribute it or pay a service to? What is the big change?
- wongarsu 8y ago
  Wit IPFS you could rent a few cheap storage servers at different providers around the world and have them all pin your content. That should be resilient against most classes of failures (unlike a cloud provider where bad code pushed to production by the cloud hoster can cause failures in all availability regions). It's not exactly the first protocol enabling that, but IPFS also has the benefit of distributing bandwidth if your clients all access the files via IPFS (since they will serve them from their cache).
  - dgreensp 8y ago
    I see, so geo distribution could be as simple as making the file available in multiple locations, with IPFS doing the rest.
- progval 8y ago
  IPFS is not a store, it's a protocol. You still have to pay someone to host it.

orliesaurus 8y ago

Total noob questions, would love some answers if anyone has them. Here it says: "IPFS is a peer-to-peer file system composed of thousands of computers around the world, each of which stores files on behalf of the network. These files can be anything: cat pictures, 3D models, or even entire websites" - If I run an IPFS node, will I be hosting other people's data? What if these files are illegal? How can my "computer" (I guess a node is the terminology but whatever) be sure I won't be hosting some bad stuff? Then it says: "The content with a hash of QmXnnyufdzAWL5CqZ2RnSNgPbvCc1ALT73s6epPrRnZ1Xy could be stored on dozens of nodes, so if one node that was caching that content goes down, the network will just look for the content on another node." - How long does it take for 1 specific file to be "re-distributed" on another node? and how many times is a file rehosted/distributed on a different node? It would be stupid to have the same file hosted 1.000.000 times - what's the break off point? Is there like a healthcheck so to speak which ensures that a certain file is in danger of disappearing from the network and therefore is automatically pushed for re-distribution And ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? Imagine I want to retrieve cat.gif and cloudflare intercepts my request and serves me cat1.gif - I guess it all boils down to trusting them? But hold up, isn't p2p file system like this all about trusting the network and not 1 server?

diggan 8y ago
The IPFS network works in a pull-model, not push. So when nodes add content, they only add it locally. It's not until some other nodes request it, as it actually transfers. So if you start a IPFS node locally, it won't get content pushed to it (well, some DHT data which is basically routing information, but not the content itself). > ultimately, how can I know for sure that CloudFlare won't play the game where as acting as a proxy they will modify some of the files served? You'll have to add some checks on your end for this as HTTP/browsers cannot provide these guarantees currently (maybe in the future?). So, download file A and add it to IPFS from your end. If you end up with the same hash, the content was not modified. > isn't p2p file system like this all about trusting the network and not 1 server Yes, IPFS <> IPFS is trustless as this verification happens automatically and the clients can verify content locally when it gets fetched. IPFS <> HTTP is harder as there is no functionality for doing this verification.

antpls 8y ago

That's a nice initiative. BitTorrent files are also content addressable, and the DHT allows for routing to several provider nodes. Only files that are downloaded are shared too. What's the plus-value of IPFS over BitTorrent in this scenario?

lgierth 8y ago
Torrents don't have the concept of links at all, they're not a web. Also: Being able to inter-link any content-addressed data structures. With IPLD [1], you can link from an IPFS directory to a Bitcoin transaction, to a Git repository, to a Torrent. [1] https://ipld.io https://ipld.io && https://github.com/ipld/ipld#implementations https://github.com/ipld/ipld#implementations
- antpls 8y ago
  From what I understand from the ipld.io website, IPLD allows to add links to any content-adressable systems. BitTorrent + IPLD could be an alternative to IPFS, I guess? (BTW, notice that BitTorrent isnt used as an example on the front page). When I looked at IPFS some months ago, it looked like a lot of on-going work. It wasn't clear to me what was new and what were the reusable components of the project, and what was reused from the past. In BitTorrent, the DHT for example is a reusable block emerged from previous P2P systems, but it didn't try to reinvent the wheel, and it didn't try to add a FS on top of it, so it solves a specific problem. "IPFS" does a lot of things at once, it would be clearer if the project was cut into sub-specifications that could be reused with legacy protocols. Tho I believe the project is going into that direction, and for example IPLD emerged from it. IPLD looks like a real step-forward. It looks like a "federation" of previous content-addressable protocols, to make them work together, rather than trying to compete with them. BitTorrent + IPLD makes more sense than IPFS to me, more developers have a heard of BitTorrent and they knows what are BitTorrent features, so the introduction of a concept on top of it is easier to grasp.
- imhoguy 8y ago
  > Torrents don't have the concept of links at all, they're not a web. BitTorrent has content-addressable links: https://en.wikipedia.org/wiki/Magnet_URI_scheme https://en.wikipedia.org/wiki/Magnet_URI_scheme You can perfectly form a web with BitTorrent network by sharing e.g. HTML file which links to further resources.

zapita 8y ago

Is Cloudflare doing this in partnership with Protocol Labs, the company backing IPFS? Or are they doing it on their own? I remember that the way Cloudflare communicated around Nginx rubbed a lot of people the wrong way, because they aggressively positioned themselves as "the de-facto Nginx company", to the point where they would sometimes announce a new Nginx feature on their blog before the Nginx developers themselves had a chance to announce it... And of course nobody was more pissed off than Nginx.com, the actual official sponsor of Nginx. I'm wondering if they will behave the same way with Protocol Labs and the existing IPFS community? I sure hope not, I like Cloudflare and love IPFS, and would like to see everyone collaboratig in good spirits, and not competing for cheap marketing points.

headcrack 8y ago

...and the devil introduces an own church. IPFS is about decentralisation! And Cloudflare is the devil of centralization!

dboreham 8y ago
If you analyze any decentralized system long enough you'll discover that it can't be used in practice by regular end-users without some element of centralization. (Dboreham's Law)
- headcrack 8y ago
  Central system Google does not know anything about any Dboreham's "Law".
auslander 8y ago
... and with access to plaintext, SSL stripped data of all traffic
- headcrack 8y ago
  Exactly, it becomes more and more obvious, cloudflare's role in intelligence apparatus becomes bigger.
  - bren2013 8y ago
    IPFS offers content integrity, which you can use to bootstrap confidentiality.
    - headcrack 8y ago
      Confidentiality without central gateways. The topic was, "with stripped down SSL" and evey user should be aware of, cloudflare can read and store every content of every connection made to a domain connected to cloudflare gateways even if they seem to be end-to-end-encrypted.
8y ago

llama052 8y ago

This kind of reminds me of usenet, in a good way. It will be interesting to see what kind of uses this has over time, or if it will be something sustainable if it gains a lot of traction. Good stuff!

brtknr 8y ago

Does anyone know the answer to whether large content that is widely available becomes quicker to download similar to peers in BitTorrent?

lgierth 8y ago
Yes it does -- IPFS will fetch from multiple peers at the same time. It's not terribly optimized yet for finding even more peers for content if it's already fetching from some, but it already does generally fetch from multiple at the same time.

frio 8y ago

This is excellent. I've currently got a hacked together static page for my personal site, which is fronted by CloudFlare, but forwards the actual request onto ipfs.io (which, in turn, serve the files -- ultimately -- from where I've pinned them on eternum). This will let me take a step out of that chain.

timwis 8y ago

Nicely written for more general audiences! Big news :) Congrats Juan, Matt, Michelle, and team if you see this!

loeg 8y ago

> Cloudflare’s IPFS gateway is simply a cache in front of IPFS. Cloudflare does not have the ability to modify or remove content from the IPFS network. If any abusive content is found that is served by the Cloudflare IPFS gateway, you can use the standard abuse reporting mechanism described here. That's an interesting legal hypothesis for when they are prosecuted for serving kiddie porn, isn't it. Seems like they're taking on all of same types of likely abuse as running a Tor exit node. At a minimum it seems like they will have to blacklist serving known abuse.

X-Cubed 8y ago
I think what they're trying to say here is that they may be able to hide/block it, but they can't remove it outright. The same applies for most of their proxying services, they don't have access to modify the origin.
deleted 8y ago
[deleted]
mirimir 8y ago
I expect that they'll blacklist anything iffy. The ipfs.io does, so hey. But you can run your own ipfs demon, and gateway. Even as a tor .onion service.

andrewchambers 8y ago

semi self hosted pastebin alternative with annotations :) (You may need to disable adblock) https://genius.it/cloudflare-ipfs.com/ipfs/Qmc8qNo5adHBnLGSJUCw1WL2gBDyzjQ5NBDWiSHTetbDJQ https://genius.it/cloudflare-ipfs.com/ipfs/Qmc8qNo5adHBnLGSJ...

ezoe 8y ago

After quickly glancing the IPFS document, I can't find the superiority over... say, BitTorrent with DHT and Magnet link(no tracker server or torrent file required.) Save for data deduplication and IPNS maybe. And isn't it rather contradictory? A gateway for P2P-based filesystem.

lgierth 8y ago
You can't easily interlink torrents in way that torrent clients could understand -- torrents are mostly just packages of data, not made to be a web. On your point about the gateway -- I've co-maintained the ipfs.io gateway for the last 3 years and my perspective on its importance is the upgrade path from the existing systems. When developing and deploying new protocols, you need to onboard both users and applications/websites, and the most effective way to do so is seamless interoperability between old-web and new-web. I would imagine it tricky to get any critical mass of adoption if everyone has to choose either old-web or new-web.
- ezoe 8y ago
  But as I understand, BitTorrent can express directory structure. So the IPFS features like mounting by FUSE or browsing through web browser via local server can be implemented as the frontend of BitTorrent too. Currently, there are tons of independently developed BitTorrent protocol implementations while IPFS has very few implementations from the same body. You can't beat the existing software by slightly better implementations. Apart from probably IPNS, most IPFS features can be achieved by BitTorrent frontend too. As for the gateway. If somebody want a gateway for P2P network, that's because the same P2P network is inefficient or difficult to use. If the P2P network is difficult to use, not much people run their own node. Rather, they relies on somebody else's node. We see the bad consequence of this number of times in the crypto-currency. In order to trust the P2P crypto-currenncy, It's essential that you run your own node for your wallet from the computer you own and trust. But the reality is, it's pretty inconvenient or difficult to do for most of the users. So what happened was, very few big central node which manage the wallet for you, exchange between different currencies for you. At this point, this is not a P2P crypto-currency. The big central node become the single point of failure. If it goes down or breached, you're doomed. For the healthy P2P network that can replace the existing central-authority network of today, all users must run their own node and it must be very very easy to do so. So there should no demand for the gateway.

blacksoil 8y ago

The part I'm still confused is: how does the cache being updated? For example in the traditional web architecture if I go to www.example.com/index.htm, the host server of example.com tells the hash of index.htm and depending on that my web browser decides to use its cache or do a fresh request. How would this work in IPFS? How about dynamic pages? Does that mean my browser still has to contact www.example.com to get the latest version's hash but then has the option to request the file from IPFS instead of www.example.com? What about if example.com goes down?

e12e 8y ago
In basic ipfs the address is content-dependent hash - same url should always return same content (excepting hash collisions which should be so rare and improbable as to never occur in practice). So the short answer is that there's nothing to update: and url will return the same content (or no content if no one has it pinned anymore). AFAIK the ipfs approach to this conundrum is a name/resolver system - which essentially adds indirection. This is Thomas he'll always be Thomas. But for now, he's also class valedictorian. Some time in the future, Thomas will still be Thomas, but someone else might be class valedictorian.

Tetris1 8y ago

File sharing community should adopt and thrive on this technology. Viva!

shalabhc 8y ago

Is there a way to uniformly represent links and relationships between IPFS files in IPFS itself?

diggan 8y ago
Yes, that's what IPLD (https://ipld.io/ https://ipld.io/) is for and more. It's used in IPFS to do links between files in IPFS and also links between files and other formats like Git or Ethereum transactions. You can see the data structure itself if you use the `ipfs object` commands: $ tree test-directory test-directory ├── directory-a │ └── file-b └── file-a 1 directory, 2 files $ ipfs add -r test-directory added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH test-directory/directory-a/file-b added QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH test-directory/file-a added QmVSi16sFvaKxJXGHWj9yCVcbxJBLRePBARKt1KUyWMGAK test-directory/directory-a added Qmbm9JEWeD8EX4pQUYpx4eDsjqyzGjQMdGzdJEdwJvcHb4 test-directory $ ipfs object get Qmbm9JEWeD8EX4pQUYpx4eDsjqyzGjQMdGzdJEdwJvcHb4 {"Links":[{"Name":"directory-a","Hash":"QmVSi16sFvaKxJXGHWj9yCVcbxJBLRePBARKt1KUyWMGAK","Size":58},{"Name":"file-a","Hash":"QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH","Size":6}],"Data":"\u0008\u0001"} $ ipfs object get QmVSi16sFvaKxJXGHWj9yCVcbxJBLRePBARKt1KUyWMGAK {"Links":[{"Name":"file-b","Hash":"QmbFMke1KXqnYyBBWxB74N4c5SBnJMVAiMNRcGu6x1AwQH","Size":6}],"Data":"\u0008\u0001"}