7 ms·

The basis of this post isn't too sound: * The "5k% spikes" being discussed here are on a tiny base load. His peak is 6K page views in a day. That's 4 pages a M

by matt2000 7y ago

The basis of this post isn't too sound: * The "5k% spikes" being discussed here are on a tiny base load. His peak is 6K page views in a day. That's 4 pages a MINUTE. A raspberry pi could handle that. Anything can handle that. There's no load here. * It's not clear what IPFS is getting him, but it's certainly not performance. Any performance improvements are coming from using the Cloudflare caching proxy, which is basically just using Cloudflare as a CDN. You can get that with a normal website behind Cloudflare with a lot less hassle. It's fine to check out the implementation details for interest's sake, but let's just be clear that it all could have been done with a $5 VPS.

prophesi 7y ago
Just a little nitpick: * That's 4 pages a MINUTE. That's assuming that those 6k views were distributed evenly throughout the day. The article mentions the Hacker News / Reddit effect, so I'm assuming there was, indeed, a sizable increase in traffic for a short duration that a raspberry pi wouldn't be able to handle. I do wish it showed precisely what happened on that 6k view day, however
- aflag 7y ago
  Even if he got 6k views in the course of 30 minutes, I still think a raspberry pi running nginx and serving static pages could handle. It would have trouble if everyone open the link within 5s or so, but I doubt that's what happened. I think we'd had seen that graph if that was the case.
  - rudolph9 7y ago
    If you take a step back and look it from the context that the data is somewhat being served by untrusted distributed devices in a verifiable way; it’s pretty freaking amazing it works at all! Sure a raspberry pie could replace this in heartbeat but it’s not about current throughput being good, it’s about making it work as intended, ensuring the design is theoretically scalable, and marching toward a fully distributed network. I realize the author is talking mostly about performance but the think that is exciting about ipfs is it’s not some monolithic project trying to take over the world but rather a modular set of standards (IPLD, MULTICODE, libp2p) that generally do a very good job of integrating with legacy and have reasonable future proofed design. Ipfs is just one application bring all these thing together is in a particular confirmation but many more interoperable variations are likely soon to come.
    - aflag 7y ago
      Not sure if this post showed that, though. I think it can be safely assumed that people coming from hacker news or reddit are not likely to be using ipfs clients. Therefore, this didn't work any different than having a server behind a CDN.
marknadal 7y ago
The question is, is CloudFlare caching IPFS or not? If CloudFlare is running IPFS naked, then IPFS seems to be scaling. If CloudFlare is caching IPFS, then it is CloudFlare saving their butts. Can we confirm which it is? I'm trying to chat with CloudFlare about my own P2P protocol, which I've seen handle HackerNoon's 15M monthly users (I saw about 10K concurrent users per second at peak load), to run GUN ( https://github.com/amark/gun https://github.com/amark/gun ). Because I do think it is important to test all these protocols out at bigger/larger scales, that certainly makes it easier to then debug and then fix problems (assuming CF isn't caching).
- Skunkleton 7y ago
  Datapoint: I installed the FF extension mentioned in the article, and the performance of /ipns/withblue.ink was inferior to what cloudflare provided.
- cmiles74 7y ago
  Yes, it's the CloudFlare caching that is providing the site to the majority of people. For the IPFS version, people need to have a plugin installed or have the IPFS software running on their machines.
- joecot 7y ago
  > If CloudFlare is running IPFS naked, then IPFS seems to be scaling. > If CloudFlare is caching IPFS, then it is CloudFlare saving their butts. They are caching IPFS for people who don't support IPFS. They are not caching IPFS for people who do support IPFS, and instead passing the user through to the IPFS servers. Which means Cloudflare is indeed saving their butts, because as soon as IPFS has widespread support, this falls flat on its face. Same as traditional hosting would without a CDN. Their IPFS setup may be able to distribute the load enough to avoid an internet hug crippling it (that's part of IPFS's point), but this setup in no way demonstrates that.
joecot 7y ago
> It's fine to check out the implementation details for interest's sake, but let's just be clear that it all could have been done with a $5 VPS. If you're just hosting a static site, it can be done for free. You can use Github pages (or any similar free static site host) + Cloudflare for free. Getting Slackdotted, HNed, or reddit hugged will all survive just fine, for free. So to recap: * Use a static site, maybe with a static site generator like Hugo * Use a static site host, which could be github pages (free), s3 (cheap), or a $5 VPS anywhere running apache or nginx * Use a free CDN, likely Cloudflare, to handle the brunt of the bandwidth. The IPFS is interesting but way more work currently than any of this is worth. And if people are using IPFS compatible browsers, the load is being transferred to the IPFS servers anyway, which is worse than just using a regular hosting method and Cloudflare CDN. If IPFS actually had decent adoption, the method described would fall to a crippling internet hug. The goal of IPFS is to distribute the load (for people with IPFS browsers), and it's possible the article's loadout would do that, but given he's seeing 160GB of transfer from his servers, it certainly doesn't seem to be doing that very effectively currently. And this setup does not in any way demonstrate IPFS is ready to do that, because Cloudflare is saving his servers' butts.
- momack2 7y ago
  In IPFS "the servers" aren't restricted to the original content hosts / large-scale CDNs / a single corp, so saying "the load is being transferred to the IPFS servers" doesn't take into account the fact that you, your roommate, your neighbor, or 6K other people on the internet could be storing/serving this same data, helping compensate for increased load when accessing/re-accessing a spiking piece of content. There's still a ton of room for performance improvement here - but where this will really shine is not where Cloudflare and existing tools do great, but in the edge cases where a more resilient solution matters (be that for offline caching, local-first collaboration, or censorship resistance).
  - joecot 7y ago
    Sure, it is possible that IPFS would spread out the usage across the other users, and this could still work for people bypassing Cloudflare. But there's no demonstration here of that really working. Just that Cloudflare is really great at what it does. And yes, it's a chicken/egg scenario -- how do you prove that IPFS adoption would negate the need for CDNs, without widespread IPFS adoption? But currently this isn't some amazing Slashdot proof solution. They needed to setup 3 servers and a whole bunch of software when a single server + free CDN would've accomplished the same thing, faster and cheaper. It's presented as though CDN+IPFS is why this works so great, when really it's just CDN and IPFS just makes it more complicated. If the goal is to learn, try out new tech, break dependence on CDNs, then this is great, but a single note in the article that this is a learning opportunity and it can be done easier without IPFS would've saved the author a lot of back and forth in the comments here.
acdha 7y ago
For perspective, I was seeing 50-200 pages/second sustained for PHP + MySQL on a single modest Linux server 20 years ago. We had clients get well over 6k hits within minutes of an announcement (IPO details launching at market open) and had plenty of headroom.

prophesi 7y ago

Just a little nitpick: * That's 4 pages a MINUTE. That's assuming that those 6k views were distributed evenly throughout the day. The article mentions the Hacker News / Reddit effect, so I'm assuming there was, indeed, a sizable increase in traffic for a short duration that a raspberry pi wouldn't be able to handle. I do wish it showed precisely what happened on that 6k view day, however

aflag 7y ago
Even if he got 6k views in the course of 30 minutes, I still think a raspberry pi running nginx and serving static pages could handle. It would have trouble if everyone open the link within 5s or so, but I doubt that's what happened. I think we'd had seen that graph if that was the case.
- rudolph9 7y ago
  If you take a step back and look it from the context that the data is somewhat being served by untrusted distributed devices in a verifiable way; it’s pretty freaking amazing it works at all! Sure a raspberry pie could replace this in heartbeat but it’s not about current throughput being good, it’s about making it work as intended, ensuring the design is theoretically scalable, and marching toward a fully distributed network. I realize the author is talking mostly about performance but the think that is exciting about ipfs is it’s not some monolithic project trying to take over the world but rather a modular set of standards (IPLD, MULTICODE, libp2p) that generally do a very good job of integrating with legacy and have reasonable future proofed design. Ipfs is just one application bring all these thing together is in a particular confirmation but many more interoperable variations are likely soon to come.
  - aflag 7y ago
    Not sure if this post showed that, though. I think it can be safely assumed that people coming from hacker news or reddit are not likely to be using ipfs clients. Therefore, this didn't work any different than having a server behind a CDN.

marknadal 7y ago

The question is, is CloudFlare caching IPFS or not? If CloudFlare is running IPFS naked, then IPFS seems to be scaling. If CloudFlare is caching IPFS, then it is CloudFlare saving their butts. Can we confirm which it is? I'm trying to chat with CloudFlare about my own P2P protocol, which I've seen handle HackerNoon's 15M monthly users (I saw about 10K concurrent users per second at peak load), to run GUN ( https://github.com/amark/gun https://github.com/amark/gun ). Because I do think it is important to test all these protocols out at bigger/larger scales, that certainly makes it easier to then debug and then fix problems (assuming CF isn't caching).

Skunkleton 7y ago
Datapoint: I installed the FF extension mentioned in the article, and the performance of /ipns/withblue.ink was inferior to what cloudflare provided.
cmiles74 7y ago
Yes, it's the CloudFlare caching that is providing the site to the majority of people. For the IPFS version, people need to have a plugin installed or have the IPFS software running on their machines.
joecot 7y ago
> If CloudFlare is running IPFS naked, then IPFS seems to be scaling. > If CloudFlare is caching IPFS, then it is CloudFlare saving their butts. They are caching IPFS for people who don't support IPFS. They are not caching IPFS for people who do support IPFS, and instead passing the user through to the IPFS servers. Which means Cloudflare is indeed saving their butts, because as soon as IPFS has widespread support, this falls flat on its face. Same as traditional hosting would without a CDN. Their IPFS setup may be able to distribute the load enough to avoid an internet hug crippling it (that's part of IPFS's point), but this setup in no way demonstrates that.

joecot 7y ago

> It's fine to check out the implementation details for interest's sake, but let's just be clear that it all could have been done with a $5 VPS. If you're just hosting a static site, it can be done for free. You can use Github pages (or any similar free static site host) + Cloudflare for free. Getting Slackdotted, HNed, or reddit hugged will all survive just fine, for free. So to recap: * Use a static site, maybe with a static site generator like Hugo * Use a static site host, which could be github pages (free), s3 (cheap), or a $5 VPS anywhere running apache or nginx * Use a free CDN, likely Cloudflare, to handle the brunt of the bandwidth. The IPFS is interesting but way more work currently than any of this is worth. And if people are using IPFS compatible browsers, the load is being transferred to the IPFS servers anyway, which is worse than just using a regular hosting method and Cloudflare CDN. If IPFS actually had decent adoption, the method described would fall to a crippling internet hug. The goal of IPFS is to distribute the load (for people with IPFS browsers), and it's possible the article's loadout would do that, but given he's seeing 160GB of transfer from his servers, it certainly doesn't seem to be doing that very effectively currently. And this setup does not in any way demonstrate IPFS is ready to do that, because Cloudflare is saving his servers' butts.

momack2 7y ago
In IPFS "the servers" aren't restricted to the original content hosts / large-scale CDNs / a single corp, so saying "the load is being transferred to the IPFS servers" doesn't take into account the fact that you, your roommate, your neighbor, or 6K other people on the internet could be storing/serving this same data, helping compensate for increased load when accessing/re-accessing a spiking piece of content. There's still a ton of room for performance improvement here - but where this will really shine is not where Cloudflare and existing tools do great, but in the edge cases where a more resilient solution matters (be that for offline caching, local-first collaboration, or censorship resistance).
- joecot 7y ago
  Sure, it is possible that IPFS would spread out the usage across the other users, and this could still work for people bypassing Cloudflare. But there's no demonstration here of that really working. Just that Cloudflare is really great at what it does. And yes, it's a chicken/egg scenario -- how do you prove that IPFS adoption would negate the need for CDNs, without widespread IPFS adoption? But currently this isn't some amazing Slashdot proof solution. They needed to setup 3 servers and a whole bunch of software when a single server + free CDN would've accomplished the same thing, faster and cheaper. It's presented as though CDN+IPFS is why this works so great, when really it's just CDN and IPFS just makes it more complicated. If the goal is to learn, try out new tech, break dependence on CDNs, then this is great, but a single note in the article that this is a learning opportunity and it can be done easier without IPFS would've saved the author a lot of back and forth in the comments here.

acdha 7y ago

For perspective, I was seeing 50-200 pages/second sustained for PHP + MySQL on a single modest Linux server 20 years ago. We had clients get well over 6k hits within minutes of an announcement (IPO details launching at market open) and had plenty of headroom.