4 ms·

> ability to handle HN hug-of-death Seriously, this is bullshit. When my blog was on the frontpage, CPU load on my single 1-core VPS peaked at 5% and bandwidth

> ability to handle HN hug-of-death Seriously, this is bullshit. When my blog was on the frontpage, CPU load on my single 1-core VPS peaked at 5% and bandwidth usage was about 1 megabit per second. No Netlify, no Cloudflare, no nothing. If your website cannot handle the HN crowd, it's just badly engineered. Period. Source: https://xyrillian.de/thoughts/posts/latency-matters-aftermath.html https://xyrillian.de/thoughts/posts/latency-matters-aftermat...

saurik 7y ago
Yeah: I honestly don't understand how people build websites that can't handle Hacker News levels of traffic... this site just isn't _that_ popular in the grand scheme of things.
- jandrese 7y ago
  The instant your website requires a database query to build the front page you're susceptible to the HN hug of death. Obviously you can build a scalable solution to handle traffic spikes, but there will always be the danger. If you're talking about an ordinary webserver giving out an ordinary webpage you will run out of bandwidth long before you overload the server.
  - _bxg1 7y ago
    Yeah; static pages like blog posts should really be cached out from templates + data, to avoid this problem.
  - weberc2 7y ago
    Depends on your database. S3 will handle the hug just fine.
  - AlchemistCamp 7y ago
    That's a bit dramatic. I've load-tested database backed sites on $5/month DO droplets and gotten > 1k requests per second for extended tests. My primary site not only stores articles in the DB, but runs them through an extended, custom markdown parser upon retrieval, and processes and reports event data on every page load. It was still successfully returning ~250 reqs/sec before I'd bothered implementing any caching or pre-compilation of markdown at all. In contrast, people recently HNed[1] are reporting about 12-15k visits over a day. That's something a single server on a $5/month VPS can handle in a minute 1) https://getpolarized.io/2019/04/02/getting-hacker-news-lessons-for-entrepreneurs.html https://getpolarized.io/2019/04/02/getting-hacker-news-lesso...
    - majewsky 7y ago
      > In contrast, people recently HNed[1] are reporting about 12-15k visits over a day. This matches my experience. I think I got about 10k visits over the course of a day, 80% within the first three hours.
weberc2 7y ago
I would expect open connections to be the bottleneck (not CPU or mem). Do you just have very low latency and thus relatively few concurrent open connections (yes, I saw the graphs in your link, but it's hard to figure out how many concurrent connections are open at a time)? EDIT: Downvoters, would really like to know why you're downvoting? Am I being naive in thinking a blog would be bottlenecked on connections?
- nickjj 7y ago
  My site has been on the front page a few times. Concurrent requests were usually in the 400-800 range depending on where it ranked and what it was about. That's going by Google Analytics too which means it's likely higher since devs tend to disable as much tracking as possible. nginx won't even break a sweat if you're on a low end VPS and are serving static content. You'll have pretty much the same latency as serving 1 concurrent request (single digit milliseconds + whatever network latency there is between your server and the visitor).
  - seba_dos1 7y ago
    I have once handled a website being HNed and /.ed a few years ago. The issue turned out to be the default Apache config from distro, which was very ill-suited for such traffic, generating tons of processes which struggled to run on a small VPS. Switching to nginx fixed it; I suppose switching Apache to mpm_event model (or even just tuning the threading configuration) would be enough too.
- noxToken 7y ago
  Or low total bandwidth. A personal blog not expecting a lot of traffic may only have puddle of transfer bandwidth per month. Depending on the content of the post, that can max out a cap rather quickly.
  - forrestthewoods 7y ago
    I don’t understand the downvotes here. A small blog can randomly get a 100,000 view spike. If the blog contains images and maybe even a gif it’s easy to blow through bandwidth caps for most free tiers. I use netlify’s free tier but actually host my videos on BunnyCDN. I also make sure to optimize my pngs and jpgs. This isn't precautionary. It's actually necessary.
Gravityloss 7y ago
Worthy of a submission of its own.
llao 7y ago
A blog typically comes with a comment function. That's something dynamic which requires lookup in some sort of database. Your page is purely static and does not even have pagination between posts. Hosting that is trivial and nothing to boast about. Your comparison is really unfair and needlessly nasty.
- PeterisP 7y ago
  A comment function doesn't necessarily require a database lookup for every request, you can cache the blog page for a second or two so that it doesn't collapse when getting a hundred requests per second.
  - llao 7y ago
    Yes and that makes things magnitudes more complex than just serving a static HTML page.
    - PeterisP 7y ago
      Not really, it's just basic configuration of nginx or something like that. Putting up nginx with microcaching in front of your webserver is literally a five minute task. If you have no idea on how to do it, then it's going to take half an hour with some tutorials, so even in that case it's not that much more work than serving a static HTML page.
      - llao 7y ago
        Oh how cool is that, I never even heard that term before. Thanks! It does look quite complex though, I have to learn about proxying and nginx.
- eterps 7y ago
  Displaying comments doesn't have to be dynamic. Most need reviewing anyway. https://github.com/eduardoboucas/staticman https://github.com/eduardoboucas/staticman
- 7y ago
jolmg 7y ago
I wonder how much variance there is in traffic for sites that are on the front-page. That variance could be based on a combination of the hour of the day (because of the varying timezones of HN readers), the day of the week (we don't all check every single day), and the particular topic of a post (I would think that the average individual doesn't click on everything). It could be that while your site was on the front-page, others at other times, other days, or of other topics could have received visits many times more (or less) than what you received.
louwrentius 7y ago
Same for me, I have a static HTML site and it never even broke a sweat.
bdcravens 7y ago
A quick check shows the site is on DreamHost which offers $2.59 a month shared hosting which isn't going to handle the load (though they have the full assortment of hosting options you'd see, up to dedicated hardware).

saurik 7y ago

Yeah: I honestly don't understand how people build websites that can't handle Hacker News levels of traffic... this site just isn't _that_ popular in the grand scheme of things.

jandrese 7y ago
The instant your website requires a database query to build the front page you're susceptible to the HN hug of death. Obviously you can build a scalable solution to handle traffic spikes, but there will always be the danger. If you're talking about an ordinary webserver giving out an ordinary webpage you will run out of bandwidth long before you overload the server.
- _bxg1 7y ago
  Yeah; static pages like blog posts should really be cached out from templates + data, to avoid this problem.
- weberc2 7y ago
  Depends on your database. S3 will handle the hug just fine.
- AlchemistCamp 7y ago
  That's a bit dramatic. I've load-tested database backed sites on $5/month DO droplets and gotten > 1k requests per second for extended tests. My primary site not only stores articles in the DB, but runs them through an extended, custom markdown parser upon retrieval, and processes and reports event data on every page load. It was still successfully returning ~250 reqs/sec before I'd bothered implementing any caching or pre-compilation of markdown at all. In contrast, people recently HNed[1] are reporting about 12-15k visits over a day. That's something a single server on a $5/month VPS can handle in a minute 1) https://getpolarized.io/2019/04/02/getting-hacker-news-lessons-for-entrepreneurs.html https://getpolarized.io/2019/04/02/getting-hacker-news-lesso...
  - majewsky 7y ago
    > In contrast, people recently HNed[1] are reporting about 12-15k visits over a day. This matches my experience. I think I got about 10k visits over the course of a day, 80% within the first three hours.

weberc2 7y ago

I would expect open connections to be the bottleneck (not CPU or mem). Do you just have very low latency and thus relatively few concurrent open connections (yes, I saw the graphs in your link, but it's hard to figure out how many concurrent connections are open at a time)? EDIT: Downvoters, would really like to know why you're downvoting? Am I being naive in thinking a blog would be bottlenecked on connections?

nickjj 7y ago
My site has been on the front page a few times. Concurrent requests were usually in the 400-800 range depending on where it ranked and what it was about. That's going by Google Analytics too which means it's likely higher since devs tend to disable as much tracking as possible. nginx won't even break a sweat if you're on a low end VPS and are serving static content. You'll have pretty much the same latency as serving 1 concurrent request (single digit milliseconds + whatever network latency there is between your server and the visitor).
- seba_dos1 7y ago
  I have once handled a website being HNed and /.ed a few years ago. The issue turned out to be the default Apache config from distro, which was very ill-suited for such traffic, generating tons of processes which struggled to run on a small VPS. Switching to nginx fixed it; I suppose switching Apache to mpm_event model (or even just tuning the threading configuration) would be enough too.
noxToken 7y ago
Or low total bandwidth. A personal blog not expecting a lot of traffic may only have puddle of transfer bandwidth per month. Depending on the content of the post, that can max out a cap rather quickly.
- forrestthewoods 7y ago
  I don’t understand the downvotes here. A small blog can randomly get a 100,000 view spike. If the blog contains images and maybe even a gif it’s easy to blow through bandwidth caps for most free tiers. I use netlify’s free tier but actually host my videos on BunnyCDN. I also make sure to optimize my pngs and jpgs. This isn't precautionary. It's actually necessary.

Gravityloss 7y ago

Worthy of a submission of its own.

llao 7y ago

A blog typically comes with a comment function. That's something dynamic which requires lookup in some sort of database. Your page is purely static and does not even have pagination between posts. Hosting that is trivial and nothing to boast about. Your comparison is really unfair and needlessly nasty.

PeterisP 7y ago
A comment function doesn't necessarily require a database lookup for every request, you can cache the blog page for a second or two so that it doesn't collapse when getting a hundred requests per second.
- llao 7y ago
  Yes and that makes things magnitudes more complex than just serving a static HTML page.
  - PeterisP 7y ago
    Not really, it's just basic configuration of nginx or something like that. Putting up nginx with microcaching in front of your webserver is literally a five minute task. If you have no idea on how to do it, then it's going to take half an hour with some tutorials, so even in that case it's not that much more work than serving a static HTML page.
    - llao 7y ago
      Oh how cool is that, I never even heard that term before. Thanks! It does look quite complex though, I have to learn about proxying and nginx.
eterps 7y ago
Displaying comments doesn't have to be dynamic. Most need reviewing anyway. https://github.com/eduardoboucas/staticman https://github.com/eduardoboucas/staticman
7y ago

jolmg 7y ago

I wonder how much variance there is in traffic for sites that are on the front-page. That variance could be based on a combination of the hour of the day (because of the varying timezones of HN readers), the day of the week (we don't all check every single day), and the particular topic of a post (I would think that the average individual doesn't click on everything). It could be that while your site was on the front-page, others at other times, other days, or of other topics could have received visits many times more (or less) than what you received.

louwrentius 7y ago

Same for me, I have a static HTML site and it never even broke a sweat.

bdcravens 7y ago

A quick check shows the site is on DreamHost which offers $2.59 a month shared hosting which isn't going to handle the load (though they have the full assortment of hosting options you'd see, up to dedicated hardware).