11 ms·

On a completely unrelated note, I'm so surprised that static sites like this that are basically just simple advertising pages with text and some internal links

On a completely unrelated note, I'm so surprised that static sites like this that are basically just simple advertising pages with text and some internal links just completely fail like this so often when they get a little attention. It's just serving HTML, CSS and some JQuery (looked at the source on Google cache). If they set up even the simplest caching solution with nginx or some other reverse proxy or hosting tool it wouldn't fail so spectacularly. Unless it's doing backend requests, there are extremely simple solutions to keep sites like these up

praptak 5y ago
It's probably bandwidth. You may serve static html from L1 cache and still get DoSed.
- aniforprez 5y ago
  In this case, nginx is serving a 502 status. It's possible whatever stack was running that was being proxied to by nginx was killed, possibly by RAM or CPU limitations. I don't think it's a bandwidth issue here
  - praptak 5y ago
    Fair enough, for me it's just timeouts.
robertlagrant 5y ago
The default option is to just front it with Cloudflare.
- SXX 5y ago
  The best way is to use GitHub Pages + CloudFlare. This way you can also use site generator like Hugo that runs by Github Actions and you can even deploy some code to CloudFlare Actions. Here how I setup this for OSS game engine I work on: https://github.com/vcmi/VCMI.eu/ https://github.com/vcmi/VCMI.eu/
  - uncertainrhymes 5y ago
    Github pages already has a CDN in front of it, but I like your workflow with Hugo. The widelands site seems to go directly to netcup in Germany. It is likely just a DDoS detector mistaking sudden interest as an attack.
    - dwild 5y ago
      I believe Cloudflare in his case isn't acting as a CDN (though it's still is) but a way to use his own domain (vcmi.eu)
      - SXX 5y ago
        While I mostly use CloudFlare as DNS manager it's absolutely not required to use own domain on GitHub Pages. https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site https://docs.github.com/en/pages/configuring-a-custom-domain...
  - robertlagrant 5y ago
    Yeah I do that. I used Pelican and it works well.
- hallway_monitor 5y ago
  I can't bring myself to believe that an external caching layer is absolutely required to handle the load from something like a hacker news front page. WordPress with no caching I can see being taken down by 10 to 20 requests per second, but any kind of local caching should be able to handle the load easily. Does HN produce more traffic than that?
  - lixtra 5y ago
    The one time I was involved in something on the top, I remember about 400 concurrent users according to google analytics. Our cheap server had no problem to serve the static site.
  - calpaterson 5y ago
    I've been on the frontpage before and my logs indicate that the peak load was somewhere north of 350 requests/sec though the non-peak (while still on the frontpage) was typically 10-20 requests/second. That is for a very simple page consisting of an html document, css, an image, a favicon and no javascript (eg https://calpaterson.com/printers.html https://calpaterson.com/printers.html). If you can serve from a reverse proxy cache locally that will be fine but if some PHP/Python/NodeJS is running on each request you can run into trouble with the small servers people typically use for their side-project's website.
- capableweb 5y ago
  Not only is it a "default option" that is making the web less decentralized, it is also way overkill for something as simple as this. Make sure nginx serves cached content (right headers) instead of reading the content from disk and maybe, just maybe, throw in a static cache like varnish in front. For the times my websites hit the frontpage of HN, a simple nginx instance (without any cache in front like Varnish) on a $5/month Digital Ocean server was enough to handle things.
  - majewsky 5y ago
    Same for my site: https://xyrillian.de/thoughts/posts/latency-matters-aftermath.html https://xyrillian.de/thoughts/posts/latency-matters-aftermat...
  - brundolf 5y ago
    You don't even need nginx; I've got a single Node process on a Heroku "hobby" node serving my site, and it's weathered several front-page visits peaking at 15 requests per second without problems. The important thing is the static-rendering (assuming it applies for your site's content)
    - mst 5y ago
      Having a blog post of mine get posted to HN (many years ago) was how I discovered I'd accidentally introduced a bug into my web server config that meant the app server behind it was serving the static assets as well as the dynamic stuff. It entered a state of wedgitude quite rapidly when traffic ramped up.
  - robertlagrant 5y ago
    > making the web less decentralized In this case, this is not a virtue. Using a CDN is option, reversible and is likely to spread your content out to be much less centralised than anything hand rolled.
    - capableweb 5y ago
      Except when 20% of the web does that the web as a whole become less decentralized, while your files might still be further spread apart in the world. What I was thinking about was the web as a whole, not my specific files.
- sildur 5y ago
  Cloudflare is awful, I get tons of captchas while browsing with tor.
la_fayette 5y ago
I think the website is built with django, maybe it has some community stuff on it, which requires server side processing... Git repo: https://github.com/widelands/widelands-website https://github.com/widelands/widelands-website
- deleted 5y ago
  [deleted]
wutwutwutwut 5y ago
> I'm so surprised that static sites like this The person you replied to linked the github account and if you ckecked it you would see that it was not a static site. I am not surprised that you decided to write a comment instead though.
- account42 5y ago
  This is also obvious from looking at a cached version of the site: there are there is a list of online users and posts in the sidebar.
- deleted 5y ago
  [deleted]
- maerF0x0 5y ago
  downvoted for snark: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
deleted 5y ago
[deleted]
INTPenis 5y ago
You were wrong technically but you have a point. These sites should leverage existing services like reddit or AWS. Personally I'd put a static information site on AWS and a community on Reddit if I was the creator for something like this game. It's just like SMB's today don't really need much infrastructure, well open source projects shouldn't either.
- capableweb 5y ago
  That's a horrible idea. Yes, open source should definitely get off closed and abusive platforms like Reddit or AWS. Really, the open source community should start dog fooding more. It seems like yesterday all the communities were on IRC and it was easy to find everyone as long as you were connected to Freenode. Now everyone is using Slack, Discord or other closed-sourced systems that are not even built for the use case of "open by default". It's really sad to see, especially when a lot of developers in the ecosystem are cheering on projects moving to platforms like reddit or AWS. To run your own infrastructure is not as hard as you think, and it gets a lot cheaper too.
  - INTPenis 5y ago
    I completely get where you're coming from, I'm a staunch self hoster myself. But why re-invent the wheel before you have to? That's just a waste of time and resources that could be used to promote and build your project. Like in this very case, the website is now down and unable to promote the project because it was made in a traditional way. Imho it's enough to retain ownership of domains to really own a project. Like the widelands.org domain. So whatever happens with AWS, or reddit, that domain can always move and point visitors to new communities. In my personal selfhosting environment I have my own domain pointed to protonmail. So even if protonmail goes belly up I can always move my domain somewhere else. And even my domain is picked to give me maximum control. I use my own country's ccTLD because they have a good conflict resolution department that will benefit me as a citizen.
- mst 5y ago
  The forum's a more complicated question but for the static information site for an open source project that's already hosting its code on github, configuring a custom domain for the github pages site would seem even simpler than sticking it on AWS.
tomc1985 5y ago
Or they can just go down and people don't have to freak out. Stop recommending people join the CDN cartel
- qnsi 5y ago
  I guess people want their site to be available when it gets viral?
  - tomc1985 5y ago
    By contracting with a private company known for shielding propagandists Totally in line with the OSS ethos.... not
    - mst 5y ago
      Being against censorship of even very stupid shit is absolutely in line with the old school hacker ethos. "The internet treats censorship as damage and routes around it" was absolutely a thing. [[shakes crutch at the kids on his lawn]]
      - tomc1985 5y ago
        I am generally against censorship but had to revise that stance after I realized we've given a global platform to manipulative idiots truly unworthy of global self-expression. If they want to espouse their crap they can stand on a real soapbox and shout at crowds, but nothing more
        gameman144 5y ago
        Neat, and who decides whose "crap" is unworthy of freedom of speech? Like, my main objection with censorship isn't "I think that this particular idea is worthy of defending", it's that any mechanism that lets people censor bad opinions could also by used to censor good opinions, and that mechanism is way too powerful and prone to abuse for my comfort.
zild3d 5y ago
generally yes, but their site is also a forum (and some other things). Looks like its made with django
- account42 5y ago
  They could still make the landing page static or at least enable some (micro)caching - there is no reason for every request to hit the backend. Of course, this is obvious only in hindsight.

praptak 5y ago

It's probably bandwidth. You may serve static html from L1 cache and still get DoSed.

aniforprez 5y ago
In this case, nginx is serving a 502 status. It's possible whatever stack was running that was being proxied to by nginx was killed, possibly by RAM or CPU limitations. I don't think it's a bandwidth issue here
- praptak 5y ago
  Fair enough, for me it's just timeouts.

robertlagrant 5y ago

The default option is to just front it with Cloudflare.

SXX 5y ago
The best way is to use GitHub Pages + CloudFlare. This way you can also use site generator like Hugo that runs by Github Actions and you can even deploy some code to CloudFlare Actions. Here how I setup this for OSS game engine I work on: https://github.com/vcmi/VCMI.eu/ https://github.com/vcmi/VCMI.eu/
- uncertainrhymes 5y ago
  Github pages already has a CDN in front of it, but I like your workflow with Hugo. The widelands site seems to go directly to netcup in Germany. It is likely just a DDoS detector mistaking sudden interest as an attack.
  - dwild 5y ago
    I believe Cloudflare in his case isn't acting as a CDN (though it's still is) but a way to use his own domain (vcmi.eu)
    - SXX 5y ago
      While I mostly use CloudFlare as DNS manager it's absolutely not required to use own domain on GitHub Pages. https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site https://docs.github.com/en/pages/configuring-a-custom-domain...
- robertlagrant 5y ago
  Yeah I do that. I used Pelican and it works well.
hallway_monitor 5y ago
I can't bring myself to believe that an external caching layer is absolutely required to handle the load from something like a hacker news front page. WordPress with no caching I can see being taken down by 10 to 20 requests per second, but any kind of local caching should be able to handle the load easily. Does HN produce more traffic than that?
- lixtra 5y ago
  The one time I was involved in something on the top, I remember about 400 concurrent users according to google analytics. Our cheap server had no problem to serve the static site.
- calpaterson 5y ago
  I've been on the frontpage before and my logs indicate that the peak load was somewhere north of 350 requests/sec though the non-peak (while still on the frontpage) was typically 10-20 requests/second. That is for a very simple page consisting of an html document, css, an image, a favicon and no javascript (eg https://calpaterson.com/printers.html https://calpaterson.com/printers.html). If you can serve from a reverse proxy cache locally that will be fine but if some PHP/Python/NodeJS is running on each request you can run into trouble with the small servers people typically use for their side-project's website.
capableweb 5y ago
Not only is it a "default option" that is making the web less decentralized, it is also way overkill for something as simple as this. Make sure nginx serves cached content (right headers) instead of reading the content from disk and maybe, just maybe, throw in a static cache like varnish in front. For the times my websites hit the frontpage of HN, a simple nginx instance (without any cache in front like Varnish) on a $5/month Digital Ocean server was enough to handle things.
- majewsky 5y ago
  Same for my site: https://xyrillian.de/thoughts/posts/latency-matters-aftermath.html https://xyrillian.de/thoughts/posts/latency-matters-aftermat...
- brundolf 5y ago
  You don't even need nginx; I've got a single Node process on a Heroku "hobby" node serving my site, and it's weathered several front-page visits peaking at 15 requests per second without problems. The important thing is the static-rendering (assuming it applies for your site's content)
  - mst 5y ago
    Having a blog post of mine get posted to HN (many years ago) was how I discovered I'd accidentally introduced a bug into my web server config that meant the app server behind it was serving the static assets as well as the dynamic stuff. It entered a state of wedgitude quite rapidly when traffic ramped up.
- robertlagrant 5y ago
  > making the web less decentralized In this case, this is not a virtue. Using a CDN is option, reversible and is likely to spread your content out to be much less centralised than anything hand rolled.
  - capableweb 5y ago
    Except when 20% of the web does that the web as a whole become less decentralized, while your files might still be further spread apart in the world. What I was thinking about was the web as a whole, not my specific files.
sildur 5y ago
Cloudflare is awful, I get tons of captchas while browsing with tor.

la_fayette 5y ago

I think the website is built with django, maybe it has some community stuff on it, which requires server side processing... Git repo: https://github.com/widelands/widelands-website https://github.com/widelands/widelands-website

deleted 5y ago
[deleted]

wutwutwutwut 5y ago

> I'm so surprised that static sites like this The person you replied to linked the github account and if you ckecked it you would see that it was not a static site. I am not surprised that you decided to write a comment instead though.

account42 5y ago
This is also obvious from looking at a cached version of the site: there are there is a list of online users and posts in the sidebar.
deleted 5y ago
[deleted]
maerF0x0 5y ago
downvoted for snark: https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

deleted 5y ago

[deleted]

INTPenis 5y ago

You were wrong technically but you have a point. These sites should leverage existing services like reddit or AWS. Personally I'd put a static information site on AWS and a community on Reddit if I was the creator for something like this game. It's just like SMB's today don't really need much infrastructure, well open source projects shouldn't either.

capableweb 5y ago
That's a horrible idea. Yes, open source should definitely get off closed and abusive platforms like Reddit or AWS. Really, the open source community should start dog fooding more. It seems like yesterday all the communities were on IRC and it was easy to find everyone as long as you were connected to Freenode. Now everyone is using Slack, Discord or other closed-sourced systems that are not even built for the use case of "open by default". It's really sad to see, especially when a lot of developers in the ecosystem are cheering on projects moving to platforms like reddit or AWS. To run your own infrastructure is not as hard as you think, and it gets a lot cheaper too.
- INTPenis 5y ago
  I completely get where you're coming from, I'm a staunch self hoster myself. But why re-invent the wheel before you have to? That's just a waste of time and resources that could be used to promote and build your project. Like in this very case, the website is now down and unable to promote the project because it was made in a traditional way. Imho it's enough to retain ownership of domains to really own a project. Like the widelands.org domain. So whatever happens with AWS, or reddit, that domain can always move and point visitors to new communities. In my personal selfhosting environment I have my own domain pointed to protonmail. So even if protonmail goes belly up I can always move my domain somewhere else. And even my domain is picked to give me maximum control. I use my own country's ccTLD because they have a good conflict resolution department that will benefit me as a citizen.
mst 5y ago
The forum's a more complicated question but for the static information site for an open source project that's already hosting its code on github, configuring a custom domain for the github pages site would seem even simpler than sticking it on AWS.

tomc1985 5y ago

Or they can just go down and people don't have to freak out. Stop recommending people join the CDN cartel

qnsi 5y ago
I guess people want their site to be available when it gets viral?
- tomc1985 5y ago
  By contracting with a private company known for shielding propagandists Totally in line with the OSS ethos.... not
  - mst 5y ago
    Being against censorship of even very stupid shit is absolutely in line with the old school hacker ethos. "The internet treats censorship as damage and routes around it" was absolutely a thing. [[shakes crutch at the kids on his lawn]]
    - tomc1985 5y ago
      I am generally against censorship but had to revise that stance after I realized we've given a global platform to manipulative idiots truly unworthy of global self-expression. If they want to espouse their crap they can stand on a real soapbox and shout at crowds, but nothing more
      - gameman144 5y ago
        Neat, and who decides whose "crap" is unworthy of freedom of speech? Like, my main objection with censorship isn't "I think that this particular idea is worthy of defending", it's that any mechanism that lets people censor bad opinions could also by used to censor good opinions, and that mechanism is way too powerful and prone to abuse for my comfort.

zild3d 5y ago

generally yes, but their site is also a forum (and some other things). Looks like its made with django

account42 5y ago
They could still make the landing page static or at least enable some (micro)caching - there is no reason for every request to hit the backend. Of course, this is obvious only in hindsight.