7 ms·
As people mentioned this site is an exception to how to do things, in that PG actively does not care about search engine results. However, for the people who ar
by tedivm 13y ago
As people mentioned this site is an exception to how to do things, in that PG actively does not care about search engine results. However, for the people who are interested, here's a few ways you can handle a situation like this.
1. If you add the "max-stale" header to your content you can tell your CDN to hold on to things longer in the event of an outage. What happens is that the CDN will look at the cache time you set and check in as normal when that expires, but if it gets back no response or a 503 code (server maintenance status code- if you work with CDNs this is your friend!) it will continue serving the stale content for as long as you tell it.
2. Lets say you're beyond that, or your site is too dynamic. Instead of setting up an error page that responds to everything, setup a redirect with the 302 status code (so crawlers and browsers know it's a temporary thing). Point that redirect at an error page and you're golden. The best part is these types of requests use minimal resources.
What I do is keep a "maintenance" host up at all times that responds to a few domains. It responds to all requests with a redirect that points to an error page that issues the 503 maintenance code. Whenever there's an issue I just point things at that and stop caring while I deal with the problem. I've seen webservers go down for hours without people realizing anything was up, although with dynamic stuff everything is obviously limited. The other benefit to this system is that it makes planned maintenance a hell of a lot easier too.
Oh, another thought- you can use a service like Dyn (their enterprise DNS) or Edgecast to do "failover dns". Basically, if their monitoring systems notice an issue they just change your DNS records for you to point at that maintenance domain. You can also trigger it manually for planned things.
- username223 13y ago"actively does not care about search engine results. " Or data -- they tossed a day or so of submissions and comments when restoring from backup, AFAICT. I'm not saying that's a problem; 4chan loses data and works just fine.
- n09n 13y agoThat's a poor comparison. Users expect 4chan to lose that same data as part of normal operation anyways, so you can hardly consider it a problem. That's not the case for HN.
- ealexhudson 13y agoSearch result ranking is an obvious casualty, but it's not the only one. My Firefox start page (as an example) now has a cached but empty version of the HN pages that doesn't really look quite right. Any system that pulls content automatically, like that, or an offline reader, etc. is going to get confused. It's simply good practice to get the HTTP status codes correct in the ways you outline.
- danielweber 13y agoHN was down much longer than I thought it was because the bad pages were stuck in caches. I had to do a force-reload to see that it was back up this morning.
- thirdsight 13y agoSimilar problem on my phone and you can't Ctrl+F5 to force a reload on this device.
- andyroid 13y agoYou can add a random query parameter to the url to force a reload if the option isn't given by other means.
- thirdsight 13y agoGood point. Didn't think of that!
- tracker1 13y agoon some phones holding shift and enter will work for a hard refresh... (it's awkward to say the least)
- oneeyedpigeon 13y agoIt was also down for much longer than a lot of services (e.g. status checkers) thought, since their only sensible method of working is to inspect the HTTP status. The result being that HN uptime looks better than it really is; of course, this isn't a paid service or something mission-critical, so exaggerating uptime doesn't really make much difference.
- ErsatzVerkehr 13y ago> As people mentioned this site is an exception to how to do things, in that PG actively does not care about search engine results. Is pleasing Google now the only reason to obey the HTTP spec?
- vog 13y agoUnfortunately: Yes. For example, I had a hard time telling people that they should use a proper HTTP redirect from their domain "example.com" to "www.example.com" (or vice versa), instead of serving the content on both domains. All arguments about standards, best practice etc. were unconvincing. But since Google startet to punish "duplicate content", I never had problems convincing people again.
- tomkin 13y agoThis a strange reaction. I agree that perhaps "this is different", but I am hoping I can assume that the next time I see an article on HN, about another website going down, or losing their data, that the HN community shows as much compassion and understanding. In my experience, when not related to the work of pg, there tends to be a lot of "you should have done X and there is no excuse for not doing Y!!" If I'm being honest, it's off putting to watch community-wide apologist response to what would normally be outrage of poor execution.
- gtirloni 13y agoI think it's an expected response from a community that is largely comprised of people that desire YC-bucks. But you nail it. Had Gmail, Reddit or any other unpaid service gone down like this, the uproar would be heard from another galaxy.
- tedivm 13y agoPeople depend on email. This is just a link aggregating and discussion site. I think people should have some sense of perspective here.
- DanBC 13y agoHaving spent months telling people that email is not reliable and can fail at any minute and must not be relied on, and then having to put up with the fall out when the shitty provider broke something, I can confirm that people get angry when email breaks.
- ghshephard 13y ago"email is not reliable and can fail at any minute and must not be relied on," In what universe? If email were to stop working in most major corporations that I've worked in for the last 8+ years, the company would basically come to a halt. Email, for many, many companies is the message/workflow bus, and if it stops - communication comes to a halt. It is, after electricity, and the network, the one essential function in a company in 2013. Telephones, Photocopiers, Printers can all cease functioning , with little impact in most technologies companies - but not email.