14 ms·

Browsers barely care what HTTP status code your web pages are served with

• voz_ 3y ago
  The internet of today is awesome, it largely just works. Mountains out of molehills people.
• NoZebra120vClip 3y ago
  The web today is full of silent failures. I've noticed this all the time at work and in my personal business. If loading some resource runs into trouble, it's simply not feasible for the Node.js or React or PHP to surface an error to the user. Chris says that the loading of sub-resources is a different matter. Well, the most common silent failure I have is not due to status codes per se, but because of my DNS-based adblocker, many sub-resources fail to load (GET or POST) with a network error (TCP RST). Unless a blocked domain corresponds to the front page I visit in a URL, the unavailable resource represents a silent failure, and I have foolishly contacted Support desks multiple times because my own use of the adblocker caused a silent failure that was difficult to diagnose. In fact, I now have no way of knowing what web apps are like without an adblocker. Because all my devices use the same configuration, and it's non-trivial to disable it, I simply don't know what I am missing. Indeed, in the case of one web app which I've used as a PWA, it was simply missing features that I knew should have been there, and whitelisting enough domain names caused those features to re-appear. I will just leave you with the most adorable 404 page anyone has ever seen: https://diopitt.org/pray/amen https://diopitt.org/pray/amen
  • oneeyedpigeon 3y ago
    You'd like to think that the owners of those sites would fix them. A site shouldn't just not work if it can't access a certain JavaScript file - it should at least warn you of the problem.
  • mjan22640 3y ago
    What dns adblock do you use?
    • namtab00 3y ago
      not OP pihole as cache, with the router as its upstream, with NextDNS as the router's dns server
      • OJFord 3y ago
        What's the point of the PiHole in that setup? If the router doesn't have the storage for as large a cache you're still only using the dnsmasq part of PiHole by the sounds of it? (Not to mention most consumer routers support USB mass storage?)
    • NoZebra120vClip 3y ago
      NextDNS. They have a feature where if you're a free user, and you exceed your query quota for the month, they turn off all features and it acts like a normal resolver. For some reason, there is no access to such a setting to do it manually, and I am a subscriber now, too. There are profiles that can be configured separately, but even just to activate an alternate profile, one must reconfigure the DNS settings on the devices.
  • bhelkey 3y ago
    > Unless a blocked domain corresponds to the front page I visit in a URL, the unavailable resource represents a silent failure Is this not intended behavior? Were you not expecting ads to be silently removed?
    • sclu1034 3y ago
      From the user's point of view, yes, that is intended behavior. But the browser can't know that, all it sees is a dropped connection. For all it knows, there could be an actual issue with the network. But unless you open the dev tools' network tab, you don't know about that request failing.
      • NoZebra120vClip 3y ago
        TCP RST is indeed considered a transient failure, and the RFCs specify that it should be retried several times before giving up. My ad blocker returns "0.0.0.0" for A records, which in practice means attempts to connect to 127.0.0.1, which isn't running a server of any kind on port 443/tcp or 80/tcp. So a standards-compliant client would really hang awhile, or hold rendering, until several connects have been attempted and they all failed.
  • radicalism 3y ago
    > I will just leave you with the most adorable 404 page anyone has ever seen: https://diopitt.org/pray/amen https://diopitt.org/pray/amen In the context of your comment, I was amused by how my own ad-blocker failed to properly show the (image on the) 404 page, since it is loaded from a different domain :D
  • daveidol 3y ago
    > Because all my devices use the same configuration, and it's non-trivial to disable it This is exactly why I do not use a DNS-based ad blocking solution (it’s trivial to disable uBlock Origin and refresh if something feels broken).
    • IggleSniggle 3y ago
      This is why I run my DNS-based ad blocking solution through a VPS. It's trivial to toggle the VPN on/off on desktop or mobile. On desktop I have a single key to swap my network. On mobile, a gesture followed by two taps does the same.
    • jondwillis 3y ago
      I’m not sure if we are talking about PiHole here, but I just log into the web interface and whitelist or temporarily disable blocking. Not as trivial as a browser extension, but still quite easy and fast.
  • kaoD 3y ago
    > If loading some resource runs into trouble, it's simply not feasible for the Node.js or React or PHP to surface an error to the user. In what way? Do you mean not feasible or just lazily coded?
    • methodical 3y ago
      Also was confused by this sentence. I have worked on several Node, React, and even PHP (albeit a long time ago!) applications, and every single one of them caught and surfaced internal server errors. Never was aware there were actually people out there regardless of language who weren't catching and surfacing 500s in a straightforward fashion to the user.
  • account42 3y ago
    Browsers used to display an icon and beveled borders to indicate broken images. Not sure when they stopped but I wish they hadn't.
    • nullindividual 3y ago
      They still do. The problem is, as always, JavaScript and how the assets are loaded and images rendered on pages.
  • dredmorbius 3y ago
    "The Web is an error condition": <http://deirdre.net/programming-sucks-why-i-quit/ http://deirdre.net/programming-sucks-why-i-quit/> See also Richard I. Cook's "How Complex Systems Fail": <https://how.complexsystems.fail/ https://how.complexsystems.fail/> (Latter is an HN perennial, with 55 sumbissions presently: <https://hn.algolia.com/?q=how%20complex%20systems%20fail https://hn.algolia.com/?q=how%20complex%20systems%20fail>. Most recent 10 months ago, so not quite ready for a re-up: <https://news.ycombinator.com/item?id=32895812 https://news.ycombinator.com/item?id=32895812>.)
    • lcnPylGDnU4H9OF 3y ago
      The first one is a 404. I love the irony of there being an error condition in this case of a missing redirect. https://deirdre.net/2014/programming-sucks-why-i-quit/ https://deirdre.net/2014/programming-sucks-why-i-quit/
      • dredmorbius 3y ago
        I was being lazy and had copied the URL from an earlier comment of mine mentioning it. The Internet Archive has a copy as well: <https://web.archive.org/web/20200922230737/https://deirdre.net/programming-sucks-why-i-quit/ https://web.archive.org/web/20200922230737/https://deirdre.n...>
        • lcnPylGDnU4H9OF 3y ago
          Even their blog has such dead links. I just happened to notice that other URLs included the year, so I tracked that one down to see if that’s all what was needed. Of course, forgetting about that little backwards compatibility is easy, especially considering it’s a personal blog and they likely have other personal things to worry about. I was just amused by the irony in context. I otherwise appreciated the read. :)
          • dredmorbius 3y ago
            If you happen to remember any of those, drop her a line. I'm pretty sure Deirdre's fairly meticulous about that sort of thing.
• janosdebugs 3y ago
  Does anyone still remember the days when a certain browser would ignore what the server sent with a 404 page unless it was larger than 512 bytes? Yeah, I'm happy with the browser just showing what I send it without trying to second guess the content.
  • Dylan16807 3y ago
    Oh yeah, I still have a 404 page that's padded with comments because its only contents are a title and an image.
    • oneeyedpigeon 3y ago
      I hope that image has good alt text...
    • Jiocus 3y ago
      base64 inline your image with the HTML and you should be well past that threshold. It's preferable to not make additional requests (like for an image) on an error page - in case the requested server is at fault - breaking the error page.
      • javajosh 3y ago
        Fun fact: did you know you can inline an image without base64 encoding it? Use svg you can inline the markup using a data url without base64 encoding. This is how I do my favicons at simpatico[0]: <link id="favicon" rel="icon" type="image/svg+xml" href="data:image/svg+xml, <svg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 1 1'> <rect width='1' height='1' fill='DodgerBlue' /> </svg>" > What's nice is that this opens up the possibility to generate dynamic favicons really easily using only string interpolation. 0 - https://simpatico.io/svg.md https://simpatico.io/svg.md
        • Jiocus 3y ago
          Yes, it makes a lot more sense to use svg when the image content is a good fit for the svg format, like most icons and line-drawn graphical elements. Thanks for the tip, and the many examples on your site!
• asplake 3y ago
  To me, none of this is surprising or undesirable - ie the browsers are doing the right thing. Content is there to be rendered, not hidden. Status codes are not there for the average end user.
• makeitdouble 3y ago
  This feels surprising for a minute until comes to mind all the little things that would be broken if browsers didn't behave that way. Custom 404 pages to give the user a friendly explaination and further directions, 500 pages with the exception stack trace in our frameworks to debug the errors etc. A status different from 200 rarely means there's nothing that could be displayed to the user.
  • acumenical 3y ago
    I agree. The way I see it is that browser is more of an HTTP response viewer than a REST client.
• entuno 3y ago
  I remember a talk years ago where someone had written an Nginx plugin that returned a random status code (from a list) for every request. Normal users would never notice and could browse the site fine - but bots (and particularly vulnerability scanning tools) couldn't cope with it and would show all kinds of weird results and false positives/negatives.
  • chii 3y ago
    security by obscurity only stops casual bots, and any determined bot would easily be able to ignore the status code and inspect the contents to get at what they wanted anyway. And you run the risk of having a transparent proxy fail on your real users doing this too. It's why nobody does it.
    • entuno 3y ago
      It wasn't really intended as a security feature - more a way to screw around with script kiddies.
    • tredre3 3y ago
      > security by obscurity only stops casual bots Casual bots is what 95% of website operators have to deal with, blocking them is still a win. Same reason people move ssh ports around. Of course it doesn't add security if you're targeted, but it makes casual bruteforce bots go away. Call that log hygiene if you really object to the security aspect.
  • misswaterfairy 3y ago
    This one? https://www.youtube.com/watch?v=4OztMJ4EL1s https://www.youtube.com/watch?v=4OztMJ4EL1s
    • entuno 3y ago
      Yes - that's the one. I remember the slides showing the very variable results for the vuln scanners running against it.
• _nalply 3y ago
  This is the cause of a split in failure situations. Failures fall into two often very distinct cases: 1. Failures the end-user is able to understand and in some circumstandes to fix themselves. Examples: (HTML) page not found, no connection, etc. 2. Failures the end-user is in no case able to fix; it's up to the programmer. Examples: Server misconfigured, missing media files on server, etc. It never makes sense to display the full error messages in case 2. It suffices to hide the error and perhaps make it available in DevTools. Often a good way is: give a generic message, and because some failures are transient, tell the user to retry. Also it can make sense to add an error number so that the end-user can give support the error number. So it is useful to let the browser be lenient with HTTP status codes except in some specific cases which the article does mention (HTTP status code 500 for JavaScript files, for example).
• masswerk 3y ago
  I think, this is a general design feature. The specs explicitly allow a possible document body. If the status doesn't affect the process of retrieving the requested document (like a redirect), a client is free and even expected to display the document body. Having said that, if the status code is in the 4xx client error range, I'd wish that any document.title would be strictly ignored and browsers would always display the status code (together with the human readable error title) as the page title.
• onion2k 3y ago
  (I'm not sure there's any way to find the HTTP status code in a modern Firefox environment short of using web developer tools. It's not in places like 'Page Info' as far as I can see.) I'm old enough to remember times before browsers showed 'friendly' error messages, and a 500 response from a server would display a '500 Server Error' message to the user. Users hated it. They'd panic and think they'd broken the website somehow. It was horrible from a support perspective, and if the user was new to the web it sometimes made them very anxious about using the web in the future. Users have no business seeing error codes. Those are for devs. Hiding them away in devtools is a very good thing.
  • falcor84 3y ago
    When did we stop building software for the "Power User" persona? I don't disagree that there are some (many?) people who start screaming and running in circles when they see an error code, but not every user is like that. You don't have to be a dev to care any the distinction between a system doing its job and silently failing. Why not have some indication that there's smoke coming out from the engine?
    • Raed667 3y ago
      Opening the dev tools and looking at the network tab is easy enough for power users.
      • bmacho 3y ago
        It is not working in Firefox or in Edge though, only if you already had your dev tools open before you navigated to the website.
        • rhaway84773 3y ago
          Create a chrome shortcut that auto opens dev tools for you. You can replace your regular chrome with this instead. https://developer.chrome.com/docs/devtools/open/#auto https://developer.chrome.com/docs/devtools/open/#auto
• 22c 3y ago
  Servers barely care what HTTP status code their users are served with. 500 / 503 seems to fit any and every error these days, or the classic 200 OK for a page not found.
  • mrweasel 3y ago
    Or you can take the route that a former client took: Return 200 OK http status code and just print "500" in the response body. Took me a few minutes to figure out why the monitoring kept insisting that everything was OK when my browser kept showing me "500".
• hooby 3y ago
  > It's a convention (and a useful one) that the HTML served for a web server's error pages will include the HTTP status code in the text (and often the <title> as well). But it's only a convention and it can be violated, both accidentally and deliberately (both in omitting the status code and listing the wrong one). If it is violated, you probably won't notice for a while (if ever). A malicious server can just as easily send the wrong status code, as well as listing the wrong status code in the accompanying html - so if the browser did show the real status code more prominently to the user, that wouldn't actually help in such a case, or would it?
• nickdothutton 3y ago
  This is the web equivalent of “no user serviceable parts inside”. Watch Joe Average use a computer. If he’s shown an error either he recoils as if he’s physically broken it (or the website) or repeatedly just performs the same action again and again like a chicken pecking at a corn dispenser, or finally mashes the keyboard with his fists and slams the laptop shut in frustration. The days of a bespectacled web user, thoughtfully musing over an error message, while fiddling with a self-propelling pencil, sat at a Sun/SGI/HP/IBM X11 session are long gone.
• bawolff 3y ago
  This is silly. Web browsers handle the status code correctly by displaying the error in the response body. 500 isn't for end users - it is for web crawlers and caching proxies.
  • OJFord 3y ago
    What's silly? The article just describes the way it is, there's no judgement or call to action?
    • bawolff 3y ago
      It implies (especially the headline) that there is something wrong with that or it is at least unexpected.
• jiggawatts 3y ago
  The author is wrong, browser behaviour does change significantly in one major way: errors are not cached. Or much more importantly, errors incorrectly returned with 200/OK success codes are cached, semi-permanently breaking your site. If you've get helpdesk staff telling customers to "clear their browser cache", this is one common reason why this "works" as a fix. Please, I beg you: stop being "nice". Stop catching exceptions and dropping them on the floor! Let them bubble up to the web framework pipeline so that an error code is sent. Your CDN will thank you. Browser caches will thank you. Corporate proxy servers will thank you. Application monitoring tools will thank you. Load balancers will thank you. I kid, I kid. None of those will send you thanks in an email. Instead, they'll work tirelessly in silence to deliver your application as intended, instead of holding on to transient errors for days on end and upsetting your users.
  • kaoD 3y ago
    Also: search engines. I spent a good chunk of a month fixing an app initially developed with this mindset. Immense value delivered just by making things work as they should've from the beginning.
  • theginger 3y ago
    Letting an uncaught exception error reach the user is generally bad. It risks exposing secrets / confidential information to the user and any returned client side secrets risk getting into intermediary caches. The biggest risk is the user input being returned in the response creating an XSS (cross site scripting) risk. This is a very common source of reflected XSS vulnerabilities. You should always catch exceptions like this so you can at the very least sanitize them for XSS. This is already built in to some frameworks. If you are using something that doesn't you need to do it yourself and it's probably easier to just strip it all in production, you just need to make sure you are still setting an appropriate status code.
    • jiggawatts 3y ago
      Who said anything about letting exceptions reaching the users!? The exception needs to bubble up to the request pipeline, which will set an integer error code, and then... redirect to a generic "sorry we broke" error page. Showing the full exception stack trace and other sensitive bits and pieces is for local debug mode, not for production.
• theginger 3y ago
  Just because they rendered the text doesn't mean they've ignored the status code. There can be a lot of other stuff going on behind the scenes that are happening differently due to this code. A 503 with a Retry-After header is a good example, although it has limited browser support, but in theory it will reload the page after the timeout. There are different caching/storing rules with different status codes, they may also disable certain features, eg if you have been loading a page with an error you might find the auto complete feature in the URL bar doesn't work. I often find this is really annoying when building something new and haven't got it quite working, I keep needing the full URL, where as if it had just once given me a 200 I could just type a few letters and it would come up, but it probably far more useful that auto complete doesn't get polluted with mistyped URLs and broken websites.
  • richardwhiuk 3y ago
    "but in theory it will reload the page after the timeout" - does any browser actually implement this?
    • Jiocus 3y ago
      Page reloads doesn't seem to be an anticipated or suggested feature for this header, irrespective of error code. Communicating API rate-limiting (429 Too Many Requests) or temporary downtime (503 Service Unavailable) to other services and bots seems to be it's most use.
• PascalPixel 3y ago
  With my browser I naively implemented that anything but (200) would result in an error page Boy the reports that my browser “wasn’t ready for prime-time” as a result… It now only shows if the domain can’t be resolved, people are happy
• jgalt212 3y ago
  We use "correct" status codes to communicate to spiders and scrapers that "we're on to you".
• jimmaswell 3y ago
  > if you respond to a request for a web page with a HTTP 200, 4xx, or 5xx code outside of these specific ones and some HTML, almost all browsers will display the HTML to the user Chrome outright refuses to show the HTML of a 500 response in favor of its own dinosaur animation or some garbage, or did that change?
• m3047 3y ago
  In other news, web servers barely care whether that alleged HTTP/1.1 request contains a Host: header...
• javajosh 3y ago
  Once upon a time I wrote a feature request for the Chrome extension team to allow an extension to change the response code of a request. This would be a real boon to testers wanting to see the behavior of the browser under those conditions without having to fiddle with the server. But the Chrome team closed it as unneccessary or something. That was a mistake.
• otikik 3y ago
  They also don’t care about the HTTP headers, or the latency. All that belongs in the developer view.
• nomadspiritnet 3y ago
  I use a budget mobile carrier and I noticed that it blocks http access to most websites, even with linux updates.
• throwawaaarrgh 3y ago
  Browsers are just shit user interfaces. They've never prioritized the user experience, nor actually integrated good solutions to common status problems. Designers even actively avoid common widgets because they're so bad. Every once in a while they remove widely used functionality from the UI or bury it behind a lot of other crap, so someone has to invent an extension to bring it back. It's terrible. All the alternatives have varying degrees of suckitude, and there's no way to just make an alternative browser without like $1.5 million dollars and two years of development.
• bitlax 3y ago
  https://en.m.wikipedia.org/wiki/Robustness_principle https://en.m.wikipedia.org/wiki/Robustness_principle