19 ms·

API Shouldn't Redirect HTTP to HTTPS

• akira2501 2y ago
  > Servers can now send HSTS along with the initial HTTP-to-HTTPS redirection response > Node.js's built-in fetch happily and quietly followed those redirects to the HTTPS endpoint. Okay.. does nodejs fetch respect HSTS?
  • Aachen 2y ago
    I'm not even sure I'd find it desirable for nodejs fetch() to quietly store state somewhere on my server without asking me: I wouldn't know to back that file up, it may be trashed regularly depending on the infrastructure, it could mess with version control by creating a local working directory change, or it might run into an error condition if it is on a read-only filesystem (either crashing or being unable to use this security feature, neither is great). Config file writes are to be expected for user-facing software, but for developers it should error out so they can just choose what it needs to do E.g., "Error: HSTS response received after a redirect from HTTP, but no storage location specified for security states. Use the https:// https:// protocol, specify a file location, or set insecure_ignore_hsts=true." Edit: but it's a very legitimate question?! Just saw you got downvoted, I have no idea why
    • moralestapia 2y ago
      >Just saw you got downvoted, I have no idea why I've noticed a gradual increase in this behavior during the past ... year maybe? I think for a lot of new people, downvoting equates disagreeing, which is not ideal. Although, I also have no idea why someone would disagree with a neutral question like GPs, lol. Hopefully, it's not the beginning of the end for HN as it is a great website.
      • iJohnDoe 2y ago
        These types of downvotes also discourage discussions. I’ll upvote a comment when it has been downvoted if it has a constructive discussion thread.
    • hn_throwaway_99 2y ago
      I think the original question "Okay.. does nodejs fetch respect HSTS?" goes into the "not even wrong" bucket, for the reasons you point out. HSTS really only makes sense from a browser perspective (or, rather, a "permanently installed, stateful client" perspective). For an API like fetch it doesn't even make sense as a question IMO.
      • moralestapia 2y ago
        >For an API like fetch it doesn't even make sense as a question IMO. Why not?
        • hn_throwaway_99 2y ago
          Because the way HSTS works fundamentally implies a stateful client, and because it was fundamentally created to solve a human problem (i.e. humans entering in URLs directly in the address bar). It really doesn't make sense with a non-stateful client, and it isn't intended for cases where someone isn't manually entering in the URL to hit. E.g. fetch is often loaded in transient situations - it really shouldn't be updating its own config because of responses it gets. Also, based on the other comment I was originally thinking it would be good if fetch would just error out if it gets a redirect from http -> https AND also a Strict-Transport-Security header, but in that case it would still mean it was dependent on the server setting up the STS header, and if they could do that they should just go ahead and get rid of the http -> https redirect in the first place.
          • moralestapia 2y ago
            I agree with you on things like CORS, but HSTS would actually solve the problem stated in this thread quite gracefully. Client fetches http, notices the header, retries https then ... ok no, lol. But I guess it's still of some use to let clients know "this should always happen through https" and make them fail if that's not the case. Edit: yeah I got it, client fetches http, notices the header and then explicitly fails because HSTS is there.
      • Aachen 2y ago
  • NewJazz 2y ago
    I'm not aware of any general programming language http clients that honor HSTS.
    • pzmarzly 2y ago
      libcurl supports HSTS, but the client has to specify a file where the state should be stored https://curl.se/docs/hsts.html https://curl.se/docs/hsts.html Many languages/libraries use libcurl in some shape or form, but whether they set up an on-disk HSTS store or not - I don't know either.
  • g15jv2dp 2y ago
    How would that even work? It's up to the developer to consider the response and act correctly on it.
    • Aachen 2y ago
      Same way as TLS session resumption can be handled by libraries without you having to touch it, or perhaps requiring you to specify a storage file and taking it from there
    • andersa 2y ago
      Just do it like a web browser - when you install Chrome it comes with a list of many tens of thousands of domains that had HSTS set when GoogleBot visited it.
    • akira2501 2y ago
      So if you occasionally forget and use http when you meant https and are worried about the consequences of that, you should just implement your own HSTS checking layer? Why not just implement your own fetch wrapper that throws if it's not an https connection?
      • g15jv2dp 2y ago
        > So if you occasionally forget and use http when you meant https and are worried about the consequences of that, you should just implement your own HSTS checking layer? Or use a library to do it. The core fetch functionality shouldn't have to deal with HSTS. There may be legitimate reasons to fetch over HTTP even after you received an HSTS header - for testing purposes, for example. > Why not just implement your own fetch wrapper that throws if it's not an https connection? That's the developer dealing with HSTS.
• medellin 2y ago
  I fully support this and have always pushed for this. One because it becomes a huge mess to maintain over time but also because it long term will lower traffic through the LB. Unfortunately what i see happen all the time is quick fixes are pushed to the infra. For example they deploy and typo the URL. Now we have a prod outage and infra is pulled in to fix this asap. No time to wait for that 10 minute deploy pipeline that requires all the tests to run and a deploy to dev. This happens once and then infra is asked why we don’t already redirect all URLs. Management doesn’t care about security and they just lost money. Guess what you are doing now. This is the world we live in.
  • blowski 2y ago
    Indeed. It’s probably why so many APIs accept the api key in the URL.
• superkuh 2y ago
  Or better: actually provide the API on HTTP and HTTPS if your use case allows it (ie, non-commercial/institutional, just something for human people).
  • x86a 2y ago
    I don't think this is ever a good idea. Even for non-enterprise use cases, you wouldn't want some public hotspot to be able to inject random garbage into responses, even if not done with malicious intent.
    • Wowfunhappy 2y ago
      • It allows retro computers to connect. • It allows very low power embedded devices to connect without extra overhead. • It's not a real security concern if you're on a private network.
      • rascul 2y ago
        > • It's not a real security concern if you're on a private network. I'm not convinced that private networks should be assumed secure by default.
        • TeMPOraL 2y ago
          It's definitely not improving security when, in order for a website to interact with an API that both are hosted on my private network, possibly even on the same machine, I need to set up publicly accessible DNS entries and/or hosting my own resolver. That and CORS makes local-first anything a huge PITA.
        • Control8894 2y ago
          Perhaps, but the other realistic option is a self-signed cert. Since browsers refuse to implement any kind of TOFU or otherwise 'trust history', a self-signed cert is pretty much exactly equivalent to no TLS at all.
        • 2y ago
• justin_oaks 2y ago
  I appreciate the author calling this out because creating an HTTP-redirect-to-HTTPS is something I'll do almost without thinking about it. "If it has HTTPS, I'll set up an HTTP redirect." Now I know that I need to think about it before setting that up. It also made me realize that cURL's default to not redirect automatically is probably intentional and is a good default. Praise be to Daniel Stenberg for this choice when implementing cURL.
  • deleted 2y ago
    [deleted]
  • tootie 2y ago
    Using Cloudfront, the redirect was the only built-in option for a long time. They only added pushbutton HSTS recently. But I'd say author is correct that if you're hosting an API there's no reason to support http at all. Just send a 400 on all requests and let the client developers use common sense.
    • tracker1 2y ago
      You can still return a response... 400 - HTTP is unsupported, use HTTPS.
  • koromak 2y ago
    Yeah I completely missed this as a security flaw. Time to go deploy a fix...
  • tracker1 2y ago
    I tend to use Caddy as a reverse proxy for personal projects... The default behavior is to redirect to https. May have to make a special rule for API instances.
• lxgr 2y ago
  Completely agree, and arguably why stop at API servers? Depending on server-side HTTP -> HTTPS redirects for security reinforces/rewards bad practices (linking to HTTP, users directly entering HTTP etc.), in a way that makes users vulnerable to one of the few remaining attack vectors of "scary public Wi-Fis".
  • Aachen 2y ago
    We are. Slowly, due to lots of legacy, but surely getting there. See the small steps over the years where it was first an add-on to force https-only mode (HttpsEverywhere, 2011), then browsers started showing insecure symbols for http connections (e.g. in 2019: https://blog.mozilla.org/security/2019/10/15/improved-security-and-privacy-indicators-in-firefox-70/ https://blog.mozilla.org/security/2019/10/15/improved-securi...), and more recently I think browsers are starting to try https before http when you don't specify the protocol. I've also seen a mention of strict https mode or something, not sure if that's a private navigation feature or something yet to come, but warning screens equivalent to insecure certificate pages are getting there
    • Dylan16807 2y ago
      Chrome's version of trying https first sure is annoying though. If a site is down entirely, when chrome can't connect to port 443 it confidently declares that "the connection is not secure because this site does not support https" and gives a "continue" button. Then when you click "continue" nothing happens for a while before it finally admits there's nothing responding at all. So it gives a misleading error and takes longer to figure out if a site is actually down.
      • jraph 2y ago
        I'm highly surprised by this. It seems very dumb and I have never seen anything like this, though I never use Chrome and very rarely fire Chromium for testing something. Is there something to read about this, like a dev ticket?
        • 2y ago
• snowwrestler 2y ago
  HTTPS and SVCB DNS records will hopefully make it more feasible over time to drop the traditional HTTP server-side redirect. The client agent will be able to read the DNS record and upgrade to the highest available protocol prior to sending the first request.
  • quectophoton 2y ago
    Seeing how web browsers haven't added support for SRV in decades, I'm not holding my breath. I'm guessing there's some (?) advantage of SVCB and HTTPS records that can't be achieved with SRV[1] and TXT records, but I haven't read the RFC yet to know what that advantage is. [1]: `_https._tcp` or `_http3._udp` or whatever.
• deleted 2y ago
  [deleted]
• Gigachad 2y ago
  Or just add your domain to the hsts preload list and never have to worry about this.
  • yjftsjthsd-h 2y ago
    Is the HSTS preload list used by anything other than browsers? I'd expect it to be minimally useful for an API.
  • yegle 2y ago
    That works for browsers but I doubt any non-browser HTTP clients (e.g. curl and wget) or HTTP library (e.g. Python requests lib) will check the HSTS preload list. In fact if they do follow HSTS headers, a simple `Strict-Transport-Security: ...; preload` would have fixed the issues mentioned in the article.
  • cqqxo4zV46cp 2y ago
    Please look into the myriad scenarios in which HSTS is not honoured. As usual, any comment stating that people should “just” do x, is wrong.
  • nimih 2y ago
    Did you happen to RTFA, in which the author specifically mentions HSTS preloading--helpfully styled as a bold, underlined, bright blue link--in the second paragraph? If you manage to then get to the third paragraph, a concise and compelling reason is given for why it's not applicable in the scenario the author is examining.
• iJohnDoe 2y ago
  Sort of off-topic. What is a recommended way to sell access to a one-off data API? Low code method to control access and facilitate payment?
  • Aachen 2y ago
    As in, selling API keys? Not sure what you're asking for. Are you looking for a webshop that has API key sales as a default item type or something?
    • iJohnDoe 2y ago
      Yes, more like a SaaS. Maybe a solution that is tailored to selling API access. Generates a unique URL to the user, or API key, after they sign up for the API service.
• winddude 2y ago
  That's an excellent point, and definitely something I've done without thinking about it. I'm going to stop, and disable those. Thanks!
• blahyawnblah 2y ago
  Don't have HTTP available at all
  • lxgr 2y ago
    That's literally what the article suggests: > A great solution for failing fast would be to disable the API server's HTTP interface altogether and not even answer to connections attempts to port 80.
    • davedx 2y ago
      It also says > We didn't have the guts to disable the HTTP interface for that domain altogether, so we picked next best option: all unencrypted HTTP requests made under /api now return a descriptive error message along with the HTTP status code 403. So close and yet … their misconfigured clients will still be sending keys over unencrypted streams. Doh
      • hn_throwaway_99 2y ago
        > So close and yet … their misconfigured clients will still be sending keys over unencrypted streams. Doh And how does disabling the HTTP interface altogether prevent that? In that case, any sensitive credentials are still already sent by the client before the server can do anything.
        • whoopdedo 2y ago
          Not if the server refuses a connection on port 80. addendum: How quickly can a server write a 403 response and close the connection and can it be done before the client is able to write the entire HTTP request? My guess is not fast enough.
          • lxgr 2y ago
            I'd be surprised if most HTTP clients even looked at the response code before writing at least the entire HTTP request (including authentication headers) out to the TCP send buffer. And if they do that, even if they immediately close the TCP socket upon seeing the 403, or even shut down their process, I believe most socket implementations would still write out the queued send buffer (unless there's unread inbound data queued up at the time of unclean socket close, in which case at least Linux would send an RST). And with "TCP fast open", it would definitely not work.
• sdsd 2y ago
  My personal website (darigo.su) doesn't have HTTPS. I just deployed it a few months ago and haven't really done much with it yet. I guess I'll have to get around to it eventually, but I find charm in small old sites that haven't implemented modern protocol stuff. My site also uses <font> and <center> tags all over the place. Maybe I'll do some more quirky anachronisms, like only serve the site via HTTP 1.0 or something. Who knows. Since my site has very little functionality, it doesn't really matter, it's just for fun.
  • layer8 2y ago
    That’s fine, but rather unrelated to the article, which is about the situation that you have an API served via HTTPS, and the question of whether you should also have a redirect from HTTP to HTTPS in that case, or rather return an HTTP error.
    • sdsd 2y ago
      Yeah, I should have clarified. Nothing to do with the article really, just a random thought. Sorry if too off-topic!
  • Aachen 2y ago
    I can appreciate this and also run a service that is neither meant to be commonly visited (imagine a tor exit node landing page explaining what this IP address is doing, but for a different service) nor will produce or ingest sensitive information For a personal website that people might commonly want to visit, though, consider the second point made in this other comment: https://news.ycombinator.com/item?id=40505294 https://news.ycombinator.com/item?id=40505294 (someone else mentioned this in the thread slightly sooner than me but I don't see it anymore)
  • cqqxo4zV46cp 2y ago
    A HTTP website presents an opportunity for an attacker to MITM a payload that is ultimately executed in a user’s browser. Beyond ‘getting a moustache tattoo on your finger’ quirkiness, HTTP-only websites are really inexcusable beyond some very niche cases.
• dfabulich 2y ago
  The author includes a surprising response from "Provider B" to the HackerOne report. > Provider B: Reported on 2024-05-21 through their HackerOne program. Got a prompt triage response, stating that attacks requiring MITM (or physical access to a user's device) are outside the scope of the program. Sent back a response explaining that MITM or physical access was not required for sniffing. Awaiting response. I think Provider B morally should require HTTPS, but it really surprises me that the author would say "MITM or physical access is not required for sniffing." Is that true? Isn't HTTP sniffing an example of a MITM attack, by definition? Am I using the words "MITM" or "sniffing" differently from the author? I'm familiar with the following attacks, all of which I'd call "MITM": 1. Public unencrypted (or weakly WEP encrypted) wifi, with clients connecting to HTTP websites. Other clients on the same wifi network can read the unencrypted HTTP packets over the air. 2. Public encrypted wifi, where the attacker controls the wifi network (or runs a proxy wifi with the same/similar SSID,) tricking the client into connecting to the attacker over non-TLS HTTP. 3. ISP-level attacks where the ISP reads the packets between you and the HTTP website. Aren't all of these MITM attacks, or at the very least "physical access" attacks? How could anyone possibly perform HTTP sniffing without MITM or physical access??
  • Aachen 2y ago
    Definition question. I can see your reasoning and I can see the author's, where they define MITM as requiring an active component being in the middle to actually tamper with it and not just some far-off device receiving backscatter with a high-gain antenna, or a read-only mirror port on a switch or whatever is technically not "in the middle" but in a cul-de-sac. I'm not sure I've got a strong opinion, claiming one or the other is the only correct definition might just be nitpicking They may have chosen this wording ("it's not MITM") to get the team into action rather than dismissing the risk Edit: another legitimate-sounding question downvoted in this thread without further comment (since I'm the only comment still after it got downvoted). Can people maybe just explain what's wrong with a post when it's not a personal attack, not off topic, not answered in the article, or any other obvious downvote reason? Everyone would appreciate the author learning from the problem if there is one
  • thaumasiotes 2y ago
• deleted 2y ago
  [deleted]
• eddd-ddde 2y ago
  Now that I think about it, interfaces such as Js fetch should make it an error to use http without explicitly allowing in an option. It seems to easy to make an error and end up in a situation like the post explains.
  • smaudet 2y ago
    Hmm. I think, perhaps, release versions should need this, without a flag. For testing/prototyping, it is invaluable to turn off all the security to rule out security misconfiguration instead of application error. If your API is non-sensitive/relies on out of band security (like large files with checksums), you may still not want https, so there should be some configuration to turn it off. And for "integrations" like jsdelivr, perhaps https libraries should follow this rule, while http ones can have the flag off... Then, if you mix the two (http and https) perhaps they can provide an noticeable alert to the user rather than failing silently...
• jpswade 2y ago
  Nothing should redirect ever. However, it happens.
• GauntletWizard 2y ago
  I do redirect APIs to HTTPS, but I'd prefer not to. There's a simple reason - My APIs are hosted on the same IP as the public website, behind the same load balancer, so something has to be on the HTTP port. I would prefer to separate them, and my larger customers do - But for smaller customers, it's an unnecessary added expense and complication that doesn't make sense.
• zepton 2y ago
  The Stack Exchange API used to revoke API keys sent over HTTP (and return an error message), which is my favorite way to handle this.
  • znpy 2y ago
    I've been thinking for about 5 minutes about this comment and what to write but i've come to the conclusion that this is really not the best thing to do, but the correct thing to do. It's not different levels of good or bad... everything else is wrong.
    • comex 2y ago
      One of the approaches mentioned in the article is to just not listen on port 80. Supposedly that’s equally good because the connection should get aborted before the client has the chance to actually send any API keys. But is that actually true? With TCP Fast Open, a client can send initial TCP data before actually learning whether the port is open. It needs a cookie previously received from the server to do so, but the cookie is not port-specific, so – assuming the server supports Fast Open – the client could have obtained the cookie from a prior connection over HTTPS or any other valid port. That’s the impression I get from reading the RFC, anyway. The RFC does mention that clients should distinguish between different server ports when caching refusals by the server to support Fast Open, but by that point it’s too late; the data may have already been leaked.
      • pixl97 2y ago
        If someone is in your path they can just fake listen to 80 and intercept, then forward your call to 443. Probably best to listen on 80 and trash the token right then as the majority of the time there won't be a MITM and breaking the application will force the developer to change to https
        • jedberg 2y ago
          > If someone is in your path they can just fake listen to 80 and intercept, then forward your call to 443. They can do that whether or not you are listening on port 80 though.
• croes 2y ago
  Maybe shttp would have been better than https to reduce typo errors or maybe even something completely different from http.
  • toomim 2y ago
    Ah... you know, that idea sounds not too shtty to me.
  • jimbobthrowawy 2y ago
    I'm sure it was added to the end following a pattern of other protocols doing the same for their ssl-wrapped version. Right now, shttp reads like http over ssh to me.
• ikisusi 2y ago
  I hope that providers whose APIs responded and interacted fully over unencrypted HTTP would go back to their historical access logs and check how widespread using plaintext HTTP is. If they don't have access logs for their API then they could just sample next 24 hours for API accesses. Popular providers have so many API users today that even a rare mistake could expose quite many users in absolute numbers. Would rather have providers to check this out rather than have this poor practice abused by the next DNS hijacking malware affecting home routers.
  • mikeocool 2y ago
    I wouldn’t hold my breath. As I recall a similar article appeared a few years ago, and the author called out a major SaaS provider as having this issue. The provider ultimately decided not to do anything about it, because it would break too many clients. If you make a breaking API change like this, some portion of clients are just never going to update. If you’re a usage-based billing SaaS provider, that means lost revenue. Likely the only way this issue is fixed widely is if it ends up on a security audit checklist.
• barryrandall 2y ago
  I agree that APIs shouldn't automatically redirect HTTP to HTTPS, but I also think that client libraries shouldn't follow redirects by default.
• amanzi 2y ago
  Interesting - I hadn't considered this before, but makes perfect sense. Feels like it's something that's easy to miss, as lots of APIs are hosted behind generic web application firewalls that often have automatic HTTPS redirection as a base rule.
• athyuttamre 2y ago
  Great article! We've updated the OpenAI API to 403 on HTTP requests instead of redirecting. $ curl http://api.openai.com/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer 123" \ -d '{}' { "error": { "type": "invalid_request_error", "code": "http_unsupported", "message": "The OpenAI API is only accessible over HTTPS. Ensure the URL starts with 'https://' and not 'http://'.", "param": null } }
  • alberth 2y ago
    Doesn’t returning a 403 on HTTP break HSTS? https://security.stackexchange.com/questions/122441/should-hsts-header-be-sent-on-an-error-response https://security.stackexchange.com/questions/122441/should-h... Doesn’t HSTS require only responding to a user via HTTPS (even for error codes).
    • gerdesj 2y ago
      HSTS is a note to the browser to insist on TLS when hitting a website. It is sent as a header, with a timescale, regardless of http/https.
    • kji 2y ago
      HSTS is intended for browsers. For API clients the correct behavior (following curl's lead) is probably to never follow/make any redirects by default.
    • josephcsible 2y ago
      What about this then? When the request is made over insecure HTTP, revoke the API key used, but then send the usual redirect response for HSTS. Then, when/if the request gets repeated over HTTPS, notice the key is revoked and so respond to that one with 403.
      • freedomben 2y ago
        If your goal is to waste people's time, cause them to question their sanity, and guarantee that they're way too pissed off when they finally figure out what happened that instead of teaching others about how important it is too use HTTPS from the start, they talk about how awful your API is and how much they hate your company for a terrible design, then yes this sounds like a good plan.
• andrewaylett 2y ago
  I've stopped opening port 80 at all for some of my web services. The parent domain is in the HSTS preload list, so no modern browser should ever be trying to connect to port 80. And (as the fine article intimates) API calls shouldn't be available on port 80 anyway.
• victorbjorklund 2y ago
  Makes sense.
• deleted 2y ago
  [deleted]
• dramm 2y ago
  Yes, great article. And now can we convince folks with http+https websites to shut down http access and only offer https. I've seen simple mistakes like only partial redirects happening. Large numbers of internal links that still go to the http site, and some of those not redirect, etc. (you would think they are simple to find and just clean up), etc. And it is frustrating when sites like some online forums may be interesting targets for password theft.
• xyst 2y ago
  Revoking an API key upon a single http request assumes you have a competent team. Have worked with people that have committed sensitive credentials into public repositories, used PRODUCTION secrets for testing, and of course sharing secrets in plain text over IM and group chats. The number of times I have had to deal with password or secret key resets because of this is way too high. I remember working with a guy that thought sharing a demo of an internal application on YouTube was okay. Of course it had snippets of company secrets and development API keys clearly visible in the demo.
  • loa_in_ 2y ago
    It assumes nothing like that. It provides you and your team with a safe opportunity to learn and grow. Revoking a key like this is a problem that's solution is at worst a dozen clicks away fix. The alternative, leaking keys can be much worse. So go and reset those creds for the guy who made a mistake happily and be grateful. He had an opportunity to learn.
    • rocqua 2y ago
      It might be optimistic to say that rolling over to a new key is 'at most a dozen clicks'. Hardcoded keys are a big one. But even just hunting down all config where the key needs to change can be a major hassle. Is there some ci yaml that expects a runner to have the key in a .env file that only runs on major release? Good chance you won't realize that key exists. Still a great idea to revoke those keys. But it will damage some customers in the short term. And they will be angry, as people often are when you demonstrate their mistake.
    • xyst 2y ago
      There’s the guy that made _a_ mistake. Then there’s the guy who continues to make the same mistake over and over again. That second person is way to common in companies that hire from the bottom of the barrel.
      • loa_in_ 2y ago
        The thing with these kinds of mistakes is that if the service doesn't revoke the key and "cause problems" immediately, then there's no feedback to learn from. It's a fail-fast situation that's usually a better outcome anyway.
• chrismorgan 2y ago
  npm is misusing 426 Upgrade Required. https://httpwg.org/specs/rfc9110.html#status.426 https://httpwg.org/specs/rfc9110.html#status.426: > The server MUST send an Upgrade header field in a 426 response to indicate the required protocol(s) (Section 7.8). https://httpwg.org/specs/rfc9110.html#field.upgrade https://httpwg.org/specs/rfc9110.html#field.upgrade: > The Upgrade header field only applies to switching protocols on top of the existing connection; it cannot be used to switch the underlying connection (transport) protocol, nor to switch the existing communication to a different connection. For those purposes, it is more appropriate to use a 3xx (Redirection) response (Section 15.4). If you’re going to talk cleartext HTTP and issue a client error rather than redirecting, 403 Forbidden or 410 Gone are the two most clearly correct codes to use. Ignoring the mandated semantics and requirements of status codes is sadly not as rare as it should be. A few I’ve encountered more than once or twice: 401 Unauthorized without using WWW-Authenticate and Authorization; 405 Method Not Allowed without providing Allow; 412 Precondition Failed for business logic preconditions rather than HTTP preconditions; 417 Expectation Failed for something other than an Expect header. I think it only ever really happens with 4xx client errors.
  • Titan2189 2y ago
    How is this not more upvoted? HTTP Code 426 sounds like the best code to send?
  • notpushkin 2y ago
    TLS is in fact a valid protocol to use in an Upgrade header: HTTP/1.1 426 Upgrade Required Upgrade: TLS/1.0, HTTP/1.1 Connection: Upgrade So you can use 426 Upgrade Required here, and I'd argue it's the most correct code to use in such a case. npm doesn't send the Upgrade header though, so that's a mistake. https://www.iana.org/assignments/http-upgrade-tokens/http-upgrade-tokens.xhtml https://www.iana.org/assignments/http-upgrade-tokens/http-up... https://www.rfc-editor.org/rfc/rfc2817.html#section-4.2 https://www.rfc-editor.org/rfc/rfc2817.html#section-4.2
    • chrismorgan 2y ago
      That’s something different: that’s for upgrading to TLS within the same connection. As in, approximately http://example.com/ http://example.com/ → https://example.com:80/ https://example.com:80/ (but without the URL’s scheme actually being allowed to change), whereas https://example.com/ https://example.com/ is https://example.com:443/ https://example.com:443/. I was only a child when RFC 2817 was published, but I’ve never heard of any software that supported it, other than the Internet Printing Protocol which can use it for ipp: URLs, kinda like SMTP has STARTTLS. As for the motivations of RFC 2817, they’re long obsolete: encryption should no longer be optional on these sorts of things so that the parallel secure port problem is gone (not sure when this became actual IETF policy, but I’m going to guess towards ten years ago), and the virtual hosting problem is solved by SNI (supported by everything that matters for well over a decade).
• zeeb0t 2y ago
  Hard to argue against this.
• perlpimp 2y ago
  "This unencrypted part of the communication flow has its flaws. Third parties in shared networks, as well as network intermediaries, could sniff passwords and other secrets from the initial HTTP traffic or even impersonate the web server with a MITM attack." A strawman fallacy.
• 1vuio0pswjnm7 2y ago
  As a non-developer, ordinary computer user "providing service" for one user (yours truly) it's easy for me to configure the TLS forward proxy listening on the loopback to send _all_ HTTP requests, from _any_ application, including ones sent to port 80, via HTTPS. This I find preferable to letting a browser try to convert HTTP to HTTPS, e.g., "HTTPS Everywhere", or letting a developer do it with a redirect. Personally, I compile clients without linking to a TLS library. They are smaller without it and I do not have to worry about every author having correctly added TLS support. When SSL/TLS changes, applications using it often need to be updated, and some authors have made mistakes, socat being one example that comes to mind. I do 100% of TLS negotiation using a single program: the proxy. Every HTTP request on the home network goes to the proxy.
  • jagger27 2y ago
    There’s absolutely nothing ordinary or easy about that setup, but I admire it. I’ve only seen that level of paranoia at a three letter agency. Do applications with pinned certificates break? A bunch of mobile apps do that to get in the way of Wireshark.
    • 1vuio0pswjnm7 2y ago
      https://f-droid.org https://f-droid.org
      • jagger27 2y ago
        Someday on iOS, if Tim wills it.
        • 1vuio0pswjnm7 2y ago
          https://www.metacritic.com/movie/steve-jobs/ https://www.metacritic.com/movie/steve-jobs/ According to this film, which Wozniak said was very accurate, Steve Jobs was a douchebag. Android sucks, too. I only use it out of necessity, not choice. Why do I use a TLS forward proxy. Because I loathe using popular, so-called "modern" web browsers. I prefer using smaller, simpler clients.
• ac130kz 2y ago
  There is a limited number of cases, where unencrypted http is still applicable, e.g. verifiable packages. In general though, it feels wrong even to think about putting such a glaring useless hole.
• tonymet 2y ago
  You can’t make this decision until you know how many customers are on http and how lucrative they are . Breaking an API because you read a blog post is a bad idea
  • notpushkin 2y ago
    You can only do it for new customers, though. Just save the list of API keys used over HTTP, then whitelist those who earn more than $xxx/year. Then you can work with those customers to help them upgrade their clients to HTTPS.
• zzo38computer 2y ago
  I think that it would be better to allow both as much as possible. One way to handle authentication is to use HMAC; you can do that even without needing TLS (and HMAC won't leak the keys if one of the systems (e.g. a reverse proxy or something else) is compromised). If you do not want to do that, then don't accept connections on port 80, at least when version 6 internet is being used. (For version 4 internet, it is possible that you might use the same IP address for domain names that do want to accept connections on port 80, so you cannot easily block them in such a case.) And, if you want to avoid compromising authentication data, then TLS is not good enough anyways. The client will need to know that the server certificates have not been compromised. HMAC will avoid that problem, even without TLS. There is also the payload data. TLS will encrypt that, but the URL will be unencrypted if TLS is not used, whether or not the API key is revoked; the URL and headers may contain stuff other than API keys. Some APIs also might not need keys; e.g. read-only functions for public data often should not need any kind of API keys (and should not have mandatory TLS either, although allowing optional TLS is helpful, since it does provide some security, even though it doesn't solve everything). HSTS is even worse. TLS prevents spies from reading and tampering with your messages, but does not prevent the server from doing so (although in this case it might be unimportant, depending on the specific file being accessed). It also is complicated and wastes energy, and there are sometimes security vulnerabilities in some implementations so does not necessarily improve security. Of course, these ideas will not, by itself, improve security, and neither will TLS; their combination also won't do. You will have to be more careful to actually improve security properly. Some people think that, if you add TLS and HTTPS, and insist on using it, then it is secure. Well, it is very wrong!!! TLS will improve security in some ways as described in the previous paragraph, but does not solve everything. It is also problematic if a client does not have an option to disable TLS (or use unencrypted proxies), since if you deliberately want to do MITM on your own computer, you will then have to effectively decrypt and encrypt the data twice. (If the client uses TLS by default, that would work, although if it is according to the URL then it might be by the configurable URL instead; however, the URLs do not always come from the configuration file, and even if it does, and if you want to avoid typographical errors (although even if "https" is specified, typographical errors are still possible (e.g. in the domain name), so just checking for "https" won't even necessarily help anyways; specifying what certificates to expect might sometimes help), then you might have your program to display a warning message, perhaps.) Another problem is if the client and server require different versions of TLS and it is difficult to change the software (there are reasons you might want to change only some parts of it, and that can be difficult); using a local unencrypted proxy which connects to the server using TLS, can also avoid problems like this, too.
• kgeist 2y ago
  I think a good approach would be for developers to always use a custom HTTPClient class which throws an error if HTTP is used. I.e. you MUST opt in to use HTTP.
• tgma 2y ago
  If you are paying the cost of a TLS handshake, which you will have to anyway, why not just use client certificates to authenticate (mTLS) instead of hand rolled and rudimentary auth tokens on top. gRPC has built-in support for mTLS and it could be a good time to modernize your endpoints if you are looking to invest time to improve your API security.
• deleted 2y ago
  [deleted]
• usrbinbash 2y ago
  > and revoke API keys sent over the unencrypted connection. Excuse me, short question: If I am not offering a non-TLS endpoint in the first place, and the client, for some reason, decides to take something that is literally called "SECRET", and decides to shout it across the open internet unencrypted... ...how is that my problem again? Why should my setup be more complex than it needs to be, to make up for an obvious mistake made by the client?
• fl0ki 2y ago
  This makes sense, and I wondered for a moment why I hadn't noticed the issue in any of our projects. I realized it's because we just don't expose HTTP in the first place, whether for API or UI purposes. However, that doesn't mean there's no danger. I believe it's still possible for a client to leak its credentials if it makes a bare HTTP/1 call to an actual HTTPS endpoint. The server only gets a chance to reject its invalid TLS handshake after it's already sent headers that may contain sensitive information. After all, the TCP negotiation did go through, and the first layer 7 packet is the bare HTTP headers. Of course the port numbers should help here, but that's not guaranteed in an environment where port numbers are assigned dynamically and rely on service discovery. Serving exclusively HTTP/3 should close this gap but requires all clients to be ready for that. I know many internal deployments do this, but it's not a universal solution yet.
• jensenbox 2y ago
  After all this chatter, I am considering blocking all outgoing traffic to port 80 in my local firewall This would prevent my fat fingers from ever even making the mistake. BLOCK outgoing port 80 How bad would that be? Would I be shooting myself in the foot somehow? Perhaps I would do it on the egress rule for where my requesting service is running like in ECS.
  • brobinson 2y ago
    It would block requests to OCSP responders, for one.
• smashface 2y ago
  Headline reads like a hot take. Actual recommendation is rather useful. Click-bait used for good.
• tk90 2y ago
  If you want to implement this in a AWS/ALB (Load Balancer) setup, you have to: - Remove the ALB's HTTP listener - Remove port 80 from the ALB's security group
• bradley13 2y ago
  https is great, and I'm glad most web traffic uses it. However, sometimes you want http. Example: when teaching low-level socket programming, plain http is the easiest place to start. Example: when providing simple, nonsensitive, read-only info via API, the overhead of https and certificate management seems unnecessary.
• plagiat0r 2y ago
  It would be great to not listen on tcp/80 for your API, except that acme http-01 uses tcp/80 explicitly for domain validation. So in order to get https, you open http. Sure, there is dns-01 and tls-alpn-01 but I presume that majority uses http-01 and migrating it is not a trivial matter.