5 ms·

I think this post gets the complexity situation backwards. Sure, you can use a different protocol between your load balancer and your application and it won't d

by lmm 2y ago

I think this post gets the complexity situation backwards. Sure, you can use a different protocol between your load balancer and your application and it won't do too much harm. But you're adding an extra protocol that you have to understand, for no real benefit. (Also, why do you even want a load balancer/reverse proxy, unless your application language sucks? The article says it "will also take care of serving static assets, normalize inbound requests, and also probably fend off at least some malicious actors", but frankly your HTTP library should already be doing all of those. Adding that extra piece means more points of failure, more potential security vulnerabilities, and for what benefit?)

pixelesque 2y ago
> Sure, you can use a different protocol between your load balancer and your application and it won't do too much harm. But you're adding an extra protocol that you have to understand, for no real benefit. Well, that depends... At a certain scale (and arguably, not too many people will ever need to think about this), using UNIX sockets (instead of HTTP TCP) between the application and load balancer can be faster in some cases, as you don't go through the TCP stack... > Also, why do you even want a load balancer/reverse proxy, unless your application language sucks? Erm... failover... ability to do upgrades without any downtime... it's extra complexity yes, but it does have some benefits...
- lmm 2y ago
  > At a certain scale (and arguably, not too many people will ever need to think about this), using UNIX sockets (instead of HTTP TCP) between the application and load balancer can be faster in some cases, as you don't go through the TCP stack... Sure (although as far as I can see there's no reason you can't keep using HTTP for that). You can go even further and use shared memory (I work for a company that used Apache with Jk back in the day). But that's an argument for using a faster protocol because you're seeing a benefit from it, not an argument for using a slower protocol because you can't be bothered to implement the latest standard.
  - tuukkah 2y ago
    > using a slower protocol because you can't be bothered to implement the latest standard. I thought we were discussing HTTP/2 but now you seem to be invoking HTTP/3? It's even faster indeed but brings a whole lot of baggage with it. Nice comparison point though: Do you want to add the complexity of HTTP/2 or HTTP/3 in your backend? (I don't.)
    - lmm 2y ago
      > I thought we were discussing HTTP/2 but now you seem to be invoking HTTP/3? The article talks about HTTP/2 but I suspect they're applying the same logic to HTTP/3. > Do you want to add the complexity of HTTP/2 or HTTP/3 in your backend? (I don't.) I'd like to use the same protocol all the way through. I wouldn't want to implement any HTTP standard by hand (I could, but I wouldn't for a normal application), but I'd expect an established language to have a solid library implementation available.
tuukkah 2y ago
Answers from the article - the "extra" protocol is just HTTP/1.1 and the reason for a load balancer is the ability to have multiple servers: > But also the complexity of deployment. HTTP/2 is fully encrypted, so you need all your application servers to have a key and certificate, that’s not insurmountable, but is an extra hassle compared to just using HTTP/1.1, unless of course for some reasons you are required to use only encrypted connections even over LAN. > So unless you are deploying to a single machine, hence don’t have a load balancer, bringing HTTP/2 all the way to the Ruby app server is significantly complexifying your infrastructure for little benefit.
- conradludgate 2y ago
  I've deployed h2c (cleartext) in many applications. No tls complexity needed
  - tuukkah 2y ago
    Good to know - neither the parent nor the article mention this. h2c seems to have limited support by tooling (e.g. browsers, curl), which is a bit discouraging. EDIT: Based on the HTTP/2 FAQ, pure h2c is not allowed in the standard as it requires you to implement some HTTP/1.1 upgrade functionality: https://http2.github.io/faq/#can-i-implement-http2-without-implementing-http11 https://http2.github.io/faq/#can-i-implement-http2-without-i...
    - _ache_ 2y ago
      Why do you think that curl doesn't support h2c ? It does. Just use `--http2` or `--http2-prior-knowledge`, curl deduce the clear or not clear by `http` or `https` URL protocol prefix (clear the default).
      - tuukkah 2y ago
        I said limited support and gave curl as an example because curl --http2 sends a HTTP/1.1 upgrade request first so fails in a purely HTTP/2 environment. Thanks for bringing up --http2-prior-knowledge as a solution!
Galanwe 2y ago
> Also, why do you even want a load balancer/reverse proxy, unless your application language sucks - To terminate SSL - To have a security layer - To load balance - To have rewrite rules - To have graceful updates - ...
- lmm 2y ago
  > To terminate SSL To make sure that your connections can be snooped on over the LAN? Why is that a positive? > To have a security layer They usually do more harm than good in my experience. > To load balance Sure, if you're at the scale where you want/need that then you're getting some benefit from that. But that's something you can add in when it makes sense. > To have rewrite rules > To have graceful updates Again I would expect a HTTP library/framework to handle that.
  - fragmede 2y ago
    You terminate SSL as close to the user as possible, because that round trip time is greatly going to affect the user experience. What you do between your load balancer and application servers is up to you, (read: should still be encrypted) but terminating SSL asap is about user experience.
    - lmm 2y ago
      > You terminate SSL as close to the user as possible, because that round trip time is greatly going to affect the user experience. What you do between your load balancer and application servers is up to you, (read: should still be encrypted) but terminating SSL asap is about user experience. That makes no sense. The latency from your load balancer to your application server should be a tiny fraction of the latency from the user to the load balancer (unless we're talking about some kind of edge deployment, but at that point it's not a load balancer but some kind of smart proxy), and the load balancer decrypting and re-encrypting almost certainly adds more latency compared to just making a straight connection from the user to the application server.
      - 2y ago
toast0 2y ago
Load balancers are nice to have if you want to move traffic from one machine to another. Which sometimes needs to happen even if your application language doesn't suck and you can hotload your changes... You may still need to manage hardware changes, and a load balancer can be nice for that. DNS is usable, but some clients and recursive resolvers like to cache results for way beyond the TTL provided.
harshreality 2y ago
> why do you even want a load balancer/reverse proxy, unless your application language sucks? Most load balancer/reverse proxy applications also handle TLS. Security-conscious web application developers don't want TLS keys in their application processes. Even the varnish authors (varnish is a load balancer/caching reverse proxy) refused to integrate TLS support because of security concerns; despite being reverse-proxy authors, they didn't trust themselves to get it right. An application can't load-balance itself very well. Either you roll your own load balancer as a separate layer of the application, which is reinventing the wheel, or you use an existing load balancer/reverse proxy. Easier failover with fewer (ideally zero) dropped requests. If the app language isn't compiled, having it serve static resources is almost certainly much slower than having a reverse proxy do it.
- lmm 2y ago
  > Security-conscious web application developers don't want TLS keys in their application processes. If your application is in a non-memory-safe language, sure (but why would you do that?). Otherwise I would think the risk is outweighed by the value of having your connections encrypted end-to-end. If your application process gets fully compromised then an attacker already controls it, by definition, so (given that modern TLS has perfect forward secrecy) I don't think you really gain anything by keeping the keys confidential at that point.
  - nickelpro 2y ago
    I write application servers for a living, mostly for Python but previously for other languages. Nobody, nobody, writes application servers with the intent of having them exposed to the public internet. Even if they're completely memory safe, we don't do DOS protections like checking for reasonable header lengths, rewriting invalid header fields, dropping malicious requests, etc. Most application servers will still die to slowloris attacks. [1] We don't do this because it's a performance hog and we assume you're already reverse proxying behind any responsible front-end server, which all implement these protections. We don't want to double up on that work. We implement the HTTP spec with as low overhead as possible, because we expect to have pipelined HTTP/1.1 connections from a load balancer or other reverse proxy. Your application server, Gunicorn, Twisted, Uvicorn, whatever, does not want to be exposed to the public internet. Do not expose it to the public internet. [1]: https://en.wikipedia.org/wiki/Slowloris_(cyber_attack) https://en.wikipedia.org/wiki/Slowloris_(cyber_attack)
guappa 2y ago
C fast, the rest slow. You don't want to serve static assets in non-C.
- SahAssar 2y ago
  If you call sendfile with kTLS I imagine it'd be fast in any language

by lmm 2y ago

pixelesque 2y ago

> Sure, you can use a different protocol between your load balancer and your application and it won't do too much harm. But you're adding an extra protocol that you have to understand, for no real benefit. Well, that depends... At a certain scale (and arguably, not too many people will ever need to think about this), using UNIX sockets (instead of HTTP TCP) between the application and load balancer can be faster in some cases, as you don't go through the TCP stack... > Also, why do you even want a load balancer/reverse proxy, unless your application language sucks? Erm... failover... ability to do upgrades without any downtime... it's extra complexity yes, but it does have some benefits...

lmm 2y ago
> At a certain scale (and arguably, not too many people will ever need to think about this), using UNIX sockets (instead of HTTP TCP) between the application and load balancer can be faster in some cases, as you don't go through the TCP stack... Sure (although as far as I can see there's no reason you can't keep using HTTP for that). You can go even further and use shared memory (I work for a company that used Apache with Jk back in the day). But that's an argument for using a faster protocol because you're seeing a benefit from it, not an argument for using a slower protocol because you can't be bothered to implement the latest standard.
- tuukkah 2y ago
  > using a slower protocol because you can't be bothered to implement the latest standard. I thought we were discussing HTTP/2 but now you seem to be invoking HTTP/3? It's even faster indeed but brings a whole lot of baggage with it. Nice comparison point though: Do you want to add the complexity of HTTP/2 or HTTP/3 in your backend? (I don't.)
  - lmm 2y ago
    > I thought we were discussing HTTP/2 but now you seem to be invoking HTTP/3? The article talks about HTTP/2 but I suspect they're applying the same logic to HTTP/3. > Do you want to add the complexity of HTTP/2 or HTTP/3 in your backend? (I don't.) I'd like to use the same protocol all the way through. I wouldn't want to implement any HTTP standard by hand (I could, but I wouldn't for a normal application), but I'd expect an established language to have a solid library implementation available.

tuukkah 2y ago

Answers from the article - the "extra" protocol is just HTTP/1.1 and the reason for a load balancer is the ability to have multiple servers: > But also the complexity of deployment. HTTP/2 is fully encrypted, so you need all your application servers to have a key and certificate, that’s not insurmountable, but is an extra hassle compared to just using HTTP/1.1, unless of course for some reasons you are required to use only encrypted connections even over LAN. > So unless you are deploying to a single machine, hence don’t have a load balancer, bringing HTTP/2 all the way to the Ruby app server is significantly complexifying your infrastructure for little benefit.

conradludgate 2y ago
I've deployed h2c (cleartext) in many applications. No tls complexity needed
- tuukkah 2y ago
  Good to know - neither the parent nor the article mention this. h2c seems to have limited support by tooling (e.g. browsers, curl), which is a bit discouraging. EDIT: Based on the HTTP/2 FAQ, pure h2c is not allowed in the standard as it requires you to implement some HTTP/1.1 upgrade functionality: https://http2.github.io/faq/#can-i-implement-http2-without-implementing-http11 https://http2.github.io/faq/#can-i-implement-http2-without-i...
  - _ache_ 2y ago
    Why do you think that curl doesn't support h2c ? It does. Just use `--http2` or `--http2-prior-knowledge`, curl deduce the clear or not clear by `http` or `https` URL protocol prefix (clear the default).
    - tuukkah 2y ago
      I said limited support and gave curl as an example because curl --http2 sends a HTTP/1.1 upgrade request first so fails in a purely HTTP/2 environment. Thanks for bringing up --http2-prior-knowledge as a solution!

Galanwe 2y ago

> Also, why do you even want a load balancer/reverse proxy, unless your application language sucks - To terminate SSL - To have a security layer - To load balance - To have rewrite rules - To have graceful updates - ...

lmm 2y ago
> To terminate SSL To make sure that your connections can be snooped on over the LAN? Why is that a positive? > To have a security layer They usually do more harm than good in my experience. > To load balance Sure, if you're at the scale where you want/need that then you're getting some benefit from that. But that's something you can add in when it makes sense. > To have rewrite rules > To have graceful updates Again I would expect a HTTP library/framework to handle that.
- fragmede 2y ago
  You terminate SSL as close to the user as possible, because that round trip time is greatly going to affect the user experience. What you do between your load balancer and application servers is up to you, (read: should still be encrypted) but terminating SSL asap is about user experience.
  - lmm 2y ago
    > You terminate SSL as close to the user as possible, because that round trip time is greatly going to affect the user experience. What you do between your load balancer and application servers is up to you, (read: should still be encrypted) but terminating SSL asap is about user experience. That makes no sense. The latency from your load balancer to your application server should be a tiny fraction of the latency from the user to the load balancer (unless we're talking about some kind of edge deployment, but at that point it's not a load balancer but some kind of smart proxy), and the load balancer decrypting and re-encrypting almost certainly adds more latency compared to just making a straight connection from the user to the application server.
    - 2y ago

toast0 2y ago

Load balancers are nice to have if you want to move traffic from one machine to another. Which sometimes needs to happen even if your application language doesn't suck and you can hotload your changes... You may still need to manage hardware changes, and a load balancer can be nice for that. DNS is usable, but some clients and recursive resolvers like to cache results for way beyond the TTL provided.

harshreality 2y ago

> why do you even want a load balancer/reverse proxy, unless your application language sucks? Most load balancer/reverse proxy applications also handle TLS. Security-conscious web application developers don't want TLS keys in their application processes. Even the varnish authors (varnish is a load balancer/caching reverse proxy) refused to integrate TLS support because of security concerns; despite being reverse-proxy authors, they didn't trust themselves to get it right. An application can't load-balance itself very well. Either you roll your own load balancer as a separate layer of the application, which is reinventing the wheel, or you use an existing load balancer/reverse proxy. Easier failover with fewer (ideally zero) dropped requests. If the app language isn't compiled, having it serve static resources is almost certainly much slower than having a reverse proxy do it.

lmm 2y ago
> Security-conscious web application developers don't want TLS keys in their application processes. If your application is in a non-memory-safe language, sure (but why would you do that?). Otherwise I would think the risk is outweighed by the value of having your connections encrypted end-to-end. If your application process gets fully compromised then an attacker already controls it, by definition, so (given that modern TLS has perfect forward secrecy) I don't think you really gain anything by keeping the keys confidential at that point.
- nickelpro 2y ago
  I write application servers for a living, mostly for Python but previously for other languages. Nobody, nobody, writes application servers with the intent of having them exposed to the public internet. Even if they're completely memory safe, we don't do DOS protections like checking for reasonable header lengths, rewriting invalid header fields, dropping malicious requests, etc. Most application servers will still die to slowloris attacks. [1] We don't do this because it's a performance hog and we assume you're already reverse proxying behind any responsible front-end server, which all implement these protections. We don't want to double up on that work. We implement the HTTP spec with as low overhead as possible, because we expect to have pipelined HTTP/1.1 connections from a load balancer or other reverse proxy. Your application server, Gunicorn, Twisted, Uvicorn, whatever, does not want to be exposed to the public internet. Do not expose it to the public internet. [1]: https://en.wikipedia.org/wiki/Slowloris_(cyber_attack) https://en.wikipedia.org/wiki/Slowloris_(cyber_attack)

guappa 2y ago

C fast, the rest slow. You don't want to serve static assets in non-C.

SahAssar 2y ago
If you call sendfile with kTLS I imagine it'd be fast in any language