3 ms·

We pull unhealthy VMs and containers from rotation in real time. Serving a bunch of 500 series errors to clients because your application server took a nosedive

We pull unhealthy VMs and containers from rotation in real time. Serving a bunch of 500 series errors to clients because your application server took a nosedive isn't great even with a 300 second TTL. We don't use NAT. When people say RFC1918 addresses aren't routable they mean you can't advertise them on the public Internet. You can totally have 10/8 switched and routed internally. Even if you don't want to mess with that, it's entirely possible to set a proxy to proxy from its public IP to another public IP, but why waste the IP space, especially in v4? I can server 80 applications each with half a dozen or more backend systems from about six proxy servers. The proxy isn't running the application and connecting to the RDBMS so each one can handle way, way more traffic than even an efficient application. Using a proxy that's layer 7 aware also lets you do things like scale different parts of a URI tree individually. If you're not proxying, every copy of app.example.com serves everything under https://app.example.com/ https://app.example.com/ even if it's one of 23 (about the most A records that will fit in UDP DNS) servers doing it. With a proxy, I can decide that https://app.example.com/freestuff/ https://app.example.com/freestuff/ is too busy and spikes the backends, and that the load of that and the rest of the stuff under / is too hard to scale for properly when taken together. So I just tell my proxies that everything for https://app.example.com/freestuff https://app.example.com/freestuff goes to server set A and the rest of stuff in / goes to server set B. Then the different performance and different demand for the two can be studied, improved, monitored, and reported separately at the VM or container (or even bare metal backend) level. I can also throw memcached or Redis into the mix and do site-wide limiting on an IP that's scanning or attacking my entire laundry list of applications. Even without those I can rate-limit what each proxy will accept from a single visitor globally or per backend type.

fulafel 6y ago
How often does it happen that working containers suddenly fail like that though and it's critical to shave the minutes? I appreciate proxying does give you slightly better reaction time but this has to be weighed against the costs and outages caused by the complexity in the proxying. Re you app.example.com scenario: I was not thinking of serving the static assets and frontend code from the individual services, but rather having eg your SPA know about the different endpoints and serving the resources from their own service which would be stable. Re the memcached angle - the static serving can have caching by whatever mechanism and the API endpoints can also use memcached all they want, with at least as good control granularity as in the proxy case. But if you were thinking about a caching reverse proxy, I think that would be more prone to poisoning the cache with error replies than having the endpoints do their caching individually. Again, think of all the failure prone machinery you can elide in this scenario.
- cestith 6y ago
  Reasonable rules for something like HAProxy are not at all prone to errors. Ours aren't even applied by hand. It only has to be made right when the automation module is updated, and that gets tested well before it goes to production. Varnish rules are also not that difficult, but I said nothing of caching reverse proxy. I mentioned Memcached specifically in the context of federating traffic counts across multiple non-caching proxies. Whether an application uses Memcached or not is irrelevant to the proxies.
fulafel 6y ago
Re the v4 space question, for v4 from the public internet was covered in my original (fallback proxy switched on host header). Internally, using v4 10.x networks for internal systems is fraught with problems as you'll inevitably end up trying to connect other 10.x ambiguously addressed networks, and get bit by the ambiguity in security, monitoring, configuration etc. End-to-end addrssing with globally unique addresses is just a good idea, it's a big reason the internet model won over competing networking technologies.
- cestith 6y ago
  It's not inevitable. Our core routers aren't even the same routers as our edge routers. The edge routers won't touch a 10/8 in any direction. The core routers only allow them on internal interfaces. I'm not sure what you mean by fallback proxies without proxying being involved. Could you elaborate and clarify?

fulafel 6y ago

How often does it happen that working containers suddenly fail like that though and it's critical to shave the minutes? I appreciate proxying does give you slightly better reaction time but this has to be weighed against the costs and outages caused by the complexity in the proxying. Re you app.example.com scenario: I was not thinking of serving the static assets and frontend code from the individual services, but rather having eg your SPA know about the different endpoints and serving the resources from their own service which would be stable. Re the memcached angle - the static serving can have caching by whatever mechanism and the API endpoints can also use memcached all they want, with at least as good control granularity as in the proxy case. But if you were thinking about a caching reverse proxy, I think that would be more prone to poisoning the cache with error replies than having the endpoints do their caching individually. Again, think of all the failure prone machinery you can elide in this scenario.

cestith 6y ago
Reasonable rules for something like HAProxy are not at all prone to errors. Ours aren't even applied by hand. It only has to be made right when the automation module is updated, and that gets tested well before it goes to production. Varnish rules are also not that difficult, but I said nothing of caching reverse proxy. I mentioned Memcached specifically in the context of federating traffic counts across multiple non-caching proxies. Whether an application uses Memcached or not is irrelevant to the proxies.

fulafel 6y ago

Re the v4 space question, for v4 from the public internet was covered in my original (fallback proxy switched on host header). Internally, using v4 10.x networks for internal systems is fraught with problems as you'll inevitably end up trying to connect other 10.x ambiguously addressed networks, and get bit by the ambiguity in security, monitoring, configuration etc. End-to-end addrssing with globally unique addresses is just a good idea, it's a big reason the internet model won over competing networking technologies.

cestith 6y ago
It's not inevitable. Our core routers aren't even the same routers as our edge routers. The edge routers won't touch a 10/8 in any direction. The core routers only allow them on internal interfaces. I'm not sure what you mean by fallback proxies without proxying being involved. Could you elaborate and clarify?