14 ms·
Setting Up HTTP(S) Load Balancing – WebSocket proxy support
- nodesocket 10y agoFeature request for GCE load balancers: Attach multiple/lots of SSL certificates to a single LB instance.
- manigandham 10y agoYou need to ask on the Google issue tracker, not HN. This is the outstanding ticket: https://issuetracker.google.com/issues/35904767 https://issuetracker.google.com/issues/35904767
- nodesocket 10y agoThanks for the link. I added my +1 with the comment: "Would be a competitive advantage as well since AWS ELB's don't support multiple certificates attached to a single ELB."
- jpillora 10y agoDoes this include AppEngine?
- oomkiller 10y agoAccording to the docs this is not yet supported on the App Engine Flexible product[1] [1] https://cloud.google.com/appengine/docs/flexible/ruby/how-requests-are-handled#not_supported https://cloud.google.com/appengine/docs/flexible/ruby/how-re...
- cobookman 10y agoIt doesn't include appengine, but it enables appengine flex to one day support it. Also FYI Appengine Flex does support HTTP2 Server Push. You just got to do the following: Set Headers to: Content-Type: text/event-stream Cache-Control: no-cache Connection: keep-alive // https://www.nginx.com/resources/wiki/start/topics/examples/x-accel/#x-accel-buffering https://www.nginx.com/resources/wiki/start/topics/examples/x... X-Accel-Buffering: no And use your favorite http2 server push framework.
- luhn 10y agoWhat you showed in your example is Server-Sent Events, not HTTP2 Server Push.
- spullara 10y agoIt does work really well but is unsupported on IE.
- jahewson 10y agoSever push is really a different use case from web sockets. It's also an optional part of the HTTP/2 protocol, no guarantee of client support. Also SSE has been around for a decade or so, not to be confused with HTTP/2 push.
- 19eightyfour 10y agoThis is really useful, thanks to provide this.
- jahewson 10y agoApparently this is being actively worked on: https://issuetracker.google.com/issues/35886348 https://issuetracker.google.com/issues/35886348
- thesandlord 10y agoI'm pretty sure this means Kubernetes Ingress on GCP now support WebSockets as well! People have been asking for this feature for a long time, glad it's available now. (I work on GCP)
- numbsafari 10y agoSNI support would also be really awesome. For certain kinds of traffic, the risks associated with SSL proxying means the HTTP(s) Load Balancer is off limits. Which is a shame, because it's an incredibly power feature compared to AWS offering.
- doubleorseven 10y agoSince when AWS is offering SNI on their load balancers? I work at a SaaS company and we were facing the same issue in the cloud and have to implement our own TCP LB's to support SNI. I really hope we will see it in the near future but as i see it, Will I allow my clients to push bytecode into my production machines on demand with the lowest sandboxing possible to provide low execution times? I guess not.
- tyingq 10y agoThey don't have SNI, but the newer ALB does have path based routing. So you can use a wild card or multidomain cert, if you can segment routes to endpoints by uri path.
- manigandham 10y agoUsually this is done with a CDN in front. Cloudfront supports SNI.
- numbsafari 10y agoAs far as I know it doesn't. I should have been more clear: GCP's Anycast, no-warming-need LB is way cooler than AWS traditional, single-region with totally unreliable DNS "routing" on top (because you always want to rely on client DNS TTLs to handle a service cutover...). The problem is, if you can't use a TLS proxy for security reasons, then you can't use the HTTP(s) LB, which will make you less happy than if you could use it.
- amasad 10y agoIt doesn't say how this changes the load distribution algorithm. Since websocket connections are usually long-running it wouldn't be as useful to use the default RPS (requests per second) algorithm. Ideally, it would be something that takes into account the number of connection each backend instance has.
- Cidan 10y agoYou can already adjust the load distribution algorithm when creating the LB, so I imagine this doesn't really change anything.
- amasad 10y agoThe distribution algorithm options are either a) rate or b) utilization. Re rate, you can configure the Requests Per Second but this doesn't mean much in a long-running connection context because some connections can be drastically longer (hours, days?). This may lead to starvation or overloading an instance.
- jhgg 10y agoWhat's the benefit here instead of using the tcp load balancer? I'm guessing it's primarily URL mapping. Does it support websocket compression (meaning the load balanced servers don't need to do compression - as it'd be handled by the proxy) We would probably switch in a heartbeat if it did
- searchfaster 10y agoNot sure if it GCP does, but the load balancer can handle HTTPS termination and even filter out DOS attacks or handle rate limiting.
- dyngts 10y agoHi searchfaster, Can you elaborate more from documentation that this new features can filter out DDOS attacks or handle rate limiting? Because i can't find out that statement Thank you
- Elect2 10y agohttp(s) load balancer only route http(s) requests to your compute instances, and hide the instance IPs. So at least Layer 3 DDOS can not touch your instances.
- dyngts 10y agoBut, when talking Kubernetes. it is currently no support for internal load balancer, right? so it's still have vulnerability to get exposed to the world. Any comment about this? Thank you!
- manigandham 10y agoGoogle's load balancers are global - as in they will route requests to the nearest region to the user without you having to run separate load balancers in each with dns routing. The backend services hooked up to the load balancer can have healthchecks and capacity limits based on req/sec or CPU usage, so a region that's close by but overloaded will be skipped for a further region that has available capacity. GCP also absorbs layer 4 and below attacks automatically. https://cloud.google.com/files/GCPDDoSprotection-04122016.pdf https://cloud.google.com/files/GCPDDoSprotection-04122016.pd...
- 19eightyfour 10y agoCongratulations! This has been a long time coming. Good to see it landing. There must be some technical challenges with doing a load balancer that is optimized for both HTTP round trips, AND, HTTP upgrades to long lived TCP connections. One issue is that, as stated in the Troubleshooting section of the linked docs, source IP is not forwarded to application. Maybe in future the X-Forwarded-For header, or some other source identification, will be provided, as well.
- manigandham 10y agoThe HTTP load balancer does provide the X-Forwarded-For header. This is standard behavior for all CDNs and load balancers now.
- 19eightyfour 10y agoIf that's the case...how to explain that the docs say source IP is not provided? Troubleshooting Load balanced traffic does not have a source address of the original client Traffic from the load balancer to your instances has an IP address in the ranges of 130.211.0.0/22 and 35.191.0.0/16. When viewing logs on your load balanced instances, you will not see the source address of the original client. Instead, you will see source addresses from this range. Is it the case that, if the X-Forwarded-For header is present, then it is not providing a source IP, or it is providing a source IP, but that this IP is not listed in the traffic logs?
- seanp2k2 10y agoTo be slightly pedantic, websockets specifically don't have HTTP headers like XFF, which is IMO part of the problem with Websockets -- you end up re-inventing basic functionality. IMO (and combined with other factors) this is a great reason to choose http/2 over websockets. EDIT: ALB also supports http/2, but they can also do websockets as well as WSS (where you'd term SSL on the ALB for websockets).
- manigandham 10y ago
- Kiro 10y agoWhat applications can use a load balancer with WebSockets? All the stuff I've built has required the WS connection to point to the same server anyway (multiplayer etc) and I feel that's the most common case but I might be completely wrong.
- jdub 10y agoIf your backends aren't stateless, then yeah, you'll have to make sure connections hit the same backend (with session affinity via client IP, cookies, etc). But session affinity is a scalability anti-pattern, which should generally only be used to support legacy applications. Ideally, you'd design your application to use stateless backends, which tend to share state at another layer (the database, memcache, Redis, etc).
- caleblloyd 10y agoI disagree that session affinity, or any affinity for that matter is a scalability anti-pattern. Why add more load on Redis or the Database if you can do it in the app server? Your just adding latency and increasing database load. Unused memory in the app server is free candy.
- jdub 10y agoThere are plenty of good reasons to avoid app server state (and thus session affinity), among them: Inefficiently imbalanced load balancing can affect performance and reliability, app server changes (service restart, reboot, scale down) will drop state causing interruptions (often user visible), etc. If you don't want unused memory on your app servers, don't provision it. :-)
- caleblloyd 10y agoThere's pros and cons to each method, as we've both pointed out. My main point is that it's not an anti-pattern, it's an architecture decision, and the pros and cons of each should be weighed when designing an app. I just finished working on a real-time collaborative document editing service and chose to use document affinity so that the app server could handle collaboration directly. It is extremely performant and I'd choose this approach again any day versus farming out to Redis/RethinkDB/etc. It persists to a database as writes come in, and performs initial load from the database. Local memory accesses are orders of magnitude faster than going over the network, and it reaps the benefits.
- cemerick 10y agoHeroku has supported websockets through its routing infrastructure for ages, along with things like SNI (which has been mentioned elsewhere here as things people wish were available in GCP). Not a ding on GCP necessarily, just notable how featureful mature cloud platforms are, such that "even Google" is playing catch-up.
- rdeckard0104 10y agoI wouldn't say playing catch up so much as I would say they're focused on features that provide the greatest amount of benefit to the greatest number of users. SNI is easily solved by setting up a Nginx proxy behind their HTTPS load balancer. Sure, I would really like it if SNI were baked in, but where they may seem like they fall short on small things, like this they are eons ahead of other providers with products, like BigQuery. BigQuery has made all the difference in the world for our engineering department. We have a vast, robust data warehouse, it's easy to read/write, easy to manage, and best of all we do not have the brunt maintenance around supports its own infrastructure.
- nailer 10y agoHey cool, they support HTTP/2 [1] and ECC (using P-256 curve) HTTPS certs [2] [1] https://cloud.google.com/compute/docs/load-balancing/http https://cloud.google.com/compute/docs/load-balancing/http [2] https://cloud.google.com/compute/docs/load-balancing/http/ssl-certificates https://cloud.google.com/compute/docs/load-balancing/http/ss...
- stephenr 10y agoHttp/2 is only supported from clients to the load balancer. Connections back to your instances will always be http/1.1
- stephenr 10y agoI find it quite ironic that Google - who pushed so hard to make http/2 happen, and is pushing so hard to make https the defacto default for the web - don't fully support either in their own load balancing solution.