4 ms·
Cloud load balancers can be sneakily expensive. Few months ago, we spent a few weeks replacing an ELB with naive client side load balancing via round robin, whi
by ublaze 6y ago
Cloud load balancers can be sneakily expensive. Few months ago, we spent a few weeks replacing an ELB with naive client side load balancing via round robin, which saves us > 200k/year. ELBs charge per byte transmitted, which seems reasonable, but can end up really expensive.
- user5994461 6y agoThey charge per byte and per request if I remember well, which can be really expensive for serving both small API call and large files. Another limitation is that the ELB only works in AWS to AWS instances in the same location. Gotta use something else for geographic load balancing and for other datacenters.
- cutemonster 6y ago> client side load balancing In the browser? Or a mobile app? They send 1 api req to server 1, then 1 to server 2 and so on? What about any session cookies maybe tied to a specific server?
- Nextgrid 6y agoPresumably round-robin DNS. A DNS response would only return a handful of servers, of which the client will itself only pick one at random for the duration of the session. Now this approach has drawbacks (DNS responses are cached, and the DNS record picked initially by the client will typically be cached until the app/browser is restarted) but if they are acceptable to you then it's an easy, proven solution.
- cutemonster 6y agoHmm I'd guess they have DNS cnames like api1.x.com and api2 and 3, 4 And then the client picks one, and if that server is offline, picks another Seems as simple as DNS based? And works with broken server(s)
- boring_twenties 6y agoExcept you don't need that because you can just return all four IP addresses for one record, e.g. api.x.com
- cutemonster 6y agoI think if such a DNS/ip based round robin server is down, or replies 500 error, the client won't try another server Unless there's a way to get all ip addrs in js? By custom client code that queries the DNS system?
- Nextgrid 6y agoDNS resolution is handled by the DNS server and the browser. JS isn't involved, it's just telling the browser to connect to a certain hostname and the browser itself decides which IP to map it to (based on its DNS cache). If the DNS server is down the website wouldn't load at all, but this is an acceptable trade-off considering DNS is a very simple system (not many things can go wrong) and servers can be redundant.
- cutemonster 6y agoThere's a misunderstanding. That reply is to me off topic, I knew about those DNS things already Thanks anyway for replying