4 ms·
I'm curious what their load balancing layer looks like. There's alot of interesting options. (Disclaimer: I've worked in the CDN and the storage space in the pa
by shanemhansen 6y ago
I'm curious what their load balancing layer looks like. There's alot of interesting options. (Disclaimer: I've worked in the CDN and the storage space in the past)
If their load balancer is smart enough it can call the dispatcher, and make use of something like https://zaiste.net/nginx_x_accel_header/ https://zaiste.net/nginx_x_accel_header/ to figure out where to forward the request. Unfortunately this still requires uploads be proxied through the dispatcher.
You could get crazy and involve a CDN (akamai or cloudflare or fastly) that could do some smart logic, especially if you can emit your dispatcher as a lookup table that's updated frequently. I don't know what bandwidth costs would be for that though. Probably high.
It's an interesting problem space and I'd love to talk to these folks about it.
- ADefenestrator 6y agoHi! Backblaze employee who did some of the LB stuff here. It's relatively standard/straightforward. There's a L4 load balancing layer using IPVS and ECMP-via-BGP, then a custom application that does the actual proxying/forwarding to the appropriate vault.