5 ms·
I didn't realize message queues were used for this type of task. I'm assuming you would then also use autoscaling pods that respond to the number of messages in
by ngngngng 8y ago
I didn't realize message queues were used for this type of task. I'm assuming you would then also use autoscaling pods that respond to the number of messages in the queue. How do you scale pods fast enough for a messaging application or anything else trying for 100ms or less per operation?
- jcmi 8y agoMessaging queues are a core part of a lot of high scale distributed systems (source - Twitter) You want enough queue space to handle the expected volume and then some. Assuming you have that, you don't need to instantly scale instances out to match the amount of messages, you just need to catch up before the queue space runs out.
- underwater 8y agoFacebook runs its own hardware. How would "autoscaling" help them?
- tybit 8y agoDynamically allocating hardware to different services makes sense for self managed DCs too. Breaking down cost per service of team is useful, as is seeing how much hardware is necessary for peak and trough I imagine. I wouldn’t be surprised if on NYE services like chat take priority and get extra resources from background jobs that can wait a few hours.
- gmmeyer 8y agoMessage queues (or similar things like Kafka, which isn't quite a proper "message queue") are used for basically everything at this scale. Messages are being passed indirectly. An event happens, it gets popped on a queue, and then the recipients do something with it.
- latch 8y agoMaybe you'll consider this pedantic, or maybe even wrong, but I think you meant to say "asynchronous" as opposed to "indirectly". I think anyone googling to learn more will get better results.
- projektfu 8y agoTrue, but there is also the store-and-forward subscription model where the message is placed on the queue without an intended recipient. Message passing can also be asynchronous but direct, e.g., Mach messages.
- latch 8y agoI think over-provisioning is way more common and sane approach that can address the bulk of spikes versus auto-scaling. Especially if you have these big known events (new years day, black friday, ...) where you can over-provision (or controlled auto-scale if you will) for a short window. My guess is they're doing both. Anecdote time. I worked at a company where one project was over-provisioned on dedicated hardware and another auto-scaled in the cloud. The over-provisioned project was much cheaper, had significantly better response times and was easier to manage. It was load tested to handle over an order of magnitude more traffic than the all-time-peak and even though fully over-provisioned, it was cheaper than the baseline usage (and slower, and harder to manage) cloud solution.
- rhizome 8y agoThe back of the napkin numbers I've seen say metal (onprem or colo) has a 100x price/performance advantage: 10x faster at 1/10th the cost.
- sl1ck731 8y agoDo you have any interesting reads about this? Most of the material I find is biased one way or the other by cloud hype or data centers trying to reel back customers.
- rhizome 8y agoFor one, I thought Bitnami had done the comparison a few years back, but for the life of me I can't find it in the GOOG.
- ridruejo 8y agoI don't remember that we did any benchmarks, but I don't doubt that if you look at the raw performance / price it can easily be an order of magnitude more expensive to go to the cloud. But once you take into account all the automation and services that the major cloud providers have, it can get much more cost effective as it saves significant time / expertise for most companies
- int0x80 8y agoOne way is to over-allocate in the first place. When your spare pool is draining below a watermark, you scale in. Hopefuly there is enough time for that scale event to complete before the pool drains completely.