One data structure that helps a lot is the grouped queue. I cover it in the essay here: https://www.instantdb.com/essays/architecture#:~:text=is%20to%20parall

One data structure that helps a lot is the grouped queue. I cover it in the essay here: https://www.instantdb.com/essays/architecture#:~:text=is%20to%20parallelize.-,Grouped%20Queues,-Since%20our%20database https://www.instantdb.com/essays/architecture#:~:text=is%20t... To summarize: In places where we process throughput, we generally stick a grouped queue and a threadpool that takes from it. The mechanics for this queue make it so that if there's one noisy neighbor, it can't hog all the threads. There's more too (runbooks, rate limiting systems, buffers, isolated instances), but I thought this particular data structure was really fun to share.