3 ms·
You're mostly right; computers are fast at core/thread parallelism now and leaving capacity on the table is bad. But there are a lot of common problems where t
by zbentley 1y ago
You're mostly right; computers are fast at core/thread parallelism now and leaving capacity on the table is bad.
But there are a lot of common problems where the overhead of threads shows up a lot earlier in the scaling process than you'd think. Probing sets of ~100 URLs to see which are live? Even in a fast, well-threaded language, the overhead of creating/probing/shutting down all those threads adds up to noticeable latency very quickly. Creating lingering threadpools requires pre-planning available thread-parallelism and scheduling (say) concurrent web requests' workloads onto those shared pools with care. Compared to saying "I know these concurrent computations are 99.9% slow IO-wait-bound while probing URLs, let the event loop's epoll take care of it", that's a hassle--and just as hard to get right as manually messing about with select/poll/epoll multiplexers.
You're right that there are a lot of buggy IO multiplexer implementations out there; that kind of stuff is best farmed out to a battle-hardened event loop, ideally one that's distributed with your language runtime. But such systems aren't "google scale only", they're needful in a lot of places for ordinary programs' work--whether or not they're ever coded manually or interacted with directly in those programs.
If you don't often see problems like the hypothetical URL liveness checker, consider the main socket read/write components of a webserver harness: do you really want to spin up a thread as soon as bytes arrive on a socket to be read/written? Or would you rather allocate a precious request-handler thread (or process, or whatever) only after an efficient IO-multiplexed event loop has parsed enough of a request off of the socket to begin handling it? Plenty of popular webservers take both approaches. The ones that prefer evented I/O for their core request flow are less vulnerable to SlowLoris.
There are more benefits to async/await concurrency (and costs, too), including cancellation/timeout semantics, prioritization, and more, but even without getting into those, near-automatic IO multiplexing is pretty huge.
- PaulHoule 1y agoAs a pythoner one of the things that drives me crazy is that none of the concurrency control mechanisms are completely x-platform; different aio loops have different strengths and weaknesses in Windows and if I want to use gunicorn well I better do it in WSL2. I used aiohttp for my YOShInOn RSS reader [1] and Fraxinus image sorter and gave up on it when Fraxinus was serving images and other content with a high concurrency count, it was getting hung up on something and I could either try to debug it with an X% chance of succeeding or just go to gunicorn and I know I can use all the cores on my machine. It is still fronted by IIS and IIS serves the images because that's even better for performance. If I wasn't on my Frankstack it would be nginx. The real problem in web crawlers and such is supporting "one-thread per target server and limited request rate per target server" which is devilishly hard to do with whatever framework you're using on a single machine and even harder when you have a distributed crawler. I think some the rage people have about AI crawlers is the high cost of bandwidth out of AWS, but some of it has to be that the AI crawlers don't even seem to try anymore. [1] https://mastodon.social/@UP8/114887102728039235 https://mastodon.social/@UP8/114887102728039235 really!
- zelphirkalt 1y agoWhat is not working with multiprocessing pool?