9 ms·
> avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent thre
by lmm 6y ago
> avoid the per-thread stack memory allocation of 8MB per thread or whatever it is, in order to be able to scale to an extremely large number of concurrent threads/coroutines. You can't do this with threads.
It's 8 K B per thread, so you can scale a thousand times further than you thought. One dark secret of the async movement is that if your goal is C10K (10,000 concurrent clients) then actually bog standard threading will handle that fine these days.
> The better async code and interfaces become, the more it looks like regular old multithreaded code, except there aren't actual threads underlying it. You still need to make sure you serialise access to shared resources, don't share data that shouldn't be shared, etc. All the same considerations as multithreaded code.
Depends what approach you're using. I prefer making an explicit distinction between sync and async functions ( https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html ), so you effectively invert the notion of a "critical section" - instead of marking which sections can't yield, you mark which sections can yield, so your code is safe by default and you can introduce concurrency explicitly as and when you need it for performance, rather than your code being fast-but-unsafe by default and you're expected to fix a bunch of rare nondeterministic bugs with minimal support from your tools, which is how it works in a multithreading world.
- mlyle 6y ago> It's 8 K B per thread, so you can scale a thousand times further than you thought. One dark secret of the async movement is that if your goal is C10K (10,000 concurrent clients) then actually bog standard threading will handle that fine these days. Default virtual memory allocation for threads on Linux distributions tends to be 8 megabytes. Actual memory used is the peak stack depth used, rounded up a bit. It'd be pretty unusual to only use as little as 8 kilobytes per thread; just the standard per-thread libc context information for concurrency is a few kilobytes, plus at least one page of stack, plus the kernel's information about the thread (which isn't counted against the process)... Yes, you can spawn thousands of threads on relatively modest hardware; I was spawning thousands of threads a decade ago. Spawning 5000 bare-minimal python threads that do nil seems to use about 300 megs of ram on my system; real threads that do anything substantial will use a whole lot more, even if their use of the stack depth is intermittent. Not to mention allocators that cache part of freed heap per-thread, etc.
- gpderetta 6y agoI believe 8k is the size of the per thread stack on the kernel side. This is non-pageable memory, so it will consume physical memory whether it is needed or not, while of course the 8 megabytes is paged in on demand.
- mlyle 6y agoYah, I'm ignoring the kernel stack and all kernel data structures. User space memory used will be at least a page of stack (reaching up to the maximum amount used in the thread), plus the libc reentrancy data structures, plus per-thread heap caches, etc. It's can all get paged out, but we hardly want that these days. The key distinction is that the 8MB of stack VM doesn't have a backing until the memory is used in the thread, but afterwards it does forever.
- lmm 6y agoWell, it's forever if you assume that the threads live forever; for a web server it's perfectly practical to have threads that only live for a single request, or to reuse them for multiple requests but not allow a single thread to live longer than say 10 minutes.
- mlyle 6y agoDecent nits, but... Short-lived threads (for one request) are a performance and scalability disaster; tens of microseconds or worse to spawn and join, contention on important locks, bad for caches, etc. There's not much concurrency when it comes to spawning threads, too. If you have long-lived threads in a pool, yes, they may not live forever, but you generally have to assume that each thread will end up with a resident stack size equal to the largest stack use: each will get a turn to run the stack-intensive functions.
- hombre_fatal 6y agoI'd argue that the main benefits of day-to-day async programming isn't performance but actually the concurrency patterns that help you sequence your code and resource access in ways that `thread { work() }` could not. For example, future/result combinators and `await [task1, task2.then(task3)]`.
- gpderetta 6y agoyou could (and can in many languages) use futures with threads just fine though.
- lmm 6y agoNot really - it gets very messy because every time you transform a future you have to figure out where you're getting the thread for that transformation to run on.
- gpderetta 6y agosorry, what transformation? A future is simply a placeholder for something being computed asynchronously. On a threadful design you would simply spawn a thread (or pick one from a thread pool) to handle the computation. Normally your future runtime would handle it for you. Basically you end up with something similar to the fork-join model.
- lmm 6y ago> sorry, what transformation? Whenever you want to transform a result that's in a future, e.g. you have a future for a number and want to add 2 to it. > On a threadful design you would simply spawn a thread (or pick one from a thread pool) to handle the computation. If you allow yourself to spawn threads everywhere you'll quickly run out of resources. So you have to manage which thread pool you're using where and ensure you're not bringing in priority inversions etc.. It's really not that easy. > Basically you end up with something similar to the fork-join model. The fork-join model isn't really a purely thread-based model - the work-stealing technique is pretty much trying to reimplement what async-style code would do naturally.
- deleted 6y ago[deleted]