6 ms·
Why is multithreaded performance important, what are the usecases where you cannot run multiple processes to spread your numbercrunching across CPUs? I am pray
by TekMol 5y ago
Why is multithreaded performance important, what are the usecases where you cannot run multiple processes to spread your numbercrunching across CPUs?
I am praying for CPython to become faster. But I need faster singlethreaded performance, so web applications benefit from it.
- kzrdude 5y agoInteractive use of Python: plotting, working with data, also would benefit from better multithreading. It's interactive, so it's (a bit) frustrating to wait for it to compute and see that it uses just one thread (the statistics ops are usually well threaded already, but plotting is not).
- chrisseaton 5y agoCommunicating between processes is more expensive than communicating between threads.
- TekMol 5y agoOk, but what is the use case where you need high bandwidth inter-thread/process communication?
- adgjlsfhk1 5y agomatrix multiply is an obvious one. partial differential equations are another. Sorting is one if you don't care about math.
- moron4hire 5y agoGames. Simulations. Backend servers for shared editor experiences. Teleconferencing.
- chrisseaton 5y agoExcept for embarrassingly parallel problems, the trade-off of generating more parallelism is usually needing finer-grained communication. Canonical examples in the literature are matrix multiplication, triangulation, and refinement.
- Spivak 5y agoLarge shared state is basically always the answer. You can cop-out and say use a database or Redis if that’s fast enough but that’s just making someone else use many threads with shared memory.
- VWWHFSfQ 5y agomultiple processes use a lot more memory than threads
- TekMol 5y agoCan you quantify that?
- redeyed 5y agoThere is such problem on Windows. When you create new process (spawn) it copies all memory of parent process.
- SkittyDog 5y agoYes, you can
- lazide 5y agoEven with copy-on-write? Have any actual numbers?
- SkittyDog 5y agoActual work is left as an exercise for the reader ;-)
- lazide 5y agoLast I did this, when the processes were fork()‘s of the parent (the typical way this was done), memory overhead was minimal compared to threads. A couple %. That was somewhat workload dependent however, if there is a lot of memory churn or data marshaling/unmarshalling happening as part of the workload, they’ll quickly diverge and you’ll burn a ton of CPU doing so. Typical ways around that include mmap’ng things or various types of shared memory IPC, but that is a lot of work.
- kaba0 5y agoWhat about context switches as well, much slower IPC, and basically “no native support”.
- rbjorklin 5y agoOne thing that comes to mind is for shared connection pools when you have thousands of workers connecting to the same stateful service.
- TekMol 5y agoWhich use case requires such a setup?
- lazide 5y agoMost databases (unfortunately), and since a ton of web apps use databases for state.. That said, many such databases are (or already have) rolled out connection pool proxies for reasons like this, so meh.
- lanstin 5y agoWeb front end backed by data base with expensive login, so you cache connections. Two tier architecture it would be called.
- Spivak 5y agoMySQL comes to mind. Unlike Postgres where connections are expensive MySQL encourages loading up the server with hundreds of simultaneous connections per server. Like anything, it’s possible to split this work out to a separate process but the IPC overhead is a lot.
- lostdog 5y agoHere's a use case: I was training a neutral net, and wanted to do some preprocessing (similar to image resizing, but without an existing C function). Inputs are batched, so the preprocessing is trivially parallelizable. I tried to multithread it in python, and got no speedup at all. That was a really sad moment, and I've never felt good about python since.
- isoprophlex 5y agoYeah, I had something similar. I wanted "as you wait for the GPU to churn through this batch, start reading the next batch from disk & preprocessing it on the CPU" Getting this to work turned out so ass backwards it made me sad Also I pity the fool who tries to connect a debugger to code using multiprocessing.Pool()...
- redeyed 5y ago> Also I pity the fool who tries to connect a debugger to code using multiprocessing.Pool() PyCharm works fine
- redeyed 5y agoYeah, had the exact problem. I solved it by using python threads and C function (cv2.resize)