3 ms·
Think about the datascience use case: you need to load data from disk or network as fast as possible and compute a lot of CPU-bound operations right after that.
by Fiahil 5y ago
Think about the datascience use case: you need to load data from disk or network as fast as possible and compute a lot of CPU-bound operations right after that.
Threads will allow you to split your I/O in multiple procedures, so you might start computations as soon as possible when the data is ready. They will also allow you to massively speed up aggregates without having to create a new process each time (which doesn't allow you to share memory). Threads are a BIG issue when you don't want to rely on asyncio[1].
Note [1]: Asyncio pools are single-threaded because of the GIL. This is already bad enough in practice, but they also perform very badly in CPU-bound contexts. This make them an absolute no-go when dealing with datascience code: a cpu-bounded core in an i/o-bounded wrapper.
- alanfranz 5y agoYou can release the GIL in extensions, and a lot of datascience libraries do that, so not all python computations are single threaded.
- Fiahil 5y agoSupport is partial and uneven. Moreover, you don't generally use a single datascience library, it's often a mix of pandas, numpy, scikit-learn, custom algorithms, custom optimization suite, and arrow. Letting individual library release the GIL is nice, but you need deep knowledge of those library to know what computation are thread-able or not.
- xmcqdpt2 5y agoIn practice, for single-machine workloads, it's currently mostly numpy and/or whatever deep learning framework you use that does the number crunching. This means that provided the code is operating on sufficiently large amounts of data (such that calls to numpy are each of sufficient duration), the multithreading in BLAS / Lapack within numpy usually give you weak scaling wrt to thread count without any tricks. The issue however is that this require by hand making everything into structs of arrays from arrays of structs, removing as many iterations from python as possible, potentially balancing thread usage within python and within numpy etc. By this point IMO your "python" code looks more like Fortran or SQL with better string IO...
- Fiahil 5y agoThe number crunching part is already fast enough, however the aggregations, parsing and filtering that come beforehand is really really slow. This part is often done in pure python because it's often custom code tailored to the data you're manipulating. We're not interested in scaling up the parts that are already fast, but the rather mundane, uninteresting work that come before.