4 ms·
I appreciate where you are coming from, but on many of these projects you are just misinformed. The entire point of dask is to enable parallel and bigger than
by shoyer 11y ago
I appreciate where you are coming from, but on many of these projects you are just misinformed.
The entire point of dask is to enable parallel and bigger than memory computations that are not be possible with NumPy arrays. It uses an internal graph representation for deferred computation because deferred computation is basically the only sensible way to do these computations. But in fact, a major point of dask is that users should not need to write these task graphs explicitly. The dask.array API is actually designed to be almost a perfect match for NumPy. The docs do a nice job of explaining the internal abstraction, but understanding it is by no means necessary to use it. (Dask is not my project, but I have made quite a few contributions to it.)
Numba certainly does deviate from mainstream Python, but in my experience it works very much in line with the needs and expectations of scientific python users.
As for Blaze, it's changed a long way since Travis wrote that blog post in 2012. This website provides a better overview of what Blaze is today: http://blaze.github.io/ http://blaze.github.io/. Confusingly, the name is used for both an ecosystem and one of its major components more specifically. Dask is a component of the larger Blaze project.
> Jython, which is the only multithreaded Python interpreter.
This is not quite true. Python supports multithreading, and CPython's GIL is less of a obstacle to numerical computing than you might think: http://matthewrocklin.com/blog/work/2015/03/10/PyData-GIL/ http://matthewrocklin.com/blog/work/2015/03/10/PyData-GIL/