4 ms·
Just for clarification, dask lets you run any python function on any python datatype. Numpy/pandas is faster, but anything is doable with dask. This makes it em
by pletnes 6y ago
Just for clarification, dask lets you run any python function on any python datatype. Numpy/pandas is faster, but anything is doable with dask. This makes it eminently flexible, unlike many «big data» tools that only work on tables, or other arbitrary limitations.
- peterthehacker 6y agoYes, you can use dask’s lower level APIs, like futures [0] or dask delayed, and it’ll just pickle the python objects. Dask provides a group of collections APIs, like dataframes [1] and arrays that use more efficient serialization methods. [0]https://docs.dask.org/en/latest/futures.html https://docs.dask.org/en/latest/futures.html [1]https://docs.dask.org/en/latest/dataframe.html https://docs.dask.org/en/latest/dataframe.html