4 ms·
Coiled Computing
- Nextgrid 6y agoI would appreciate if you clarified what Dask is at the beginning of the post. I'm a Python developer and it's the first time I hear about it and I had to scroll pretty much to the end for a link to its documentation.
- falcor84 6y agoI would've appreciated it if you clarified this in your own comment :)
- maxmcd 6y agohttps://dask.org/ https://dask.org/ > Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love edit: removed comments
- RcouF1uZ4gsC 6y ago> It will also shard data between machines to allow analysis on data that is larger than memory. https://yourdatafitsinram.net/ https://yourdatafitsinram.net/
- bearzoo 6y agohttps://distributed.dask.org/en/latest/efficiency.html#don-t-go-distributed https://distributed.dask.org/en/latest/efficiency.html#don-t...
- sgt101 6y agoIt's like a lightweight Spark but more pythonish.
- lordgrenville 6y agoThe target audience isn't all Python developers, but specifically data scientists or engineers who work with the PyData stack, to whom Dask is pretty well known.
- teetertater 6y agoHas anyone here had positive experiences with Dask? I chose it as an alternative to manual batching, where a dataset couldn't fit into memory. Anecdotally, it was great when it worked, but caused a lot of random bugs, and I spent more time fixing them than the time it saved in the first place. (This was mid-2019 so maybe it has improved since then)
- kevlar1818 6y agoI generally share your criticism, but my overall experience has been positive. Dask indeed requires some fiddling, and it's API and documentation has some blind spots, but I've successfully used it to scale up some Pandas code to run on dataframes measuring in the tens of GB. Being able to transfer my knowledge of Pandas and NumPy so readily makes Dask's speed bumps worth it.
- teetertater 6y agoThanks for sharing your experience!
- Optimal_Persona 6y agoIt would be more true to GVR's original inspiration for the language if Python-based businesses named themselves things like "Ministry of Silly Walks", "The Antlers of Dictation", "Tobacconist", or "Bevis, Be Gentle!" rather than "Coiled", "Anaconda", etc. ;-)