3 ms·
Check out dask for distributed and out of core parallel programming : http://www.slideshare.net/continuumio http://www.slideshare.net/continuumio Its free with
by tanlermin 10y ago
Check out dask for distributed and out of core parallel programming : http://www.slideshare.net/continuumio http://www.slideshare.net/continuumio
Its free with a permissive license.
It is also capable of native HDFS integration, Yarn etc and can do more complex and granular parallel patterns than just map reduce. Also has a API for distributed dataframes and arrays with linear algebra ops.
DISCLAIMER: I don't work for continuum. I just want to see its projects succeed because I was a user will benefit.
- gshulegaard 10y agoI think you might be interested by this talk: https://www.youtube.com/watch?v=gVBLF0ohcrE https://www.youtube.com/watch?v=gVBLF0ohcrE
- tanlermin 10y agoThanks...though the whole "GIL being a feature" is sort of a joke.
- gshulegaard 10y agoI believe he meant it as a joke. At least that's how I interpreted it :)