3 ms·
We use it to go faster than spark via dask_cudf: bottleneck becomes pci/ssd, which is in GB/s. For CPU, have not benchmarked latest CPU dask vs CPu spark. Woul
by lmeyerov 5y ago
We use it to go faster than spark via dask_cudf: bottleneck becomes pci/ssd, which is in GB/s.
For CPU, have not benchmarked latest CPU dask vs CPu spark. Would expect to see spark win on simple kernels (pandas vector ops) and lose on ML/C++ ones (ex: igraph vs graphx)
Would be interesting to see carefully done!