3 ms·
if you’re at cluster scale, i would skip right over dask and go to spark
by whimsicalism 2y ago
if you’re at cluster scale, i would skip right over dask and go to spark
- 2-3-7-43-1807 2y agoor hadoop
- cpcloud 2y agoI'll also suggest inscribing stone tablets with a chunked version of your computation and distributing the described chunked computation to your friends, and then gathering the tablets and finishing up the compute. It's scalable to ~billions of nodes.
- 2-3-7-43-1807 2y agobut that exists already and it's called databricks