4 ms·
Just recently implemented connected components in a spark job that runs on a 100TB of data.
by codingslave 6y ago
Just recently implemented connected components in a spark job that runs on a 100TB of data.
- bitL 6y agoHow long did DFS take? In my experience, with Spark it's like factor of 100 slower than what one would expect on a single machine and a sizable dataset.