2 ms·
Without details on the task he's running and the setup of his cluster, it's hard to know how meaningful these results are. Using a variation of the example, I'v
by michaelfairley 16y ago
Without details on the task he's running and the setup of his cluster, it's hard to know how meaningful these results are. Using a variation of the example, I've seen roughly 80% improvement with each doubling of the cluster size (testing up to 100 workers).
- dscannell 16y agoHow big was the dataset that you saw the 80% improvement? Remember this test was done on a relatively small dataset, 28 MB, so it may have reached a diminishing point a lot faster than with a larger dataset.
- michaelfairley 16y agoAh yes, I missed the part about the size of the dataset. I'm running with a 3GB corpus of text.