4 ms·
It seems a bit overkill to spin up a new worker per segment, especially if encountering I/O bottlenecks. Why not make sure to distribute multiple segments to mu
by treeheckler 7y ago
It seems a bit overkill to spin up a new worker per segment, especially if encountering I/O bottlenecks. Why not make sure to distribute multiple segments to multiple threads on the same (large-enough) worker so it can stitch them all together from the same machine?