2 ms·
When reading from media with high random-access latency (for instance traditional hard drives) going parallel could make things much slower, and it will reduce
by dspillett 4y ago
When reading from media with high random-access latency (for instance traditional hard drives) going parallel could make things much slower, and it will reduce the compression achieved (if only by a small amount), and it will increase the amount of memory consumed during the process, so I wouldn't want it to be the default. Nor would I particularly want it to try be clever and detect the usefulness of going parallel as that could lead to unexpected inconsistency.
I think there is a case for including it as a selectable option, as with --rsyncable, unless this adds extra dependencies (pthreads was mentioned in other comments).
- alias_neo 4y agoInteresting, I had no idea this would have such an effect, I simply assumed it would be performed in-memory and written out sequentially. I can agree with a flagged option being a nice idea, but what's so heinous about adding a dependency? I try to avoid it generally in my coffee too, but for something that could potentially offer a measurable benefit in a world of multi core-solid state computing, would it not be "worth it"?