4 ms·
>If IO is a bottleneck it would be much worse while trying to swap around text lines in a giant memory mapped file. There's no swapping of lines in a giant mem
by afiodorov 6y ago
>If IO is a bottleneck it would be much worse while trying to swap around text lines in a giant memory mapped file.
There's no swapping of lines in a giant memory mapped file, the byte array of the file remains intact.
The program only swaps the line markers.
> Just convert the date to 64 or 32 bit integer
This would require extra memory for each line, complicate user interface (data formats vary a lot) and make the utility less generic, since you don't always want to sort chronologically. The current idea is to sort using as little extra memory as possible. This is because when files are already so big fitting them into RAM is a challenge already.
---
Finally, there's no evidence why the proposed solution would be faster. Empirically I have found that sorting chunks and merging is considerably slower than sorting in memory, which makes sense given how much faster RAM is than HDD.