27 ms·
In the real world, the slow part of "parsing" a CSV file is IO: reading the file content from disk to memory, and from memory to CPU cache. You would avoid rea
by maxhou 11y ago
In the real world, the slow part of "parsing" a CSV file is IO: reading the file content from disk to memory, and from memory to CPU cache.
You would avoid reading the file content more than once if you had to parse it.
> The first line counts the number of lines in a buffer (assuming that file is read into memory and copied to gpu buffer d_readbuff).
but this is what is done here, first search to find all \n, then multi-core GPU stuff for each line content.
- azurezyq 11y agoThings have been changed in a world of SSDs and machines w/ much memory. Actually, parsing a CSV in a single thread will never reach several hundred megabytes per second.