3 ms·
Update: splitting files to 100M chunks increases search speed by 4x. So it's 1.5 sec to grep through 20GB. Smaller file sizes do not yield improvement.
by freediver 7y ago
Update: splitting files to 100M chunks increases search speed by 4x. So it's 1.5 sec to grep through 20GB. Smaller file sizes do not yield improvement.
- eesmith 7y agoYour machine's RAM bandwidth is about 20GB/second, right? It looks like you're close to that limit. The only way to go faster is to not search everything. Which means pre-computing, or otherwise knowing more about the structure of what you plan to search.