4 ms·
> The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so. How? EPYC-Milan features PCIe 4.0 and theoretical
by menaerus 3y ago
> The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so.
How? EPYC-Milan features PCIe 4.0 and theoretical maximum throughput of x4 sequential read is ~8GB/s.
In bare-metal machines, this figure is rather closer to 5-6GB/s, and in cloud environments such as CCX33, I imagine it could be even less.
So, unless I am missing something you'd need ~2 seconds only to read the contents of the ~12GB file.
- londons_explore 3y agoThis assumes the file is already in page cache, and therefore already in RAM. The 0.3 seconds is the time to get it from RAM to the CPU L3 cache.
- mgaunard 3y agoThat sounds like cheating. The exercise says to process a large file that's on disk. If the data is in RAM then that's a whole different game.
- brrrrrm 3y agoIsn’t it run multiple times and the slowest is dropped? Why would you expect the file to be evicted from ram so quickly?
- rockwotj 3y agoThe benchmark is run 5 times without flushing file system caches. If you mmap it, it will still be in memory between process runs. The author intended this FWIW
- PaulDavisThe1st 3y agoIf the cache is not flushed, it will still be in memory between runs whether you mmap it or not.
- marginalia_nu 3y agoThat depends a lot on the OS. I would not count on a 13 GB file being cached in RAM after having been read through once.
- mgaunard 3y agoThat sounds like a benchmarking flaw. But then, writing good benchmarks is an art in itself.
- menaerus 3y agoThat makes sense now, thanks.