5 ms·

A few things: Disk access can certainly be parallelized, and NVMe is blazing fast, so the bottleneck is more the CPU than disk. There are systems that are buil

A few things: Disk access can certainly be parallelized, and NVMe is blazing fast, so the bottleneck is more the CPU than disk. There are systems that are built around around modern hardware that realize this (redpanda.com, where I work is one such example) Parsing is a lot of the compute time and some SIMD tricks like SWAR for finding delimiters can be helpful. Stringzilla is a cool library if you want to see a clean implementation of these algorithms: https://github.com/ashvardanian/StringZilla https://github.com/ashvardanian/StringZilla See my reply here about the file being cached completely in memory after the first run: https://news.ycombinator.com/item?id=38864034 https://news.ycombinator.com/item?id=38864034

londons_explore 3y ago
I would hope that any reasonably performant implementation would be faster not only than NVMe, but also faster than CPU to RAM data transfers. The AMD EPYC-Milan in the test server supports memory reads at 150 gigabytes/sec, but thats a 32 core machine, and our test only gets 8 of those cores, so we probably can't expect more than 37 gigabytes per second of read bandwidth. The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so. There are only ~400 weather stations, so all the results fit in cache. It's unclear if it might be faster to avoid converting ascii to a float - it might be faster to add up all the ascii numbers, and figure out how to convert to a float only when you have the total.
- anonymoushn 3y ago
  It looks like all existing solutions are too slow by a huge factor then :)
- vintermann 3y ago
  Are you sure that would give the same result? The first thing that struck me about this challenge is that there's going to be floating point precision issues.
  - londons_explore 3y ago
    All the data is only to one decimal place. You can shift the decimal place and do the whole thing with int's and get a 100% precise result.
    - vintermann 3y ago
      You can, but will it be the same result? :) binary floating point can't represent 0.1 or 0.2 exactly. To be fair, with only additions, and only a billion of them, you're probably fine working in floating point and rounding to one decimal in the end.
      - bmicraft 3y ago
        Just x10 it and count in integers?
        vintermann 3y ago
        I said that works here. The point is that floating point may give a different answer.
- codeflo 3y ago
  > any reasonably performant implementation would be faster not only than NVMe Saturating NVMe bandwidth is wicked hard if you don't know what you're doing. Most people think they're I/O bound because profiling hinted at some read() function somewhere, when in fact they're CPU bound in a runtime layer they don't understand.
  - ykonstant 3y ago
    Can you elaborate on the runtime layers and how to affect them?
- menaerus 3y ago
  > The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so. How? EPYC-Milan features PCIe 4.0 and theoretical maximum throughput of x4 sequential read is ~8GB/s. In bare-metal machines, this figure is rather closer to 5-6GB/s, and in cloud environments such as CCX33, I imagine it could be even less. So, unless I am missing something you'd need ~2 seconds only to read the contents of the ~12GB file.
  - londons_explore 3y ago
    This assumes the file is already in page cache, and therefore already in RAM. The 0.3 seconds is the time to get it from RAM to the CPU L3 cache.
    - mgaunard 3y ago
      That sounds like cheating. The exercise says to process a large file that's on disk. If the data is in RAM then that's a whole different game.
      - brrrrrm 3y ago
        Isn’t it run multiple times and the slowest is dropped? Why would you expect the file to be evicted from ram so quickly?
      - rockwotj 3y ago
        The benchmark is run 5 times without flushing file system caches. If you mmap it, it will still be in memory between process runs. The author intended this FWIW
        PaulDavisThe1st 3y ago
        If the cache is not flushed, it will still be in memory between runs whether you mmap it or not.
        3y ago

londons_explore 3y ago

I would hope that any reasonably performant implementation would be faster not only than NVMe, but also faster than CPU to RAM data transfers. The AMD EPYC-Milan in the test server supports memory reads at 150 gigabytes/sec, but thats a 32 core machine, and our test only gets 8 of those cores, so we probably can't expect more than 37 gigabytes per second of read bandwidth. The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so. There are only ~400 weather stations, so all the results fit in cache. It's unclear if it might be faster to avoid converting ascii to a float - it might be faster to add up all the ascii numbers, and figure out how to convert to a float only when you have the total.

anonymoushn 3y ago
It looks like all existing solutions are too slow by a huge factor then :)
vintermann 3y ago
Are you sure that would give the same result? The first thing that struck me about this challenge is that there's going to be floating point precision issues.
- londons_explore 3y ago
  All the data is only to one decimal place. You can shift the decimal place and do the whole thing with int's and get a 100% precise result.
  - vintermann 3y ago
    You can, but will it be the same result? :) binary floating point can't represent 0.1 or 0.2 exactly. To be fair, with only additions, and only a billion of them, you're probably fine working in floating point and rounding to one decimal in the end.
    - bmicraft 3y ago
      Just x10 it and count in integers?
      - vintermann 3y ago
        I said that works here. The point is that floating point may give a different answer.
codeflo 3y ago
> any reasonably performant implementation would be faster not only than NVMe Saturating NVMe bandwidth is wicked hard if you don't know what you're doing. Most people think they're I/O bound because profiling hinted at some read() function somewhere, when in fact they're CPU bound in a runtime layer they don't understand.
- ykonstant 3y ago
  Can you elaborate on the runtime layers and how to affect them?
menaerus 3y ago
> The total file is ~12 gigabytes, so we should expect to be able to get the task done in 0.3 seconds or so. How? EPYC-Milan features PCIe 4.0 and theoretical maximum throughput of x4 sequential read is ~8GB/s. In bare-metal machines, this figure is rather closer to 5-6GB/s, and in cloud environments such as CCX33, I imagine it could be even less. So, unless I am missing something you'd need ~2 seconds only to read the contents of the ~12GB file.
- londons_explore 3y ago
  This assumes the file is already in page cache, and therefore already in RAM. The 0.3 seconds is the time to get it from RAM to the CPU L3 cache.
  - mgaunard 3y ago
    That sounds like cheating. The exercise says to process a large file that's on disk. If the data is in RAM then that's a whole different game.
    - brrrrrm 3y ago
      Isn’t it run multiple times and the slowest is dropped? Why would you expect the file to be evicted from ram so quickly?
    - rockwotj 3y ago
      The benchmark is run 5 times without flushing file system caches. If you mmap it, it will still be in memory between process runs. The author intended this FWIW
      - PaulDavisThe1st 3y ago
        If the cache is not flushed, it will still be in memory between runs whether you mmap it or not.
        3y ago