4 ms·

You can't just throw hardware at this one to get to 4s. At least not in 2024. The author's naive single-threaded Go solution took 1m45s on an "amd64 laptop wit

You can't just throw hardware at this one to get to 4s. At least not in 2024. The author's naive single-threaded Go solution took 1m45s on an "amd64 laptop with fast SSD drive and 32GB of RAM." So, you'd need something 25x faster than his setup in terms of single-threaded performance. Let us know when you've upgraded your hardware to the equivalent of a 75ghz AMD processor with memory and SSD bandwidth to match!

lmeyerov 3y ago
The nice thing about a GPU soln (ex: python dataframes in cudf, just a few loc) is these generally come down to your IO bandwidth, like a single 2GB/s SSD to a 16-32 GB/s PCIe to 1-2 GPUs running crazy fast. And then buy more cheap SSDs to chain together before buying more/better GPUs :)
- JohnBooty 3y ago
  I guess it depends on what we mean by "throwing hardware at it." GPUs aren't magic. You still need to come up with a parallelizable algorithm. The TL;DR is that the fastest solutions are basically map/reduce with a bunch of microoptimizations for parsing each line. But before you do that, you need to divide up the work. You can't just give each core `file_size_bytes/core_count` chunks of the file because those chunks won't align with the line breaks. So, you need to be clever about that part somehow. Once you've done that, you have a nice map/reduce that should scale linearly up to at least 20 or 30 cores. So in that sense, you can "throw hardware at it." Whether or not any of that is a good fit for GPU acceleration, I don't know. You should try the challenge. It's trickier than you think but surprisingly fun.
  - lmeyerov 3y ago
    Indeed! You may enjoy this talk where I do just that... end-to-end on GPUs, and < 100loc Python: https://www.youtube.com/watch?v=8ZMzsTbfImU https://www.youtube.com/watch?v=8ZMzsTbfImU Your intuition about mapping to kernels is good. Basically all SQL, Polars, DuckDB, Pandas, etc operators are pretty directly mappable to optimized GPU operators nowadays. This includes GPU-accelerated CSV/parquet parsing. This was theoretically true starting maybe 10 years ago, and implemented in practice about 3-5 years ago. These systems allow escape hatches via numbajit etc to do custom kernels, but it's better to stay in pure sql/pandas/etc subsets, which are already mapped and to more careful kernels. To get a feel for times, I like to think about 2 classes: constant overheads and throughput Constant overhead: - JIT'ing. By using pure SQL/pandas/etc, you can avoid most CUDA JIT costs - GPU context creation etc: Similar, after starting and initial memory pool is allocated, it gets reused - Instruction passing: The pandas API is 'eager', so "df1() + df2()" may have a lot of back-and-forth of instructions between CPU<>GPU even if the data doesn't move. Dask & Polars introduce lazy semantics that allow fusion, but GPU implementations haven't leveraged that yet AFAICT. Bandwith limits: - SSD is the biggest killer. Even "Expensive" SSDs are still < 10GB/s, so you need to chain a bunch to get 100B/s ingest - CPU pathways throttle things down again (latency+bandwidth): GDS/GDN lets you skip them - PCIe cards are surprisingly fast nowadays. With PCIe5+, the bottleneck is getting pushed quickly back to the storage, and probably easier to buy more PCIe+GPU pairs than need individual to go faster for most workloads - Once things hit the GPU, things are fast :) 4s is a LOT of time wrt what even commodity GPU hardware can do, so benchmarks showing software failing to saturate it is fascinating to diagnose
    - JohnBooty 3y ago
      Wow! Super informative, thanks!! I also apologize. As you can probably tell, I lumped you in with all the folks who were being super glib about easy hardware gains!

lmeyerov 3y ago

The nice thing about a GPU soln (ex: python dataframes in cudf, just a few loc) is these generally come down to your IO bandwidth, like a single 2GB/s SSD to a 16-32 GB/s PCIe to 1-2 GPUs running crazy fast. And then buy more cheap SSDs to chain together before buying more/better GPUs :)

JohnBooty 3y ago
I guess it depends on what we mean by "throwing hardware at it." GPUs aren't magic. You still need to come up with a parallelizable algorithm. The TL;DR is that the fastest solutions are basically map/reduce with a bunch of microoptimizations for parsing each line. But before you do that, you need to divide up the work. You can't just give each core `file_size_bytes/core_count` chunks of the file because those chunks won't align with the line breaks. So, you need to be clever about that part somehow. Once you've done that, you have a nice map/reduce that should scale linearly up to at least 20 or 30 cores. So in that sense, you can "throw hardware at it." Whether or not any of that is a good fit for GPU acceleration, I don't know. You should try the challenge. It's trickier than you think but surprisingly fun.
- lmeyerov 3y ago
  Indeed! You may enjoy this talk where I do just that... end-to-end on GPUs, and < 100loc Python: https://www.youtube.com/watch?v=8ZMzsTbfImU https://www.youtube.com/watch?v=8ZMzsTbfImU Your intuition about mapping to kernels is good. Basically all SQL, Polars, DuckDB, Pandas, etc operators are pretty directly mappable to optimized GPU operators nowadays. This includes GPU-accelerated CSV/parquet parsing. This was theoretically true starting maybe 10 years ago, and implemented in practice about 3-5 years ago. These systems allow escape hatches via numbajit etc to do custom kernels, but it's better to stay in pure sql/pandas/etc subsets, which are already mapped and to more careful kernels. To get a feel for times, I like to think about 2 classes: constant overheads and throughput Constant overhead: - JIT'ing. By using pure SQL/pandas/etc, you can avoid most CUDA JIT costs - GPU context creation etc: Similar, after starting and initial memory pool is allocated, it gets reused - Instruction passing: The pandas API is 'eager', so "df1() + df2()" may have a lot of back-and-forth of instructions between CPU<>GPU even if the data doesn't move. Dask & Polars introduce lazy semantics that allow fusion, but GPU implementations haven't leveraged that yet AFAICT. Bandwith limits: - SSD is the biggest killer. Even "Expensive" SSDs are still < 10GB/s, so you need to chain a bunch to get 100B/s ingest - CPU pathways throttle things down again (latency+bandwidth): GDS/GDN lets you skip them - PCIe cards are surprisingly fast nowadays. With PCIe5+, the bottleneck is getting pushed quickly back to the storage, and probably easier to buy more PCIe+GPU pairs than need individual to go faster for most workloads - Once things hit the GPU, things are fast :) 4s is a LOT of time wrt what even commodity GPU hardware can do, so benchmarks showing software failing to saturate it is fascinating to diagnose
  - JohnBooty 3y ago
    Wow! Super informative, thanks!! I also apologize. As you can probably tell, I lumped you in with all the folks who were being super glib about easy hardware gains!