It unfortunately also exceeded available memory. A basic approach which worked was sequentially loading each df from the filesystem, iterating through record h

It unfortunately also exceeded available memory. A basic approach which worked was sequentially loading each df from the filesystem, iterating through record hashes, and incrementing a counter; however the runtime was an order of magnitude greater than my final implementation in Polars.