3 ms·
FWIW I think df.column.value_counts() is better to use here in pandas.
by sweezyjeezy 3y ago
FWIW I think df.column.value_counts() is better to use here in pandas.
- recursive4 3y agoIt unfortunately also exceeded available memory. A basic approach which worked was sequentially loading each df from the filesystem, iterating through record hashes, and incrementing a counter; however the runtime was an order of magnitude greater than my final implementation in Polars.