3 ms·
What kind of memory efficiency gains did you see as a result of the effort?
by bredren 3y ago
What kind of memory efficiency gains did you see as a result of the effort?
- vitorbaptistaa 3y agoI didn't keep track of the benchmarks very well. However, it went from taking 3+ days and 180GB of memory to process 80M rows [1] to processing 1.3B rows in ~6 hours using ~90GB of memory. [1] I stopped the process early, as it was taking too long