4 ms·
I didn't keep track of the benchmarks very well. However, it went from taking 3+ days and 180GB of memory to process 80M rows [1] to processing 1.3B rows in ~6
by vitorbaptistaa 3y ago
I didn't keep track of the benchmarks very well. However, it went from taking 3+ days and 180GB of memory to process 80M rows [1] to processing 1.3B rows in ~6 hours using ~90GB of memory.
[1] I stopped the process early, as it was taking too long