There are quite some ways to optimize pandas usage for performance on bigger datasets. But if the amount of data gets too big there is a library called dask th

There are quite some ways to optimize pandas usage for performance on bigger datasets. But if the amount of data gets too big there is a library called dask that looks promising.