Another alternative is to try Modin. They wrap a scalable engine (doesn't have to be multiple machine, speedup happens locally from single -> multithread) with

Another alternative is to try Modin. They wrap a scalable engine (doesn't have to be multiple machine, speedup happens locally from single -> multithread) with Pandas API so you just need to change your imports. .apply() is also automatically parallelized through Modin. https://modin.readthedocs.io/en/stable/getting_started/quickstart.html https://modin.readthedocs.io/en/stable/getting_started/quick...