2 ms·
In my experience, yes, pickle is painfully slow and can't be used for anything real. It's fantastic for prototyping (especially https://mpi4py.readthedocs.io/en
by gyrovagueGeist 3y ago
In my experience, yes, pickle is painfully slow and can't be used for anything real. It's fantastic for prototyping (especially https://mpi4py.readthedocs.io/en/stable/reference/mpi4py.MPI.Pickle.html https://mpi4py.readthedocs.io/en/stable/reference/mpi4py.MPI...) but it will be your bottleneck. But I work in more computational science so I understand my constraints are different than most folks.
- aragilar 3y agoWhich pickle version did you use (usually the default is relatively old)?
- ptx 3y agoHmm. Apparently version 5 of the pickle protocol (which is faster) isn't yet used by the multiprocessing module [1] even though it was mentioned as part of the rationale in PEP 574 [2]. [1] https://github.com/python/cpython/issues/89467 https://github.com/python/cpython/issues/89467 [2] https://peps.python.org/pep-0574/ https://peps.python.org/pep-0574/