3 ms·
It's sad that the standard python multiprocessing doesn't allow you to select your own serializers.
by wdroz 9y ago
It's sad that the standard python multiprocessing doesn't allow you to select your own serializers.
- splike 9y agoIndeed it is. My workaround for the failures of pickle is to use pathos. The API for the multiprocessing module is almost identical to the default multiprocessing module which makes it easy to use as a drop in replacement. Pathos uses Dill to serialise, which can serialise many more things, but you have to pay the price in performance. Although again, you can't swap out Dill for the serialiser of your choice.
- stuaxo 9y agoUrg, when I looked inside the multiprocessing module it made my eyes bleed. There are too many bits that are no customisable like that (amongst other things).