3 ms·
(Firstly, holy moly, something I wrote is on the front page of Hacker News!? I couldn't figure out why I was suddenly getting twitter messages on my otherwise d
by goostavos 13y ago
(Firstly, holy moly, something I wrote is on the front page of Hacker News!? I couldn't figure out why I was suddenly getting twitter messages on my otherwise dormant account.)
Yeah, the requirement that things be picklable in order to send to another process is one of the worst parts of Python in my opinion. I can write the bulk of my code without running into the limitation, but when I do hit it? Holy crap. Rage.
The only work around I've found, which isn't really a work around at all, is throwing out the idea of small, side effect free functions. By which I mean, give the process everything it needs to get the resources on "its" side (thus avoiding the pickling), and then let it process stuff until it's gotten back to some serializable base object that it can finally return. Not ideal, but that's how I generally work around it.
There's also `multiprocessing.Array` and `Value` which avoid the pickle requirement completely, as they're shared memory, but limit you to very base datatypes. Really only good if your operating on some homogeneous array (which for me isn't too often).