3 ms·
If you want real irony here, it's this: powering an app called Threads with Python that's got a global interpreter lock which makes threading hard. Get it? Thr
by ambivalence 3y ago
If you want real irony here, it's this: powering an app called Threads with Python that's got a global interpreter lock which makes threading hard.
Get it? Threads, threading...? OK, I'll see myself out.
- hynek 3y agoI guess PEP 703 has to be auto-accepted now. I don't make the rules.
- deleted 3y ago[deleted]
- nerdponx 3y agoMeta does have their own higher-performance JIT-compiled implementation of Python, but I don't know if it still uses a global interpreter lock.
- pookha 3y agoGotten burned by that. Have a python utility that calls something that's very CPU intensive (GDAL) and I literally have to wrap this python process in java for real concurrency. Not a python expert by any means but I'd love to know what other people do when this happens. I'm assuming it's something revolving around containers and messaging (blech)...
- ska 3y agoIs your problem that the c part of GDAL isn't concurrent and you are calling it multiple times, or are you blocking on something you shouldn't? Python calling a concurrent c thing is super common. For more complex things there is multiprocessing, but that essentially is simplification of IPC.
- pookha 3y agoHad a massive amount of images that I was trying to process for a specific purpose. I wound up spinning up multiple python processes within a java wrapper using concurrent linked queues. Got the job done but it's not the proudest moment of my life.
- seunosewa 3y agoWhy did you need the Java wrapper? You can orchestrate stuff like that within Python. The multiprocessing module.
- giantrobot 3y agoYou could have done this with the multiprocessing module. It interfaces like the threading library but it's launching new processes for each "thread". I've done a shitload of image processing in Python with it. You'll end up with a 100% Python version of what you did.
- ska 3y agoMultiprocessing does basically the same thing, but in python.
- 3y ago
- schnable 3y agoNow that is Meta!
- est 3y agoPython does threading fine than most languges. Not every language supports native threading.