5 ms·
Getting rid of the GIL will also immediately expose all the not-thread-safe stuff that currently exists, so there's a couple of waves you would need before it w
by tyingq 4y ago
Getting rid of the GIL will also immediately expose all the not-thread-safe stuff that currently exists, so there's a couple of waves you would need before it would be broadly usable.
- tempest_ 4y agoCool, they should start now. As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language. Single threaded performance is not that useful while processors have been growing sideways for 10 years. I often look at elixir with jealousy.
- oofbey 4y agoOn the flip side, if your workload can be parallelized across thousands of cores, python has about the best CUDA support anywhere.
- Nimitz14 4y agoAs another python dev, it has basically never been a painpoint for me. Your anecdote adds little.
- IceWreck 4y agoOr maybe keep things the way they are.If you really need performance python is not the language you should be looking for. Instead of breaking decades of code, maybe use a language like Go or Rust for performance instead.
- xwdv 4y agoI would have thought convincing people they’ll just have to use Go or Rust or Elixir would have been an easy sell around here. Turns out they just want a better Python.
- fnord123 4y ago>Turns out they just want a better Python. That's Go. It gives actual types[1] and structs (so you dont have to wonder about dict, class, class with slots, dataclasses, pydantic, attrs, cattrs, marshmallow, etc). It removes exceptions and monkeypatching. It's async-first (a bit like gevent). It's inherently multicore. And you can package and distribute it without marking a pentagram on the ground and sacrificing an intern to Apep. You just need to stop being friends with C FFI. Which is fine for web and sysadmin tools. For data science and ML/AI/DL, it's less ok. [1] And the types are actually checked! I think a lot of people aren't really using python type checking given how slow it is and no one seems to complain. Or maybe everyone is using pyright and pyre and are satisfied with how slow these are.
- kaba0 4y agoGoing to the very unexpressive Go from the expressivity of python is a goddamn huge jump though. Going to JS or even TS for performance would be saner, and it has a same-ish object model even.
- fnord123 4y agoThe expressivity in Python is a problem that needs to be solved though. Moving to JS goes the wrong way.
- 4y ago
- Galanwe 4y ago> As a python dev, pythons multiprocess/multithreading story is one the largest pain points in the language. Hmm, how is that so? As a python dev as well, I don't have much complaint with multiprocessing. The API is simple, it works OK, the overall paradigm is simple to grok, you can share transparently with pickle, etc.
- nine_k 4y agoMultiprocessing is fine, but the cost of starting another interpreter is pretty visible, so you need to create a pool, and it may not be an overall speedup if the run time is short. It takes more careful planning than async in JS?, say, or goroutines.
- geysersam 4y agoYeah but for JS style async you'd use probably use an event loop in Python, not multi processing.
- nine_k 4y agoYes. But, frankly, async is also simpler in JS than in Python: e.g. no need to start a reactor loop.
- Too 4y agoStarting the event loop is no worse than any setup of a main function, it’s a oneliner: asyncio.get_event_loop().run_until_completion(my_async_main)
- BerislavLopac 4y agoErrr no, that has been replaced with asyncio.run quite some time ago.
- plonk 4y agoPickle isn't transparent though, custom objects that wrap files or database sessions need to override serialization. The ProcessPoolExecutor is nice but shouldn't be necessary.