4 ms·

The GIL has been a blocker for many years. It's nice that the team is making progress of course. IMHO it's one of those bandaids they need to rip off. I was li

The GIL has been a blocker for many years. It's nice that the team is making progress of course. IMHO it's one of those bandaids they need to rip off. I was listening to the interview with Chris Lattner with Lex Friedman a week ago or so. Very interesting discussion on his project mojo which intends to build a new language that is backwards compatible and a drop in replacement for python with opt in strict typing, better support for native/primitive types where this makes sense, easier integrations with hardware optimizations, and of course no GIL. The idea would be that the migration path for existing code is that it should just work and then you optimize it and provide the compiler with type hints and other information so it can do a better job. Very ambitious roadmap and I'm curious to see if they'll be able to deliver. The main goal seems to be to enable programmers to do the things you currently can't do in python because it's too slow in python without running into a brick wall in terms of performance. I mostly work with JVM languages and a few other things but I occasionally do a bit with python as well. I've always liked it as a language but I'm by no means an expert in it. I recently spent a day building a simple geocoder and since I know about the GIL, I went straight for the multi processing library and did not bother with threads. IMHO there's absolutely no point in attempting to use threads with python with the GIL in place. I needed to geocode a few hundred thousand things in a reasonable time frame, so all I wanted to do was use a few different processes concurrently so I could cut down the runtime to something reasonable. Python is ok for single threaded stuff but you run into a brickwall doing anything with multiple processes or threads and juggling state. In the end I just gave up and wrote a bunch of logic that splits the input into files, processes the files with separate processes, waits for that to finish, and then combines the output files. Just a lot of silly boiler plate and abusing the file system for sharing state. It does what it needs to but it feels a bit primitive and backwards and I'm not proud of the solution. Removing the GIL, adding some structured concurrency, and maybe some other features, would make python a lot more capable for data processing. And since there are a lot of people already use python for that sort of thing, I don't think that would be such a bad thing. Data science and data processing are the core use case for python. I don't think people actually care a lot about the raw python performance. It's never been that great to begin with. If it's performance critical, it's mostly being done via native libraries already.

robertlagrant 3y ago
This is an interesting writeup. Could you go for asyncio.gather[0] or TaskTaskGroups[1] these days? Or would that not help? [0] https://docs.python.org/3/library/asyncio-task.html#asyncio.gather https://docs.python.org/3/library/asyncio-task.html#asyncio.... [1] https://docs.python.org/3/library/asyncio-task.html#asyncio.TaskGroup https://docs.python.org/3/library/asyncio-task.html#asyncio....
nologic01 3y ago
> Data science and data processing are the core use case for python indeed. one would almost hope that all the different aspects of "performance" and "concurency", their memory, disk or network profile etc get their own dedicated labels. The conflation of these distinct dimensions is a major source of confusion (and thus a waste of bandwidth).
mixmastamyk 3y ago
Is this io-bound or cpu-bound? Hard to tell from your one word description, “geocode”. Is that local or a network call? If you’ve broken up the input already I’d use the shell to parallelize, ie for &. If network, async is probably what you want.
- jillesvangurp 3y ago
  Mostly IO bound. In this case the actual limit was the API rate limiting of the geocoder I was using. A couple of thousand calls per minute. Quite a bit more than what you can do with a single thread but not quite what a decent laptop would be able to do. Python has blocking IO. So a network call blocks the process. So if you have 250ms response times, you are doing 4 requests per second. Without the GIL, threads would be a good way to scale that. With 10 threads you should be able to do 2400 requests per minute. But with the GIL forget about that. With co-routines and non blocking IO, it could all be single threaded. There are some ways to do that with python of course but then you are going to have to use some specialized frameworks and step away from the standard library a bit. Using the shell vs. the multi processing module is the same difference. I've done both with python. I've a slight preference for the multi processing module so I don't have to deal with bash weirdness on top of all the python boiler plate. The first time I did stuff like that with python was probably 13 years ago or so around 2010. Not a lot has changed or progressed on this front since then in python. The GIL made scaling this unnecessarily hard then and it still does. Threads are a no go because of it, the standard library mostly offers blocking IO, and when you go with multiprocessing things like shared memory are very limited so you end up using files or databases for state. Things like node.js, go, or kotlin would handle this type of work load with a lot less fuss. With Kotlin, I'd be using co-routines and some multi threaded scope to launch them in. Or I could build on some of the Java internals. Or a mix of both. I'd be able to write similar code and choose between blocking or non blocking IO. I'd be forking and joining things. Maybe use channels and flows to pass data around and rely on back pressure to keep things progressing at a more or less optimal rate. Not an option for this project as my client just is more python focused and that's fine. But just signaling that python is a bit out of its depth here. Just singling out kotlin here because I just have spend a lot of time with it. If you have a hammer everything seems a nail. I think python could be so much better but that starts with modernizing its internals. Mojo seems like a huge step in the right direction.

robertlagrant 3y ago

This is an interesting writeup. Could you go for asyncio.gather[0] or TaskTaskGroups[1] these days? Or would that not help? [0] https://docs.python.org/3/library/asyncio-task.html#asyncio.gather https://docs.python.org/3/library/asyncio-task.html#asyncio.... [1] https://docs.python.org/3/library/asyncio-task.html#asyncio.TaskGroup https://docs.python.org/3/library/asyncio-task.html#asyncio....

nologic01 3y ago

> Data science and data processing are the core use case for python indeed. one would almost hope that all the different aspects of "performance" and "concurency", their memory, disk or network profile etc get their own dedicated labels. The conflation of these distinct dimensions is a major source of confusion (and thus a waste of bandwidth).

mixmastamyk 3y ago

Is this io-bound or cpu-bound? Hard to tell from your one word description, “geocode”. Is that local or a network call? If you’ve broken up the input already I’d use the shell to parallelize, ie for &. If network, async is probably what you want.

jillesvangurp 3y ago
Mostly IO bound. In this case the actual limit was the API rate limiting of the geocoder I was using. A couple of thousand calls per minute. Quite a bit more than what you can do with a single thread but not quite what a decent laptop would be able to do. Python has blocking IO. So a network call blocks the process. So if you have 250ms response times, you are doing 4 requests per second. Without the GIL, threads would be a good way to scale that. With 10 threads you should be able to do 2400 requests per minute. But with the GIL forget about that. With co-routines and non blocking IO, it could all be single threaded. There are some ways to do that with python of course but then you are going to have to use some specialized frameworks and step away from the standard library a bit. Using the shell vs. the multi processing module is the same difference. I've done both with python. I've a slight preference for the multi processing module so I don't have to deal with bash weirdness on top of all the python boiler plate. The first time I did stuff like that with python was probably 13 years ago or so around 2010. Not a lot has changed or progressed on this front since then in python. The GIL made scaling this unnecessarily hard then and it still does. Threads are a no go because of it, the standard library mostly offers blocking IO, and when you go with multiprocessing things like shared memory are very limited so you end up using files or databases for state. Things like node.js, go, or kotlin would handle this type of work load with a lot less fuss. With Kotlin, I'd be using co-routines and some multi threaded scope to launch them in. Or I could build on some of the Java internals. Or a mix of both. I'd be able to write similar code and choose between blocking or non blocking IO. I'd be forking and joining things. Maybe use channels and flows to pass data around and rely on back pressure to keep things progressing at a more or less optimal rate. Not an option for this project as my client just is more python focused and that's fine. But just signaling that python is a bit out of its depth here. Just singling out kotlin here because I just have spend a lot of time with it. If you have a hammer everything seems a nail. I think python could be so much better but that starts with modernizing its internals. Mojo seems like a huge step in the right direction.