5 ms·

> Expose multiple interpreters to Python code > Implement PEP 554 > PEP 554 - Multiple Interpreters in the Stdlib That's going to be fun. Why fight the GIL w

> Expose multiple interpreters to Python code > Implement PEP 554 > PEP 554 - Multiple Interpreters in the Stdlib That's going to be fun. Why fight the GIL when multithreading, when you can just get around it with more interpreters?

isthisthingon99 4y ago
If all global state is made thread safe and, then whether threads are subinterpreters or a single interpreter is conceptually irrelevant and probably easier to implement.
- kjeetgill 4y ago
  It really really depends on what you mean by global state here. If you mean the global state within the interpreter that's one thing. Preserving the global state of your application is another. But a weird "global state" (really more a global property) is the semantics between concurrent pieces of code and the expectations about things like setting variables, possibly interleavings etc. The nice part of different interpreters isn't just getting around the gil, and maintaining similar isolation, but it's almost like a Terms and Services agreement: I opened this can of worms and it's my responsibility to learn what the consequences are.
- forrestthewoods 4y ago
  > conceptually irrelevant and probably easier to implement. Well, it depends on how it’s implemented. If “made thread safe” means constantly grabbing locks around large blocks of data then the end result is concurrency (hopefully!) but not parallelism. Meaning you might only have one thread active at a time in practice. Wrapping the universe in a mutex is thread safe. But it’s not a good solution.
  - isthisthingon99 4y ago
    I presume they know what they are doing and won't be doing a big world mutex :-)
- int_19h 4y ago
  It is not conceptually irrelevant. With threads, you can create a Python object on one, store it into a global (or some shared state), and use that object from a different thread. You can't do that with subinterpreters tho.
  - samsquire 4y ago
    Couldn't you transfer ownership between subinterpreters with shared memory outside the subinterpreter? Associate some shared memory with each subinterpreter (the same array or map)
    - int_19h 4y ago
      It's not a question of having a mechanism to transfer data. Sure, you can easily use a static global in a native module to easily transfer a reference across subinterpreter boundaries. But the moment you try to increment refcount for the referenced object, things already break, because you're going to be using the wrong (subinterpreter-global) lock.
      - samsquire 4y ago
        Oh I was thinking from a native python object perspective. You could have a rule that the refcount must be 1 when sending an object between subinterpreters. In other words, you cannot use an object that was .send() to another subinterpreter. Then you invalidate the reference in that subinterpreter when it calls send to the other subinterpreter which is transferred by assignment. Can transfer any amount of data with zero copies.
        int_19h 4y ago
        Even a completely empty object will contain a reference to its type, which is itself an object. How will you marshal that? Bear in mind that each subinterpreter has its own copy of type objects, and there isn't even a guarantee that those types match even if their names do.
simonw 4y ago
I think this is so smart. The main thing holding back replacement of the GIL at the moment is that there is a VAST existing ecosystem of Python packages written in C/etc that would likely break without it. Multiple interpreters with their own GIL keep all of that existing code working without any changes, and mean we can run a Python program on more than one CPU at the same time.
- ehsankia 4y ago
  But isn't this basically just nicer multiprocessing?
  - cozzyd 4y ago
    It's much nicer if you're using the C API...
  - mixmastamyk 4y ago
    No, they are in-process and able to access the same memory, although probably put behind a communication layer.
- rtpg 4y ago
  I do think that there’s been a lot of work around GIL removal, and every talk seems to end at the reality that the GIL allows for avoiding of a loooooot of locking structures and when removing the GIL you end up needing many granular locks.
- lazka 4y ago
  Only C extensions that themselves have no global state and don't depend on the GIL for locking, which most of them do. So they will all require some porting, and it will take time since it requires newer CPython API only available in 3.9+ and some even 3.11+ (PEP 630).
DannyBee 4y ago
It comes at a cost, of course. You don't really have shared memory state, which is often easiest to conceptually think about. So you are just transforming the problem into a data sharing problem between interpreters, which requires careful thought on both the language side for abstractions, and the consumer side to use right. It also makes the tooling and verification much harder in practice - for example, you aren't reasoning about deadlocks in a single process anymore, but both within a single process and across any processes it communicates with. At an abstract level, they are transformable into each other. At a pragmatic level, well, there is a good reason you mostly see tooling for single-process multi-threaded programs :)
- girvo 4y ago
  > which is often easiest to conceptually think about Absolutely, but is also the easiest to shoot yourself in the foot with. Trade-offs! I'm biased though, I'm a big fan of deep-copy channels (which for small shallow objects is still fast), though not having the option at all for shared memory here will be a bit of a pain for certain things of course.
fny 4y ago
How is this different from multiprocessing? The examples looks like a complete nightmare... interp = interpreters.create() interp.run(tw.dedent(""" import some_lib import an_expensive_module some_lib.set_up() """)) wait_for_request() interp.run(tw.dedent(""" some_lib.handle_request() """)) I'm actually shocked this is even being contemplated. We've regressed to evaling?
- joppy 4y ago
  It's the minimum needed to get something going, and any really sane use of it would be evalling something like import worker worker.run() The PEP explicitly mentions this, and that something like subinterpreter.run(func, ...) could be considered in the future: https://peps.python.org/pep-0554/#interpreter-call https://peps.python.org/pep-0554/#interpreter-call
- ynik 4y ago
  There's a massive difference to multiprocessing: the different sub-interpreters can use a C(++)/Rust extension module to talk to shared state. In the current multi-processing world, the whole C++/Rust state needs to be duplicated for each process (in the case of our app, this means 5 GB memory usage per core); with subinterpreters, we can share the same C++/Rust state. The `interpreters` API is just the starting point. Compare it with `subprocess`, not with `multiprocessing`. Once subinterpreters are useful, people will build higher-level APIs for them.
- BiteCode_dev 4y ago
  It's just a poc to show indecently running codes, the exec() is not how you will use it, but it simulates importing python code and running it (because an exec() is what import does :)).

isthisthingon99 4y ago

If all global state is made thread safe and, then whether threads are subinterpreters or a single interpreter is conceptually irrelevant and probably easier to implement.

kjeetgill 4y ago
It really really depends on what you mean by global state here. If you mean the global state within the interpreter that's one thing. Preserving the global state of your application is another. But a weird "global state" (really more a global property) is the semantics between concurrent pieces of code and the expectations about things like setting variables, possibly interleavings etc. The nice part of different interpreters isn't just getting around the gil, and maintaining similar isolation, but it's almost like a Terms and Services agreement: I opened this can of worms and it's my responsibility to learn what the consequences are.
forrestthewoods 4y ago
> conceptually irrelevant and probably easier to implement. Well, it depends on how it’s implemented. If “made thread safe” means constantly grabbing locks around large blocks of data then the end result is concurrency (hopefully!) but not parallelism. Meaning you might only have one thread active at a time in practice. Wrapping the universe in a mutex is thread safe. But it’s not a good solution.
- isthisthingon99 4y ago
  I presume they know what they are doing and won't be doing a big world mutex :-)
int_19h 4y ago
It is not conceptually irrelevant. With threads, you can create a Python object on one, store it into a global (or some shared state), and use that object from a different thread. You can't do that with subinterpreters tho.
- samsquire 4y ago
  Couldn't you transfer ownership between subinterpreters with shared memory outside the subinterpreter? Associate some shared memory with each subinterpreter (the same array or map)
  - int_19h 4y ago
    It's not a question of having a mechanism to transfer data. Sure, you can easily use a static global in a native module to easily transfer a reference across subinterpreter boundaries. But the moment you try to increment refcount for the referenced object, things already break, because you're going to be using the wrong (subinterpreter-global) lock.
    - samsquire 4y ago
      Oh I was thinking from a native python object perspective. You could have a rule that the refcount must be 1 when sending an object between subinterpreters. In other words, you cannot use an object that was .send() to another subinterpreter. Then you invalidate the reference in that subinterpreter when it calls send to the other subinterpreter which is transferred by assignment. Can transfer any amount of data with zero copies.
      - int_19h 4y ago
        Even a completely empty object will contain a reference to its type, which is itself an object. How will you marshal that? Bear in mind that each subinterpreter has its own copy of type objects, and there isn't even a guarantee that those types match even if their names do.

simonw 4y ago

I think this is so smart. The main thing holding back replacement of the GIL at the moment is that there is a VAST existing ecosystem of Python packages written in C/etc that would likely break without it. Multiple interpreters with their own GIL keep all of that existing code working without any changes, and mean we can run a Python program on more than one CPU at the same time.

ehsankia 4y ago
But isn't this basically just nicer multiprocessing?
- cozzyd 4y ago
  It's much nicer if you're using the C API...
- mixmastamyk 4y ago
  No, they are in-process and able to access the same memory, although probably put behind a communication layer.
rtpg 4y ago
I do think that there’s been a lot of work around GIL removal, and every talk seems to end at the reality that the GIL allows for avoiding of a loooooot of locking structures and when removing the GIL you end up needing many granular locks.
lazka 4y ago
Only C extensions that themselves have no global state and don't depend on the GIL for locking, which most of them do. So they will all require some porting, and it will take time since it requires newer CPython API only available in 3.9+ and some even 3.11+ (PEP 630).

DannyBee 4y ago

It comes at a cost, of course. You don't really have shared memory state, which is often easiest to conceptually think about. So you are just transforming the problem into a data sharing problem between interpreters, which requires careful thought on both the language side for abstractions, and the consumer side to use right. It also makes the tooling and verification much harder in practice - for example, you aren't reasoning about deadlocks in a single process anymore, but both within a single process and across any processes it communicates with. At an abstract level, they are transformable into each other. At a pragmatic level, well, there is a good reason you mostly see tooling for single-process multi-threaded programs :)

girvo 4y ago
> which is often easiest to conceptually think about Absolutely, but is also the easiest to shoot yourself in the foot with. Trade-offs! I'm biased though, I'm a big fan of deep-copy channels (which for small shallow objects is still fast), though not having the option at all for shared memory here will be a bit of a pain for certain things of course.

fny 4y ago

How is this different from multiprocessing? The examples looks like a complete nightmare... interp = interpreters.create() interp.run(tw.dedent(""" import some_lib import an_expensive_module some_lib.set_up() """)) wait_for_request() interp.run(tw.dedent(""" some_lib.handle_request() """)) I'm actually shocked this is even being contemplated. We've regressed to evaling?

joppy 4y ago
It's the minimum needed to get something going, and any really sane use of it would be evalling something like import worker worker.run() The PEP explicitly mentions this, and that something like subinterpreter.run(func, ...) could be considered in the future: https://peps.python.org/pep-0554/#interpreter-call https://peps.python.org/pep-0554/#interpreter-call
ynik 4y ago
There's a massive difference to multiprocessing: the different sub-interpreters can use a C(++)/Rust extension module to talk to shared state. In the current multi-processing world, the whole C++/Rust state needs to be duplicated for each process (in the case of our app, this means 5 GB memory usage per core); with subinterpreters, we can share the same C++/Rust state. The `interpreters` API is just the starting point. Compare it with `subprocess`, not with `multiprocessing`. Once subinterpreters are useful, people will build higher-level APIs for them.
BiteCode_dev 4y ago
It's just a poc to show indecently running codes, the exec() is not how you will use it, but it simulates importing python code and running it (because an exec() is what import does :)).