4 ms·

It's interesting to compare with Go, which uses goroutines for all concurrency. Python has threading, multiprocessing, async-await and now subinterpreters. Rem

by pansa2 6y ago

It's interesting to compare with Go, which uses goroutines for all concurrency. Python has threading, multiprocessing, async-await and now subinterpreters. Remind me which language is designed to have "one obvious way to do it"...

acdha 6y ago
Those are all different kinds of “it” with differences which affect your application design. If you want a stronger criticism, pick multiprocessing/threading and concurrent, and the long migration trajectory for deprecating the older ones.
- pansa2 6y ago
  > Those are all different kinds of “it” with differences which affect your application design. "It" here is concurrency, and Go proves that its possible for a language to have "one way to do it". Goroutines are often seen as an alternative to async/await, but they're also a better parallelism solution than all of Python's threading, multiprocessing and subinterpreters. > If you want a stronger criticism, pick multiprocessing/threading and concurrent, and the long migration trajectory for deprecating the older ones. I haven't heard anything about deprecating multiprocessing/threading. Is that the long-term plan for subinterpreters?
jonahbenton 6y ago
Yeah, in his later years I think Guido stopped having the energy to keep saying no. Not a criticism. He deserves nothing but plaudits. One of best earned retirements in all of CS history, IMO.
- pansa2 6y ago
  > Yeah, in his later years I think Guido stopped having the energy to keep saying no. https://twitter.com/gvanrossum/status/773593466609610753 https://twitter.com/gvanrossum/status/773593466609610753
- deleted 6y ago
  [deleted]
wrmsr 6y ago
and greenlet, and stackless, and pypy's stm.. python's never really been about dogma. part of why its package management is such a mess is because it's been so successful at integrating with everything under the sun and still needs to support it all.
Doxin 6y ago
Honestly for python there's still one obvious way given just about all use-cases. Use multiprocessing for now. In the future a lot of stuff is going to shift over to async-await. Until enough of your dependencies do it's fine if you ignore it for now. If you need any of the other options you'll probably already know and don't need to ask. Threading is good if you're doing a lot of waiting, but this use case is more or less replaced by async await. Multiprocessing is good multi-purpose. It works around the GIL and so works for most use cases. Async-await is like threading with a nicer programmer interface and works especially well for networking code and the like. The nice thing about async-await is that it's trivially interoperable with all the previous options. When using a framework or library that has an async-await API it'll be doing whichever form works best on the backside. E.g. awaiting a database query might spawn a thread or a process and you don't need to care. Subinterpreters are only relevant if you're writing C-code that embeds the python interpreter. So e.g. the mod_wsgi developers might care. A python developer never uses these directly. So unless you're a library author or doing fairly low-level things you won't need to care about any of the options but async-await pretty soon. The only awkward thing is that async-await isn't widely used/supported yet leading to there currently being two obvious ways to do it depending on what your libraries support. tl;dr; Use multiprocessing if you must. use async-await if you can. Let library developer worry about the tricky bits. Pretty soon just use async-await always.
- pansa2 6y ago
  > Subinterpreters are only relevant if you're writing C-code that embeds the python interpreter. [...] A python developer never uses these directly. The linked article is about making subinterpreters available to pure Python code. > Pretty soon just use async-await always. I can see async/await supplanting threading for I/O-bound concurrency, but it doesn't provide parallelism. For that, maybe subinterpreters will eventually supplant multiprocessing?
  - BiteCode_dev 6y ago
    async / await is just a protocol to delegate async behavior outside of your control flow. You can provide the async behavior using any solution you want: threads, io event loop, subinterpretters, subprocess... It can all be called from async / await. In fact, if you use asyncio, the lib, it provides mechanisme to await from threads, subprocess.Popen, and multiprocessing pools.
BiteCode_dev 6y ago
Python was first released in 1991. It's older than Java. It existed before internet or multi-core was common. You have to balance "one obvious way to do it", "saying relevant" and "backward compatibility". We broke compat once in 25 years to stay relevant with I/O, and gave 13 years to migrate. And still, it was an outrage. Let's see what Go looks like in 2034.

by pansa2 6y ago

acdha 6y ago

Those are all different kinds of “it” with differences which affect your application design. If you want a stronger criticism, pick multiprocessing/threading and concurrent, and the long migration trajectory for deprecating the older ones.

pansa2 6y ago
> Those are all different kinds of “it” with differences which affect your application design. "It" here is concurrency, and Go proves that its possible for a language to have "one way to do it". Goroutines are often seen as an alternative to async/await, but they're also a better parallelism solution than all of Python's threading, multiprocessing and subinterpreters. > If you want a stronger criticism, pick multiprocessing/threading and concurrent, and the long migration trajectory for deprecating the older ones. I haven't heard anything about deprecating multiprocessing/threading. Is that the long-term plan for subinterpreters?

jonahbenton 6y ago

Yeah, in his later years I think Guido stopped having the energy to keep saying no. Not a criticism. He deserves nothing but plaudits. One of best earned retirements in all of CS history, IMO.

pansa2 6y ago
> Yeah, in his later years I think Guido stopped having the energy to keep saying no. https://twitter.com/gvanrossum/status/773593466609610753 https://twitter.com/gvanrossum/status/773593466609610753
deleted 6y ago
[deleted]

wrmsr 6y ago

and greenlet, and stackless, and pypy's stm.. python's never really been about dogma. part of why its package management is such a mess is because it's been so successful at integrating with everything under the sun and still needs to support it all.

Doxin 6y ago

Honestly for python there's still one obvious way given just about all use-cases. Use multiprocessing for now. In the future a lot of stuff is going to shift over to async-await. Until enough of your dependencies do it's fine if you ignore it for now. If you need any of the other options you'll probably already know and don't need to ask. Threading is good if you're doing a lot of waiting, but this use case is more or less replaced by async await. Multiprocessing is good multi-purpose. It works around the GIL and so works for most use cases. Async-await is like threading with a nicer programmer interface and works especially well for networking code and the like. The nice thing about async-await is that it's trivially interoperable with all the previous options. When using a framework or library that has an async-await API it'll be doing whichever form works best on the backside. E.g. awaiting a database query might spawn a thread or a process and you don't need to care. Subinterpreters are only relevant if you're writing C-code that embeds the python interpreter. So e.g. the mod_wsgi developers might care. A python developer never uses these directly. So unless you're a library author or doing fairly low-level things you won't need to care about any of the options but async-await pretty soon. The only awkward thing is that async-await isn't widely used/supported yet leading to there currently being two obvious ways to do it depending on what your libraries support. tl;dr; Use multiprocessing if you must. use async-await if you can. Let library developer worry about the tricky bits. Pretty soon just use async-await always.

pansa2 6y ago
> Subinterpreters are only relevant if you're writing C-code that embeds the python interpreter. [...] A python developer never uses these directly. The linked article is about making subinterpreters available to pure Python code. > Pretty soon just use async-await always. I can see async/await supplanting threading for I/O-bound concurrency, but it doesn't provide parallelism. For that, maybe subinterpreters will eventually supplant multiprocessing?
- BiteCode_dev 6y ago
  async / await is just a protocol to delegate async behavior outside of your control flow. You can provide the async behavior using any solution you want: threads, io event loop, subinterpretters, subprocess... It can all be called from async / await. In fact, if you use asyncio, the lib, it provides mechanisme to await from threads, subprocess.Popen, and multiprocessing pools.

BiteCode_dev 6y ago

Python was first released in 1991. It's older than Java. It existed before internet or multi-core was common. You have to balance "one obvious way to do it", "saying relevant" and "backward compatibility". We broke compat once in 25 years to stay relevant with I/O, and gave 13 years to migrate. And still, it was an outrage. Let's see what Go looks like in 2034.