4 ms·
The problem is that that completely hiding it in the runtime has to be baked into the language's design, or at the least you have to have designed your language
by geofft 5y ago
The problem is that that completely hiding it in the runtime has to be baked into the language's design, or at the least you have to have designed your language in a way that makes it easy (Haskell is a good example here - because functions are pure, you don't have to worry about implicit concurrent modifications of the same data).
Python, as a language, has a lot of reliance on global context and a lot of dynamic functionality. Imports happen at runtime, not compile time. You can modify the attributes of a class. You can reassign a function. And so forth. You can do nonsense like this, where a isn't even defined until the second-to-last line:
$ python3
>>> def f():
... yield 5
... yield a
...
>>> x = f()
>>> next(x)
5
>>> a = 10
>>> next(x)
10
All of this needs some answer when writing concurrent / parallel code; the answer of "Don't worry about it" just gets you racy and unreliable code. The answer of "Make it impossible" changes the language (which is a fine answer, but you're better off starting from scratch like Go than adapting Python). So the remaining answer is "Make the programmer explicitly acknowledge concurrency."
https://glyph.twistedmatrix.com/2014/02/unyielding.html https://glyph.twistedmatrix.com/2014/02/unyielding.html (which predates Python having proper async-await support by a few years, and references the "Tulip" project that became asyncio) makes a good argument that "Don't worry about it" doesn't work and "Make the programmer explicitly acknowledge concurrency" is fine in practice.
Anecdotally, my (limited) experience writing concurrent Python code in Trio has required no learning of the intricacies of coroutines and the implementation and also relatively easy-to-follow code, but yes, with explicit references to concurrency. Also, anecdotally, since my performance problem is just "I would like to parallelize multiple HTTP requests" and not "I am an OS scheduler and need to optimize every cycle" (in which case I wouldn't be writing Python), I'm happier writing code that accounts for context switches but in turn never has to care about locking than trying to properly manage locks and lock ordering and think about types of locks and all that.
- daenz 5y agoI mostly don't disagree with you. The only point I really disagree on is that Python should have stuck with their limited threads instead of enabling this kind of "safe" concurrency. I think they are letting a genie out of the bottle with async, and it's going to result in making it very easy for people who don't understand what they're doing to write even more complex code that nobody wants to touch because they don't understand it either. Yes you can do very magical things with Python, and no they're not very safe, but at least that level of magic requires some level of skill, and the assumption that with that skill comes a responsibility for putting up warning signs and providing guidance. But most of the time, people should not be doing magic, and if they are, it should be hidden so completely, that it appears no magic is actually happening. Anecdotally, my experience with working with engineers who are trying to understand new async code, has been dismal. I used to think "oh, these engineers just don't want to put in the work to understand what is happening in the codebase." But it's actually hard to explain what is happening to other engineers who have not been "initiated" in async. So they end up writing broken solutions and then feeling frustrated, because there's a resistance to the nature of the abstraction. Now you could argue that they should just hunker down and learn more, but personally I think if new abstractions do not come with a natural intuition, then they are not good abstractions.