5 ms·
This seems to be conflating two levels of concurrency. One is at the user level, where the user may want to write a function that atomically removes an element
by jerf 15y ago
This seems to be conflating two levels of concurrency. One is at the user level, where the user may want to write a function that atomically removes an element from one list and adds it to another, and one is at the interpreter level, where it needs to be able to complete all operations involved in, say, adding the element to the list, without another thread coming in and stomping on the first thread, with all the associated hazards.
At the interpreter level, as the article mentions, it suffices to lock the structures down as Jython does. STM would just seem to add overhead that isn't necessary.
At the user level, STM + imperative(/uncontrolled effects) is basically a known failure. A lot of effort has been spent on it, with people with similar levels of control over their VM (like the C# attempt), and it just doesn't really work.
If you've got the control necessary to automatically STM things, you might as well just equally-automatically copy the Jython-style locking. STM is "nifty" but I don't see that it actually adds anything useful. Either that or you have to rigidly control effects in user code, and that's not Pythonic in any sense whatsoever (philosophical or practical).
- Locke1689 15y agoAt the user level, STM + imperative(/uncontrolled effects) is basically a known failure. A lot of effort has been spent on it, with people with similar levels of control over their VM (like the C# attempt) Agreed. The only project that I know of that's even close to a "working" implementation of STM is Haskell and that's because Haskell doesn't have uncontrolled stateful code.
- swannodette 15y agoClojure is designed for the user level and offers some unique advantages over Haskell's STM [1]: * MVCC snapshot avoiding transactions restarts on read invalidation. * Ensure references on read-writes provides a kind of manual control over resource acquisition order. * Has explicit commute which reduces retries on commutative writes. [1] http://stackoverflow.com/questions/4560605/how-does-clojure-stm-differ-from-haskell-stm http://stackoverflow.com/questions/4560605/how-does-clojure-... EDIT: I didn't read the OP closely enough. Yes grafting fine-grained STM onto a imperative language hasn't born much fruit.
- william42 15y agoIsn't Clojure still mostly functional and mostly based on persistent data structures?
- calebmpeterson 15y agoYes very much so...and then some.
- fijall 15y ago"If you've got the control necessary to automatically STM things, you might as well just equally-automatically copy the Jython-style locking." So far thinking about how to really do that automatically failed. Do you feel like providing an idea?
- jerf 15y agoFailed for CPython or failed for PyPy? I'm just going on the claim that PyPy is in a position to automatically add STM, per that blog post. If in fact it can't automatically add locks, it isn't going to be automatically adding STM either and this is all a moot discussion.
- jholman 15y agoAdding fine-grained locks without risking deadlock is pretty tricky. Perhaps too tricky for automatic addition. As I (shallowly) understand it, STM doesn't introduce deadlock threats, so it requires less-amazing feats to add it automatically. Note, again, that the FA does admit that using STM will perform more poorly than fine-grained locking, and he only hopes that it will scale well enough that extra cores will pay for it. As I understand it, one of the reasons for the GIL is that it makes it easier to write C extensions (especially as compared with fine-grained locking, which is not friendly to extensions), and I'm not sure, but I think the STM scheme preserves this property?
- tomp 15y ago> At the interpreter level, as the article mentions, it suffices to lock the structures down That's a sufficient, but not necessary condition. The idea of STM is to provide just the necessary protection, and nothing more, so that even multiple threads can manipulate the same structure simultaneously, as long as they don't manipulate the same properties/parts of this structure. Having said that, I don't claim that the way (P|J)ython is written, there is any unnecessary protection going on, furthermore, to fully ensure that just the necessary protection is on, a lot of code may have to be rewritten.