4 ms·
I love how so much person focus on the GIL and multithreading, when GIL is much more a solution to make un-threadsafe libs safe to use, and that most people don
by SFJulie 10y ago
I love how so much person focus on the GIL and multithreading, when GIL is much more a solution to make un-threadsafe libs safe to use, and that most people don't see POSIX threads are an inherently broken abstraction.
[1] http://www.daemonology.net/blog/2011-12-17-POSIX-close-is-broken.html http://www.daemonology.net/blog/2011-12-17-POSIX-close-is-br...
In fact it pretty much boils down to signals being broken on unices
[2] https://lwn.net/Articles/683118/ https://lwn.net/Articles/683118/
Which even though I have a hatred for systemd, systemd is trying to fix by leaving the status quo.
However, POSIX signals are still a problem to systemd
[3] https://github.com/systemd/systemd/issues/1615 https://github.com/systemd/systemd/issues/1615
Having played with signal in python+C, I have the experience of python having some holes around the signals: no mask can bet set. I thought initially python sucked because of the most common denominator problem of system languages (having to make you support only small subsets of features). But, I am now thinking POSIX signal are just a broken OS level software interrupt implementation.
So going down the rabbit hole, after reading Stevens on unix/POSIX programming (a must read). I am pretty much thinking questioning fred brooks (hence K&R&T) biggest failure: OS360 followed by unices.
What if our quest for a multitasking portable OS that does not care about the HW is doomed?
It makes a darn good job for 99.999% of the case. The .001% remaining being the signals.
Look at it, what is a process meant to be?
A container running code.
A thread? Cooperative code sharing data. But how do you cooperate? You send signals.
The problem, it is in case of high use of signals the OS get "signal bound" in a way we cannot measure.
signals are like a huge software bus that is not easily measurable and at the opposite of a lot of primitive cannot be HW bound. It is basically a software bus that tries to convey the concept of HW interrupts that are normally handled with micro chips. Look at the MC2828 brochure and you can recognize the feature signals are trying to provide
[4] https://upload.wikimedia.org/wikipedia/commons/3/31/Motorola_Microcomputer_Components_1978_pg10.jpg https://upload.wikimedia.org/wikipedia/commons/3/31/Motorola...
So to wrap up, we may have a problem of HW architecture that results in a buggy implementation of a common API. Like trying to emulate MMU on a MMU less CPU.
And I would say that it is thanks to my experiments in python that I discovered that signals was an unreliable 1bit message delivery protocol. Python made it easy to experiment.
Python has problems. (mostly a weired mix of conservatism and progress on concerns I don't share and politics).
But overall it is a good system language that deals with problem. And poor support of signals, threading are not a bug from python.
A good system language does not try to fix system glitches, he let them stay obvious. All the hate against GIL/threading/signals/weired async IO may be better directed at the quest for a portable multitasking generic OS.
Threads (and implicitly signals) on the other hand are convenient fantasies that we would like to exist but are actually just fantasies. And to solve the problem, we invented the containers... based .... on cooperative multitasking system ... based ... on threads and signals.
- falsedan 10y ago> I love how so much person focus on the GIL Do you really? I don't see much love in your comment.
- dkersten 10y agoThe GIL doesn't magically make un-thread safe code thread safe. It makes Pythons reference counting implementation thread safe.
- SFJulie 10y agoThe core dev having worked on new GIL (py3.2) explained this to me. I never said it was magic. But he said GIL is a tool to achieve thread-safety in python when calling non 'thread safe' code. I am not him, I will not take on any argument of how it works. But since ruby GIL is inspired by python GIL let's hear ruby coders: http://www.rubyinside.com/does-the-gil-make-your-ruby-code-thread-safe-6051.html http://www.rubyinside.com/does-the-gil-make-your-ruby-code-t... Oh, yes, it seems some people are seeing it my way and that it is a controversial point that can be argued. So I agree to disagree. Btw, I don't multithread and share states, I multiprocess with 0MQ and communication patterns such as PUB/SUB PUSH/PULL for the obvious reasons that I really think multi-threading is an over-valued and wrong abstraction.
- __david__ 10y ago> I really think multi-threading is an over-valued and wrong abstraction. That's as wrong-headed as thinking it's the only good abstraction. Each style has its place—it really depends on what the code needs to do.
- BuckRogers 10y agoI'd tend to agree with the parent that multi-pthreading is a poor abstraction but I'd add an important note: only systems programmers should be explicitly spinning up pthreads. Explicit pthreading is a poor abstraction for everyone else. Application programmers are probably operating at the 'wrong' level and greenthreads make more sense.