8 ms·

I am sure it has been said before, but it might not be a bad idea to try what Linux is doing. The BKL is used less and more granular locks are starting to take

by afhof 16y ago

I am sure it has been said before, but it might not be a bad idea to try what Linux is doing. The BKL is used less and more granular locks are starting to take its place.

pmjordan 16y ago
It's not quite so easy. The kernel is in a special position in that it can distinguish between and defend against two types of race condition: - races caused by different CPUs accessing the same resource. Unlike userspace, the kernel knows what each CPU is doing (roughly) at any given time. - races caused by preemption. Basically, this means the running thread/process is paused and the CPU is scheduled to run a different thread/process which then does something that interferes with whatever the original thread/process was in the middle of doing. First, the kernel can prevent the latter altogether by marking sections of code non-preemptible. What this means for races is that you can stop the CPU from being forced to context switching while accessing a contented resource, thus guaranteeing speedy progress out of the critical section. Secondly, there are spinlocks. This means that if a thread of execution tries to gain exclusive control over a contented resource, instead of relinquishing control over the CPU (thus rescheduling/context switching), it just sits there in a live loop waiting for the resource to be freed. On the surface, this seems like a bad idea: the CPU can't do anything while it's waiting for another CPU to do its stuff with the contended resource, and it's just wasting precious cycles. However, because the kernel is a known entity, it's possible to guarantee that all resource accesses will be extremely short because the critical section running on another CPU is non-preemptible. Generally, this technique is used when the critical sections are (much) shorter than the cost of a context switch. In userspace, the two types of races are indistinguishable, and you don't have separate weapons for fighting them. Spinlocks are a really bad idea in userspace unless you know your software is the only (major) user of CPU time on the system, and only runs in as many threads as there are CPUs. tl;dr: The kernel (a) has more information and (b) more control, so you can't apply those techniques to Python.
btilly 16y ago
That is not necessarily a good idea. It is quite a few years old, but http://www.bitmover.com/llnl/smp.pdf http://www.bitmover.com/llnl/smp.pdf is very informative of potential problems in the path that Linux is heading down.

by afhof 16y ago

I am sure it has been said before, but it might not be a bad idea to try what Linux is doing. The BKL is used less and more granular locks are starting to take its place.

pmjordan 16y ago

It's not quite so easy. The kernel is in a special position in that it can distinguish between and defend against two types of race condition: - races caused by different CPUs accessing the same resource. Unlike userspace, the kernel knows what each CPU is doing (roughly) at any given time. - races caused by preemption. Basically, this means the running thread/process is paused and the CPU is scheduled to run a different thread/process which then does something that interferes with whatever the original thread/process was in the middle of doing. First, the kernel can prevent the latter altogether by marking sections of code non-preemptible. What this means for races is that you can stop the CPU from being forced to context switching while accessing a contented resource, thus guaranteeing speedy progress out of the critical section. Secondly, there are spinlocks. This means that if a thread of execution tries to gain exclusive control over a contented resource, instead of relinquishing control over the CPU (thus rescheduling/context switching), it just sits there in a live loop waiting for the resource to be freed. On the surface, this seems like a bad idea: the CPU can't do anything while it's waiting for another CPU to do its stuff with the contended resource, and it's just wasting precious cycles. However, because the kernel is a known entity, it's possible to guarantee that all resource accesses will be extremely short because the critical section running on another CPU is non-preemptible. Generally, this technique is used when the critical sections are (much) shorter than the cost of a context switch. In userspace, the two types of races are indistinguishable, and you don't have separate weapons for fighting them. Spinlocks are a really bad idea in userspace unless you know your software is the only (major) user of CPU time on the system, and only runs in as many threads as there are CPUs. tl;dr: The kernel (a) has more information and (b) more control, so you can't apply those techniques to Python.

btilly 16y ago

That is not necessarily a good idea. It is quite a few years old, but http://www.bitmover.com/llnl/smp.pdf http://www.bitmover.com/llnl/smp.pdf is very informative of potential problems in the path that Linux is heading down.