3 ms·
I've had fun debugging this very problem in the past, as the root cause of a very puzzling intermittent failure. The symptoms were intermittent segfaults in so
by ChrisFoster 10y ago
I've had fun debugging this very problem in the past, as the root cause of a very puzzling intermittent failure.
The symptoms were intermittent segfaults in some high level scientific python code running on a large cluster in AWS. Given we had no custom C library extensions, this should be impossible, but there they were; the jobs would fail randomly for one in every several thousand runs, if memory serves.
We eventually tracked this to a call to getenv() in the OpenBLAS thread pool per-thread initialization code. This seems innocuous enough by itself, but couple it with some other unrelated python library which happened to call setenv() and you have a nice race condition accessing the internal glibc data structures. Ultimately this ended up with a segfault due to dereferencing some already freed memory.
Ah, good times. Here's the root cause analysis if anyone wants the gory details :-)
https://github.com/xianyi/OpenBLAS/issues/716#issuecomment-164339663 https://github.com/xianyi/OpenBLAS/issues/716#issuecomment-1...