5 ms·

Dynamic linking is the root of all kinds of evil, enough said.

woodruffw 4y ago
The content of this post has nothing to do with the specifics of dynamic linking: it would be just as true if the wheels in question had static binaries instead.
- benreesman 4y ago
  Eh, somewhere in the middle. Someone else put ‘-ffast-math’ in a compile line and it poisons FP math far away with no recompile? I believe it’s a necessary price in this case, but it does highlight how suboptimal it is to pay the price in other cases.
  - woodruffw 4y ago
    It's fair to point out that shared objects surface the problem here, but I don't know if I would lay the blame with them: the underlying problem is that a FPU control register isn't (and can't be) meaningfully isolated. Python needs to use shared objects for loadable extensions, but the contaminating code might be statically linked into that shared object. (I don't say this because I want to excuse dynamic linking, which I also generally dislike! Only that I think the problem is somewhere else in this particular case.)
jeroenhd 4y ago
What is the alternative here? To provide a python.so file with all possible binary Python packages statically linked into it? You'd need to update it every hour to include all the bugfixes in every native library yanked in! To recompile Python itself every time you install a package? Even with a compiler cache you'd have the Gentoo experience of waiting for ages every time you try to use the package manager. Dynamic linking solves a real problem, especially in this space. It comes with new problems of its own but so does the alternative.
- deleted 4y ago
  [deleted]
- deleted 4y ago
  [deleted]
benreesman 4y ago
As a default (particularly an effectively mandatory default, looking at you glibc) it is indeed insane. But for something like a Python extension it’s what we’ve got. Which has the ancillary benefit of surfacing stuff like this.

The content of this post has nothing to do with the specifics of dynamic linking: it would be just as true if the wheels in question had static binaries instead.

benreesman 4y ago
Eh, somewhere in the middle. Someone else put ‘-ffast-math’ in a compile line and it poisons FP math far away with no recompile? I believe it’s a necessary price in this case, but it does highlight how suboptimal it is to pay the price in other cases.
- woodruffw 4y ago
  It's fair to point out that shared objects surface the problem here, but I don't know if I would lay the blame with them: the underlying problem is that a FPU control register isn't (and can't be) meaningfully isolated. Python needs to use shared objects for loadable extensions, but the contaminating code might be statically linked into that shared object. (I don't say this because I want to excuse dynamic linking, which I also generally dislike! Only that I think the problem is somewhere else in this particular case.)

jeroenhd 4y ago

What is the alternative here? To provide a python.so file with all possible binary Python packages statically linked into it? You'd need to update it every hour to include all the bugfixes in every native library yanked in! To recompile Python itself every time you install a package? Even with a compiler cache you'd have the Gentoo experience of waiting for ages every time you try to use the package manager. Dynamic linking solves a real problem, especially in this space. It comes with new problems of its own but so does the alternative.

deleted 4y ago
[deleted]
deleted 4y ago
[deleted]

benreesman 4y ago

As a default (particularly an effectively mandatory default, looking at you glibc) it is indeed insane. But for something like a Python extension it’s what we’ve got. Which has the ancillary benefit of surfacing stuff like this.