4 ms·

Fedora 41 Looks to "-O3" Optimizations for Its Python Build

16 points by LinuxBender 3y ago

bravetraveler 3y ago
Phoronix, if you're going to prompt me for support... at least remember that I have. I don't know what's up with your cookies or whatever, but I don't even get spared by having supported already. Login lost, fine whatever. If I have to see the modal anyway I'd rather have my money. I don't regret supporting, just saying - had I known... Anyway, on topic: I hope this doesn't mess up data/math like it tends to (IIRC?) I think the compiler options the distribution uses now are well-reasoned. Except frame pointers, I'm hesitant to paint with big brushes like that. We were told that academically it makes no difference, yet it did. Those who want to profile could do so with their own builds/tooling. Rebuilding with 'mock' and 'fedpkg' is incredibly easy for those so inclined. Customizing a build of any package is legitimately no more than ~three repeatable commands: fedpkg clone -a -b f$RELEASE $PKG # ... spec patching/whatever fedpkg mockbuild I don't know why everyone had to participate, but we are - and it's been mostly fine. I don't think we'll be so lucky with '-03'
nequo 3y ago
This makes me wonder if recompiling Python with -march=native -mtune=native would speed it up by any noticeable amount.
- lumpa 3y ago
  There are performance options[0] that enable PGO, LTO and bolt, and are known to speed things up, maybe more significantly than -march and -mtune. But I found one report[1] that says these compiler flags do (or did, some time ago) make a difference. [0] https://docs.python.org/3/using/configure.html#performance-options https://docs.python.org/3/using/configure.html#performance-o... [1] https://atleastfornow.net/posts/py3-enable-optimisations/ https://atleastfornow.net/posts/py3-enable-optimisations/
rwmj 3y ago
The thread with the actual discussion: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/D7Y6GYNL62LCER6GPOA2VQ3DBX3WOYQ6/ https://lists.fedoraproject.org/archives/list/devel@lists.fe...
deleted 3y ago
[deleted]
rst 3y ago
Mildly surprised they weren't doing this already -- the thread doesn't mention whether they're doing the build with Profile Guided Optimization, which is widely reported to yield speedups of 10-20%, depending on the script, with an additional boost from link-time optimization where available. (--enable-optimizations --with-lto in configure, which does seem to be the current recommendation from upstream: https://docs.python.org/3/using/configure.html https://docs.python.org/3/using/configure.html)

16 points by LinuxBender 3y ago

Phoronix, if you're going to prompt me for support... at least remember that I have. I don't know what's up with your cookies or whatever, but I don't even get spared by having supported already. Login lost, fine whatever. If I have to see the modal anyway I'd rather have my money. I don't regret supporting, just saying - had I known... Anyway, on topic: I hope this doesn't mess up data/math like it tends to (IIRC?) I think the compiler options the distribution uses now are well-reasoned. Except frame pointers, I'm hesitant to paint with big brushes like that. We were told that academically it makes no difference, yet it did. Those who want to profile could do so with their own builds/tooling. Rebuilding with 'mock' and 'fedpkg' is incredibly easy for those so inclined. Customizing a build of any package is legitimately no more than ~three repeatable commands: fedpkg clone -a -b f$RELEASE $PKG # ... spec patching/whatever fedpkg mockbuild I don't know why everyone had to participate, but we are - and it's been mostly fine. I don't think we'll be so lucky with '-03'

nequo 3y ago

This makes me wonder if recompiling Python with -march=native -mtune=native would speed it up by any noticeable amount.

lumpa 3y ago
There are performance options[0] that enable PGO, LTO and bolt, and are known to speed things up, maybe more significantly than -march and -mtune. But I found one report[1] that says these compiler flags do (or did, some time ago) make a difference. [0] https://docs.python.org/3/using/configure.html#performance-options https://docs.python.org/3/using/configure.html#performance-o... [1] https://atleastfornow.net/posts/py3-enable-optimisations/ https://atleastfornow.net/posts/py3-enable-optimisations/

rwmj 3y ago

The thread with the actual discussion: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/D7Y6GYNL62LCER6GPOA2VQ3DBX3WOYQ6/ https://lists.fedoraproject.org/archives/list/devel@lists.fe...

deleted 3y ago

[deleted]

rst 3y ago

Mildly surprised they weren't doing this already -- the thread doesn't mention whether they're doing the build with Profile Guided Optimization, which is widely reported to yield speedups of 10-20%, depending on the script, with an additional boost from link-time optimization where available. (--enable-optimizations --with-lto in configure, which does seem to be the current recommendation from upstream: https://docs.python.org/3/using/configure.html https://docs.python.org/3/using/configure.html)