4 ms·

I am going to start positing this into every discussion about this topic now: can we as an industry start not omitting frame pointers? It’s not so significant (

I am going to start positing this into every discussion about this topic now: can we as an industry start not omitting frame pointers? It’s not so significant (maybe puts us back three months in hardware advances) but everybody greatly benefits from that.

brancz 4y ago
Could not agree more! Unfortunately, I don't think we'll realistically be able to convince everyone, and even if we did convince everyone to have frame pointers tomorrow, it would still take years to take effect. So for the foreseeable future we'll still need things like this, even if I wish we didn't. That said, we should still try to convince people!
javierhonduco 4y ago
100% this. Disabling frame pointers is not worth it in my opinion. Hindering debugging and profiling is a massive regression. It's one of these historical (due to x86 having fewer registers) and short-term thinking changes that have unfortunately stayed with us. I'm optimistic that this can change but it will take a while. BTW, I'm a big fan of your work, thanks for Flask and Jinja!
davemarchevsky 4y ago
There is currently a Fedora proposal to add -fno-omit-frame-pointer to default compilation flags [0](mailing list discussion: [1]). [0]: https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer [1]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/OOJDAKTJB5WGMOZRXTUX7FTPFBF3H7WE/ edit: whoops, this was already mentioned in the blog post. Should read before commenting :)
- nemetroid 4y ago
  By now (but not at the time of posting), a rejected Fedora proposal.
simscitizen 4y ago
Frame pointers are enabled by default on macOS and iOS and I can't understand why they aren't enabled by default elsewhere. It just makes it so much easier to debug and profile things. See for instance Apple's ARM64 calling convention: https://developer.apple.com/documentation/xcode/writing-arm64-code-for-apple-platforms https://developer.apple.com/documentation/xcode/writing-arm6... "The frame pointer register (x29) must always address a valid frame record. Some functions — such as leaf functions or tail calls — may opt not to create an entry in this list. As a result, stack traces are always meaningful, even without debug information."
- saagarjha 4y ago
  Apple relies on this heavily when doing systemwide performance profiling. It's good practice!
fweimer 4y ago
They are part of the the psABI on aarch64 and POWER. Due to to the link register, they can even omit the frame pointer from leaf functions without losing the identity of the immediate caller. Generally, the “compile with frame pointers” request is a bit underspecified on x86-64. For example, the perf trampoline that is cited as the reason why future Python versions are going to use frame pointers itself does not have a frame pointer. (It does not clobber it, either, but backtraces through it will skip a frame.) Similarly, if you just compile glibc with frame pointers without patching a handful of assembler files, then for typical workloads, 5% to 10% of your samples will have an incomplete backtrace because the immediate caller of glibc string functions is not recorded (they don't have a frame pointer, either). I expect this x86-64 debate will be obsolete Really Soon Now because everyone will simply copy out the shadow stack on every sample. It's even faster than frame pointer traversal because it's just an array copy. We are still looking at making DWARF backtraces a bit faster (or even the more complex unwinding case), but I doubt that this work will be impactful, given the politics involved. The shadow stack will have some performance impact, too, but it can be switched off on a per-process basis if necessary.

brancz 4y ago

Could not agree more! Unfortunately, I don't think we'll realistically be able to convince everyone, and even if we did convince everyone to have frame pointers tomorrow, it would still take years to take effect. So for the foreseeable future we'll still need things like this, even if I wish we didn't. That said, we should still try to convince people!

javierhonduco 4y ago

100% this. Disabling frame pointers is not worth it in my opinion. Hindering debugging and profiling is a massive regression. It's one of these historical (due to x86 having fewer registers) and short-term thinking changes that have unfortunately stayed with us. I'm optimistic that this can change but it will take a while. BTW, I'm a big fan of your work, thanks for Flask and Jinja!

davemarchevsky 4y ago

There is currently a Fedora proposal to add -fno-omit-frame-pointer to default compilation flags [0](mailing list discussion: [1]). [0]: https://fedoraproject.org/wiki/Changes/fno-omit-frame-pointer [1]: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org/thread/OOJDAKTJB5WGMOZRXTUX7FTPFBF3H7WE/ edit: whoops, this was already mentioned in the blog post. Should read before commenting :)

nemetroid 4y ago
By now (but not at the time of posting), a rejected Fedora proposal.

simscitizen 4y ago

Frame pointers are enabled by default on macOS and iOS and I can't understand why they aren't enabled by default elsewhere. It just makes it so much easier to debug and profile things. See for instance Apple's ARM64 calling convention: https://developer.apple.com/documentation/xcode/writing-arm64-code-for-apple-platforms https://developer.apple.com/documentation/xcode/writing-arm6... "The frame pointer register (x29) must always address a valid frame record. Some functions — such as leaf functions or tail calls — may opt not to create an entry in this list. As a result, stack traces are always meaningful, even without debug information."

saagarjha 4y ago
Apple relies on this heavily when doing systemwide performance profiling. It's good practice!

fweimer 4y ago

They are part of the the psABI on aarch64 and POWER. Due to to the link register, they can even omit the frame pointer from leaf functions without losing the identity of the immediate caller. Generally, the “compile with frame pointers” request is a bit underspecified on x86-64. For example, the perf trampoline that is cited as the reason why future Python versions are going to use frame pointers itself does not have a frame pointer. (It does not clobber it, either, but backtraces through it will skip a frame.) Similarly, if you just compile glibc with frame pointers without patching a handful of assembler files, then for typical workloads, 5% to 10% of your samples will have an incomplete backtrace because the immediate caller of glibc string functions is not recorded (they don't have a frame pointer, either). I expect this x86-64 debate will be obsolete Really Soon Now because everyone will simply copy out the shadow stack on every sample. It's even faster than frame pointer traversal because it's just an array copy. We are still looking at making DWARF backtraces a bit faster (or even the more complex unwinding case), but I doubt that this work will be impactful, given the politics involved. The shadow stack will have some performance impact, too, but it can be switched off on a per-process basis if necessary.