4 ms·

Frame pointers are such a destructive micro-optimization to omit by default, I am beyond excited about this collaboration with the folks at Canonical to make Ub

by brancz 3y ago

Frame pointers are such a destructive micro-optimization to omit by default, I am beyond excited about this collaboration with the folks at Canonical to make Ubuntu debuggable by default!

account42 3y ago
It's a "micro" optimization that can be automatically applied everywhere. And its not destructive because debug info contains all you need to calculate the frame pointers after the fact. Really a no-brainer to have it on unless you are dealing with broken tools.
- brancz 3y ago
  We've "fixed"[1] all the tooling that we work on at Polar Signals, and even then I think this is the right thing to do. It enables so much more debuggability aside from profiling. [1] https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-based-stack-walking-using-ebpf https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-bas...
duskwuff 3y ago
This option was almost certainly a holdover from the bad old 32-bit x86 days, when disabling frame pointers gave you a seventh valuable general-purpose register. It's no longer beneficial on x86_64 -- even with rbp locked down, you still have fourteen registers there.
- brancz 3y ago
  100% the way I see it! On 32-bit the performance benefits is very major on just about everything, but not so on 64-bit.
- PaulDavisThe1st 3y ago
  and 14 registers should be enough for everyone!
  - mratsim 3y ago
    It's not. 14 is still low compared to other ISAs.
    - peterfirefly 3y ago
      Spilling to an XMM register instead of memory is an option.
- jandrese 3y ago
  Yeah, back in the 32 bit days -fomit-frame-pointer was the only optimization you could count on to really make a difference. It wasn't small either, often 10-15% speedup. No other flag on gcc would make even a full percentage difference in my testing. The Amd64 architecture fixed the underlying problem, so this is pretty much just a holdover. I'm surprised they even enabled it by default.
  - grrandalf 3y ago
    (iirc) I think I used `-fomit-frame-pointer` with DJGPP on DOS on a 486. It was an [unimpressive :)] software-rendered 3D graphics demo, and I was very happy with the substantial free speedup I got.
- cesarb 3y ago
  To emphasize this point: on 64-bit x86 with frame pointers, you have twice as many registers as on 32-bit x86 without frame pointers, and these registers are twice as wide. A 64-bit value (more common than you'd expect even when pointers are 32 bits) takes two registers on 32-bit x86, but only a single register on 64-bit x86.
  - jamesfinlayson 3y ago
    So there's no point in disabling frame pointers for 32-bit code running on a 64-bit processor then?
- bodyfour 3y ago
  That much is true -- the difference between 6 and 7 registers is much larger than the benefit of going from 14 to 15. However, even under zero register pressure having a frame pointer is still an extra register that needs to be touched on every function invocation, extra instructions taking space in the I-cache, etc. It's a small thing, but it's still a cost that has to be paid by all compiled code. I'm not going to claim that re-enabling frame pointers was the wrong choice -- the people involved in the debate know the tradeoffs and I have to start with the assumption that I would have made the same decision if I were in their position. It does make me slightly sad, though. The idea behind removing frame pointers isn't that backtraces aren't important, it's that computing the frame pointer after-the-fact is possible -- i.e. for normal functions without alloca() or dynamically-sized stack arrays map %rip -> frame size. The problem seems to be that despite years of experience with "no-frame-pointer" being the default I guess the profiling tools never got as reliable or good as the with-frame-pointer variants. My personal hope was that the problem would fade over time as tools improved, but it seems that's unlikely to ever happen. After all, once no-frame-pointer stops being the default there won't be any pressure for tools to improve. The towel has been thrown in.
  - brancz 3y ago
    Profiling tools have already solved the ability to reliably unwind in the absence of frame pointers[1], but there are plenty of tools that this kind of investment is simply too much that it won't ever happen, like bpftrace or bcc-tools. [1] https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-based-stack-walking-using-ebpf https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-bas...
    - account42 3y ago
      Do those tools really need to implement their own unwinding though? This really should just need to be implemented once in a library and then used wherever unwinding is needed.
      - brancz 3y ago
        With the combination of eBPF and DWARF-based it's not quite that simple, there's a pretty elaborate dance between user-space and kernel-space that needs to happen for this to work. With frame-pointers it's walking a linked list in kernel-space.
- kmeisthax 3y ago
  Also, apparently Intel is planning to extend x86_64 to 32 GPRs, with an extension called... sigh[0]... Intel APX[1]. So the overhead of frame pointers will be even lower in the future. [0] Intel APX is extremely confusable with the iAPX 432, a failed non-x86 architecture Intel made that's completely unrelated to doubling the size of the x64 register file. [1] https://www.intel.com/content/www/us/en/developer/articles/technical/advanced-performance-extensions-apx.html https://www.intel.com/content/www/us/en/developer/articles/t...
  - p1mrx 3y ago
    iAPX has been dead for almost 40 years; there probably won't be much confusion.
    - hulitu 3y ago
      It's not dead. It's sleeping.
      - p1mrx 3y ago
        Intel processors prior to the 80386SL cannot sleep.
karmakaze 3y ago
> Ubuntu leaps forward with frame pointers by default Yeah it's more like "...will stop omitting frame pointers by default".
amelius 3y ago
> to make Ubuntu debuggable by default! How's that, you'd still need the debug symbols Also has anyone else noticed that running stuff through Valgrind is really only possible if the program was made with Valgrind in mind? For example, Python and its many extensions generate numerous errors and warnings, so many that any real problem becomes hidden. I'd say that modern Linux systems are very far from being debuggable.
- rightbyte 3y ago
  You offent need to make excemption file. My tip is to run some thing one program execution, and then run it twice during the same program execution, and diff the reports.
- brancz 3y ago
  Ubuntu hosts a debuginfod server that you can automatically discover debuginfo from any binary from using the binary’s build id. https://sourceware.org/elfutils/Debuginfod.html https://sourceware.org/elfutils/Debuginfod.html

by brancz 3y ago

Frame pointers are such a destructive micro-optimization to omit by default, I am beyond excited about this collaboration with the folks at Canonical to make Ubuntu debuggable by default!

account42 3y ago

It's a "micro" optimization that can be automatically applied everywhere. And its not destructive because debug info contains all you need to calculate the frame pointers after the fact. Really a no-brainer to have it on unless you are dealing with broken tools.

brancz 3y ago
We've "fixed"[1] all the tooling that we work on at Polar Signals, and even then I think this is the right thing to do. It enables so much more debuggability aside from profiling. [1] https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-based-stack-walking-using-ebpf https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-bas...

duskwuff 3y ago

This option was almost certainly a holdover from the bad old 32-bit x86 days, when disabling frame pointers gave you a seventh valuable general-purpose register. It's no longer beneficial on x86_64 -- even with rbp locked down, you still have fourteen registers there.

brancz 3y ago
100% the way I see it! On 32-bit the performance benefits is very major on just about everything, but not so on 64-bit.
PaulDavisThe1st 3y ago
and 14 registers should be enough for everyone!
- mratsim 3y ago
  It's not. 14 is still low compared to other ISAs.
  - peterfirefly 3y ago
    Spilling to an XMM register instead of memory is an option.
jandrese 3y ago
Yeah, back in the 32 bit days -fomit-frame-pointer was the only optimization you could count on to really make a difference. It wasn't small either, often 10-15% speedup. No other flag on gcc would make even a full percentage difference in my testing. The Amd64 architecture fixed the underlying problem, so this is pretty much just a holdover. I'm surprised they even enabled it by default.
- grrandalf 3y ago
  (iirc) I think I used `-fomit-frame-pointer` with DJGPP on DOS on a 486. It was an [unimpressive :)] software-rendered 3D graphics demo, and I was very happy with the substantial free speedup I got.
cesarb 3y ago
To emphasize this point: on 64-bit x86 with frame pointers, you have twice as many registers as on 32-bit x86 without frame pointers, and these registers are twice as wide. A 64-bit value (more common than you'd expect even when pointers are 32 bits) takes two registers on 32-bit x86, but only a single register on 64-bit x86.
- jamesfinlayson 3y ago
  So there's no point in disabling frame pointers for 32-bit code running on a 64-bit processor then?
bodyfour 3y ago
That much is true -- the difference between 6 and 7 registers is much larger than the benefit of going from 14 to 15. However, even under zero register pressure having a frame pointer is still an extra register that needs to be touched on every function invocation, extra instructions taking space in the I-cache, etc. It's a small thing, but it's still a cost that has to be paid by all compiled code. I'm not going to claim that re-enabling frame pointers was the wrong choice -- the people involved in the debate know the tradeoffs and I have to start with the assumption that I would have made the same decision if I were in their position. It does make me slightly sad, though. The idea behind removing frame pointers isn't that backtraces aren't important, it's that computing the frame pointer after-the-fact is possible -- i.e. for normal functions without alloca() or dynamically-sized stack arrays map %rip -> frame size. The problem seems to be that despite years of experience with "no-frame-pointer" being the default I guess the profiling tools never got as reliable or good as the with-frame-pointer variants. My personal hope was that the problem would fade over time as tools improved, but it seems that's unlikely to ever happen. After all, once no-frame-pointer stops being the default there won't be any pressure for tools to improve. The towel has been thrown in.
- brancz 3y ago
  Profiling tools have already solved the ability to reliably unwind in the absence of frame pointers[1], but there are plenty of tools that this kind of investment is simply too much that it won't ever happen, like bpftrace or bcc-tools. [1] https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-based-stack-walking-using-ebpf https://www.polarsignals.com/blog/posts/2022/11/29/dwarf-bas...
  - account42 3y ago
    Do those tools really need to implement their own unwinding though? This really should just need to be implemented once in a library and then used wherever unwinding is needed.
    - brancz 3y ago
      With the combination of eBPF and DWARF-based it's not quite that simple, there's a pretty elaborate dance between user-space and kernel-space that needs to happen for this to work. With frame-pointers it's walking a linked list in kernel-space.
kmeisthax 3y ago
Also, apparently Intel is planning to extend x86_64 to 32 GPRs, with an extension called... sigh[0]... Intel APX[1]. So the overhead of frame pointers will be even lower in the future. [0] Intel APX is extremely confusable with the iAPX 432, a failed non-x86 architecture Intel made that's completely unrelated to doubling the size of the x64 register file. [1] https://www.intel.com/content/www/us/en/developer/articles/technical/advanced-performance-extensions-apx.html https://www.intel.com/content/www/us/en/developer/articles/t...
- p1mrx 3y ago
  iAPX has been dead for almost 40 years; there probably won't be much confusion.
  - hulitu 3y ago
    It's not dead. It's sleeping.
    - p1mrx 3y ago
      Intel processors prior to the 80386SL cannot sleep.

karmakaze 3y ago

> Ubuntu leaps forward with frame pointers by default Yeah it's more like "...will stop omitting frame pointers by default".

amelius 3y ago

> to make Ubuntu debuggable by default! How's that, you'd still need the debug symbols Also has anyone else noticed that running stuff through Valgrind is really only possible if the program was made with Valgrind in mind? For example, Python and its many extensions generate numerous errors and warnings, so many that any real problem becomes hidden. I'd say that modern Linux systems are very far from being debuggable.

rightbyte 3y ago
You offent need to make excemption file. My tip is to run some thing one program execution, and then run it twice during the same program execution, and diff the reports.
brancz 3y ago
Ubuntu hosts a debuginfod server that you can automatically discover debuginfo from any binary from using the binary’s build id. https://sourceware.org/elfutils/Debuginfod.html https://sourceware.org/elfutils/Debuginfod.html