5 ms·

Tearing apart printf() (2018)

152 points by dreampeppers99 7y ago

matheusmoreira 7y ago
The registers rdi, rsi and rdx are acknowledged in the article but they don't appear in the disassembly: 000000000040f9c0 <__libc_write>: 40f9c0: 83 3d c5 bb 2a 00 00 cmpl $0x0,0x2abbc5(%rip) # 6bb58c <__libc_multiple_threads> 40f9c7: 75 14 jne 40f9dd <__write_nocancel+0x14> 000000000040f9c9 <__write_nocancel>: 40f9c9: b8 01 00 00 00 mov $0x1,%eax 40f9ce: 0f 05 syscall This is because they are set when the write function is called. The System V AMD64 ABI specifies that rdi, rsi and rdx are used for the 1st, 2nd and 3rd arguments of a function, perfectly matching the Linux system call ABI. The 5th and 6th also match: r8 and r9, respectively. However, the 4th argument doesn't match: system calls use r10 while System V uses rcx. I wish I knew why.
- gamozolabs 7y ago
  The x86 `syscall` instruction stores the return address into `rcx` as well as `RFLAGS` into `r11`. These are unconditionally clobbered and thus cannot be saved or used in a `syscall` transition.
- nitwit005 7y ago
  Isn't that disassembly a replacement of the call to printf with a call to the write system call? Most compilers have the printf family as builtins to allow removing them from the code entirely: https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
  - saagarjha 7y ago
    Compilers will rarely inline all of printf in my experience.
emmelaich 7y ago
Excellent stuff. My couple of notes.. 1. printf may malloc so don't use it in an out of memory situation. Though I think this problem is vanishingly unlikely these days. 2. printf of floats used to require linking to the math library on some platforms (-lm) 3. I am pathetically grateful for nice diagnostics you get when you use the wrong % formatter in more recent C compilers. This used to be a rich source of errors and non-portability.
- lelf 7y ago
  4. On GNU/Linux you can add your own %-conversions (e. g. %Y) with register_printf_function.
  - saagarjha 7y ago
    register_printf_function is deprecated; you should use register_printf_specifier instead.
    - eqvinox 7y ago
      I strongly advise against using either since both are not only non-portable, but also break any compiler support for getting you warnings when you're using it wrong. The Linux kernel style suffix extensions (cf. https://www.kernel.org/doc/Documentation/printk-formats.txt https://www.kernel.org/doc/Documentation/printk-formats.txt) are much better on the latter front.
- s_m_t 7y ago
  >1. printf may malloc so don't use it in an out of memory situation. Though I think this problem is vanishingly unlikely these days. I feel like you've just saved my future self from a terrible bug in the future. Thanks
- brandmeyer 7y ago
  You can also use __attribute__((format(printf, ...))) for functions of yours that ultimately pass their varargs to a member of the printf family. It is exceedingly rare for malloc() to give you a recovery chance on failure. Folks are so used to the Linux model of overcommit+OOM killer that almost nobody's code can gracefully deal with memory exhaustion.
mwfunk 7y ago
“Linux/GNU” is a new one. Not sure if it’s a typo or a troll, but if the latter, it’s a pretty decent troll if such a thing exists? If it’s a typo, geez dude. You’re going to give RMS an aneurysm.
- 77pt77 7y ago
  > You’re going to give RMS an aneurysm. He has much more serious problems nowadays...
  - kevin_thibedeau 7y ago
    Surely he has Emacs configured to rewrite all references to the one true name.
    - HeWhoLurksLate 7y ago
      VIM?
porknubbins 7y ago
I wish there was a book length guide with this kind of tutorial style writing but haven’t found much. Everything I’ve found on Linux is more like a reference resource
dang 7y ago
Discussed at the time: https://news.ycombinator.com/item?id=17114919 https://news.ycombinator.com/item?id=17114919
Annatar 7y ago
This is a good overview of how it all works. On some systems like AmigaOS, there is no libc at all, for example standard input / output is handled by the dos.library in ROM. compiler-specific implementation is delivered as libc.a, which means that only static linking with libc is possible. UNIX®️ software cannot be compiled at all without modification and without having the 3rd party ixemul.library downloaded from "AmiNet" and installed in the LIBS: assign (which usually resolves to SYS:LIBS, which in turn usually resolves to either DF0:LIBS or DH0:LIBS).
eqvinox 7y ago
As far as the formatting part is concerned, if you just want to look at an implementation, FreeBSD's is extremely nice and readable: https://github.com/lattera/freebsd/blob/master/lib/libc/stdio/vfprintf.c https://github.com/lattera/freebsd/blob/master/lib/libc/stdi... For FRRouting, we decided we want to use Linux kernel style extensions (like "%pI4"), so to get this in a portable way we imported FreeBSD's printf into our code. Since printf() isn't exactly "hot" code getting changed a lot, we considered the maintenance / duplication cost acceptable. You can see the result here: https://github.com/FRRouting/frr/tree/master/lib/printf https://github.com/FRRouting/frr/tree/master/lib/printf http://docs.frrouting.org/projects/dev-guide/en/latest/logging.html http://docs.frrouting.org/projects/dev-guide/en/latest/loggi... stdio support is completely gone in our copy, WCHAR_SUPPORT is disabled at compile time, locale support is stubbed out/hardcoded to C locale. None of these matter to us. As a result, we can use this printf even in the SEGV handler. (That's only a bonus though, the main reason was extensibility on the format specifiers.)

The registers rdi, rsi and rdx are acknowledged in the article but they don't appear in the disassembly: 000000000040f9c0 <__libc_write>: 40f9c0: 83 3d c5 bb 2a 00 00 cmpl $0x0,0x2abbc5(%rip) # 6bb58c <__libc_multiple_threads> 40f9c7: 75 14 jne 40f9dd <__write_nocancel+0x14> 000000000040f9c9 <__write_nocancel>: 40f9c9: b8 01 00 00 00 mov $0x1,%eax 40f9ce: 0f 05 syscall This is because they are set when the write function is called. The System V AMD64 ABI specifies that rdi, rsi and rdx are used for the 1st, 2nd and 3rd arguments of a function, perfectly matching the Linux system call ABI. The 5th and 6th also match: r8 and r9, respectively. However, the 4th argument doesn't match: system calls use r10 while System V uses rcx. I wish I knew why.

gamozolabs 7y ago
The x86 `syscall` instruction stores the return address into `rcx` as well as `RFLAGS` into `r11`. These are unconditionally clobbered and thus cannot be saved or used in a `syscall` transition.
nitwit005 7y ago
Isn't that disassembly a replacement of the call to printf with a call to the write system call? Most compilers have the printf family as builtins to allow removing them from the code entirely: https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
- saagarjha 7y ago
  Compilers will rarely inline all of printf in my experience.

emmelaich 7y ago

Excellent stuff. My couple of notes.. 1. printf may malloc so don't use it in an out of memory situation. Though I think this problem is vanishingly unlikely these days. 2. printf of floats used to require linking to the math library on some platforms (-lm) 3. I am pathetically grateful for nice diagnostics you get when you use the wrong % formatter in more recent C compilers. This used to be a rich source of errors and non-portability.

lelf 7y ago
4. On GNU/Linux you can add your own %-conversions (e. g. %Y) with register_printf_function.
- saagarjha 7y ago
  register_printf_function is deprecated; you should use register_printf_specifier instead.
  - eqvinox 7y ago
    I strongly advise against using either since both are not only non-portable, but also break any compiler support for getting you warnings when you're using it wrong. The Linux kernel style suffix extensions (cf. https://www.kernel.org/doc/Documentation/printk-formats.txt https://www.kernel.org/doc/Documentation/printk-formats.txt) are much better on the latter front.
s_m_t 7y ago
>1. printf may malloc so don't use it in an out of memory situation. Though I think this problem is vanishingly unlikely these days. I feel like you've just saved my future self from a terrible bug in the future. Thanks
brandmeyer 7y ago
You can also use __attribute__((format(printf, ...))) for functions of yours that ultimately pass their varargs to a member of the printf family. It is exceedingly rare for malloc() to give you a recovery chance on failure. Folks are so used to the Linux model of overcommit+OOM killer that almost nobody's code can gracefully deal with memory exhaustion.

mwfunk 7y ago

“Linux/GNU” is a new one. Not sure if it’s a typo or a troll, but if the latter, it’s a pretty decent troll if such a thing exists? If it’s a typo, geez dude. You’re going to give RMS an aneurysm.

77pt77 7y ago
> You’re going to give RMS an aneurysm. He has much more serious problems nowadays...
- kevin_thibedeau 7y ago
  Surely he has Emacs configured to rewrite all references to the one true name.
  - HeWhoLurksLate 7y ago
    VIM?

porknubbins 7y ago

I wish there was a book length guide with this kind of tutorial style writing but haven’t found much. Everything I’ve found on Linux is more like a reference resource

dang 7y ago

Discussed at the time: https://news.ycombinator.com/item?id=17114919 https://news.ycombinator.com/item?id=17114919

Annatar 7y ago

This is a good overview of how it all works. On some systems like AmigaOS, there is no libc at all, for example standard input / output is handled by the dos.library in ROM. compiler-specific implementation is delivered as libc.a, which means that only static linking with libc is possible. UNIX®️ software cannot be compiled at all without modification and without having the 3rd party ixemul.library downloaded from "AmiNet" and installed in the LIBS: assign (which usually resolves to SYS:LIBS, which in turn usually resolves to either DF0:LIBS or DH0:LIBS).

eqvinox 7y ago

As far as the formatting part is concerned, if you just want to look at an implementation, FreeBSD's is extremely nice and readable: https://github.com/lattera/freebsd/blob/master/lib/libc/stdio/vfprintf.c https://github.com/lattera/freebsd/blob/master/lib/libc/stdi... For FRRouting, we decided we want to use Linux kernel style extensions (like "%pI4"), so to get this in a portable way we imported FreeBSD's printf into our code. Since printf() isn't exactly "hot" code getting changed a lot, we considered the maintenance / duplication cost acceptable. You can see the result here: https://github.com/FRRouting/frr/tree/master/lib/printf https://github.com/FRRouting/frr/tree/master/lib/printf http://docs.frrouting.org/projects/dev-guide/en/latest/logging.html http://docs.frrouting.org/projects/dev-guide/en/latest/loggi... stdio support is completely gone in our copy, WCHAR_SUPPORT is disabled at compile time, locale support is stubbed out/hardcoded to C locale. None of these matter to us. As a result, we can use this printf even in the SEGV handler. (That's only a bonus though, the main reason was extensibility on the format specifiers.)