3 ms·

However, who should really rely on internals of FILE? Isn't this a bad practice?

vitaut 1y ago
In general, it is a bad practice. However, it can be useful for some low-level libraries. For example, https://github.com/fmtlib/fmt https://github.com/fmtlib/fmt provides a type-safe replacement for `printf` that can write directly to the FILE buffer providing comparable or better performance to native stdio.
- Retr0id 1y ago
  Doesn't fwrite more or less write directly to the FILE buffer, if buffering is enabled? I'm curious to take a closer look at fmtlib/fmt, which APIs treat FILE as non-opaque? Edit: ah, found some of the magic, I think: https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd6b0b3029a0a8c00/include/fmt/format-inl.h#L1513-L1557 https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd... I'm curious how much speedup is gained from this.
  - vitaut 1y ago
    With fwrite that would be another level of buffering in addition to FILE's buffer. If you are interested in what {fmt} is doing, a good starting point is https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd6b0b3029a0a8c00/include/fmt/format-inl.h#L1514 https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd.... It is also possible to bypass stdio completely and get even faster output (https://vitaut.net/posts/2020/optimal-file-buffer-size/ https://vitaut.net/posts/2020/optimal-file-buffer-size/) and while it is great for files, it may introduce interleaving problems with things like stdout.
cryptonector 1y ago
In SunOS 4.x `FILE` was not opaque, and `int fileno(FILE *)` was a macro, not a funciton, and the field of the struct that held the fd number was a `char`. Yeah, that sucked for ages, especially since it bled into the Solaris 2.x 32-bit ABI.
- bodyfour 1y ago
  Indeed, that was the way it originally worked in all UNIXes: https://github.com/dspinellis/unix-history-repo/blob/Research-V7-Snapshot-Development/usr/include/stdio.h#L26 https://github.com/dspinellis/unix-history-repo/blob/Researc... It was a then-important optimization to do the most common operations with macros since calling a function for every getc()/putc() would have slowed I/O down too much. That's why there is also fgetc()/fputc() -- they're the same as getc()/putc() but they're always defined as functions so calling them generated less code at the callsite at the expense of always requiring a function call. A classic speed-vs-space tradeoff. But, yeah, it was a mistake that it originally used a "char" to store the file descriptor. Back then it was typical to limit processes to 20 open files ( https://github.com/dspinellis/unix-history-repo/blob/Research-V7-Snapshot-Development/usr/sys/h/param.h#L13 https://github.com/dspinellis/unix-history-repo/blob/Researc... ) so a "char" I'm sure felt like plenty.

In general, it is a bad practice. However, it can be useful for some low-level libraries. For example, https://github.com/fmtlib/fmt https://github.com/fmtlib/fmt provides a type-safe replacement for `printf` that can write directly to the FILE buffer providing comparable or better performance to native stdio.

Retr0id 1y ago
Doesn't fwrite more or less write directly to the FILE buffer, if buffering is enabled? I'm curious to take a closer look at fmtlib/fmt, which APIs treat FILE as non-opaque? Edit: ah, found some of the magic, I think: https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd6b0b3029a0a8c00/include/fmt/format-inl.h#L1513-L1557 https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd... I'm curious how much speedup is gained from this.
- vitaut 1y ago
  With fwrite that would be another level of buffering in addition to FILE's buffer. If you are interested in what {fmt} is doing, a good starting point is https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd6b0b3029a0a8c00/include/fmt/format-inl.h#L1514 https://github.com/fmtlib/fmt/blob/35dcc58263d6b55419a5932bd.... It is also possible to bypass stdio completely and get even faster output (https://vitaut.net/posts/2020/optimal-file-buffer-size/ https://vitaut.net/posts/2020/optimal-file-buffer-size/) and while it is great for files, it may introduce interleaving problems with things like stdout.

cryptonector 1y ago

In SunOS 4.x `FILE` was not opaque, and `int fileno(FILE *)` was a macro, not a funciton, and the field of the struct that held the fd number was a `char`. Yeah, that sucked for ages, especially since it bled into the Solaris 2.x 32-bit ABI.

bodyfour 1y ago
Indeed, that was the way it originally worked in all UNIXes: https://github.com/dspinellis/unix-history-repo/blob/Research-V7-Snapshot-Development/usr/include/stdio.h#L26 https://github.com/dspinellis/unix-history-repo/blob/Researc... It was a then-important optimization to do the most common operations with macros since calling a function for every getc()/putc() would have slowed I/O down too much. That's why there is also fgetc()/fputc() -- they're the same as getc()/putc() but they're always defined as functions so calling them generated less code at the callsite at the expense of always requiring a function call. A classic speed-vs-space tradeoff. But, yeah, it was a mistake that it originally used a "char" to store the file descriptor. Back then it was typical to limit processes to 20 open files ( https://github.com/dspinellis/unix-history-repo/blob/Research-V7-Snapshot-Development/usr/sys/h/param.h#L13 https://github.com/dspinellis/unix-history-repo/blob/Researc... ) so a "char" I'm sure felt like plenty.