5 ms·

A lot of C developers I know see value in keeping the C standard library small. I implemented my own `l_string` (length-string) struct and associated libraries

A lot of C developers I know see value in keeping the C standard library small. I implemented my own `l_string` (length-string) struct and associated libraries for a class (we weren't allowed to use external libraries) and it was not prohibitively difficult. Something like this probably covers most use cases: https://github.com/antirez/sds https://github.com/antirez/sds

deleted 7y ago
[deleted]
sdegutis 7y ago
I really love his trick of returning a pointer to the inside of his own allocated memory which is then compatible with C's stdlib string functions, while having his own string functions use pointer arithmetic to get the real struct back out and operate on that. Genius! This is the kind of thing that keeps drawing me to C every once in a while. C is almost like a puzzle, to figure out how you can accomplish what would be trivial in JavaScript, but with all these restrictions caused by an almost extreme simplicity. (Granted, C is not actually simple when you start hitting all the weird edge-cases and surprising UB, etc.)
- stkdump 7y ago
  I don't love this trick. To get this to work, they had to typedef the string type to char*, which means the API will also accept any normal string and probably crash at runtime. So to get this little convenience feature, type safety was sacrificed.
  - tmp0258 7y ago
    Seems to me that it would also prevent tools like the address sanitizer and valgrind from detecting some invalid memory accesses (corresponding to negative indexes). A "more conventional" string library [1] might be safer. [1] for example: https://github.com/websnarf/bstrlib/blob/master/bstrlib.txt https://github.com/websnarf/bstrlib/blob/master/bstrlib.txt
einpoklum 7y ago
> A lot of C developers I know see value in keeping the C standard library small. And what value what that be? ... if you're going to say "easier to fit a small implementation into the memory of some microcontroller" - not a sufficient argument. There's always some smaller system with not enough memory, and larger systems with more than enough.
- kjeetgill 7y ago
  This is C land. You're not going to make it far by making the case that bloat is acceptable. People have a range of technical and aesthetic reasons for hating bloat and C attracts a lot of them. Off the top of my head: - Smaller memory footprints - Easier reimplementation - Lower attack surface - Less room for breaking changes - Less baggage for when we inevitably wish we could deprecate things
  - AceJohnny2 7y ago
    > - Lower attack surface wut?
    - Steven_Vellon 7y ago
      String formatting libraries, and other things that work with buffers are frequent attack surfaces for buffer overflow attacks. Granted, using C means developers often implement these operations themselves which introduces the possibility of creating more attack surfaces. But it's less likely that the standard library presents an attack surface when the standard library is tiny.
      - antirez 7y ago
        Actually not having dynamic strings and buffers in the library is one of the main reasons people do pointer math and reallocations by hand introducing memory corruption bugs.
      - AceJohnny2 7y ago
        I mean, C's attack surface is like that of activated charcoal. I'm not sure that C's small standard library gives it a smaller attack surface, specifically because it means programmers who have better things to do are forced to reinvent the wheel, poorly [1]. But mostly, because C's lack of guardrails means it takes active effort on even trivial operations to be safe. I've been working with it for nearly two decades, and every year I think more that C programs should be confined to a well-guarded quarantined area with hazard trefoils and a "beware of the leopard" sign. [1] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule edit: people reimplementing their "safe" string library isn't something to brag about, but be ashamed of our entire industry for.
nneonneo 7y ago
Sadly, real C standard libraries are anything but small. glibc is pretty huge; statically linking a simple hello world program produces something like 750KB of output (libc.so itself is nearly 2MB in size). A quick peek at glibc's documentation shows just how much stuff there is: https://www.gnu.org/software/libc/manual/html_mono/libc.html https://www.gnu.org/software/libc/manual/html_mono/libc.html. Unfortunately, I wouldn't call all of it critically useful stuff. For example, "strfry" and "memfrob" are useless beyond toy applications as neither are remotely secure; "l64a" and "a64l" are useless (and possibly even dangerous) because they look like they implement base64 but with a different alphabet (and a much worse API); they have a hilariously large number of random number generators, _none_ of which produce high-quality random numbers for simulation or especially cryptography; etc. etc. Neither MSVC nor macOS will let you truly statically link a binary (because the system interfaces aren't constant), so it's harder to directly compare, but on macOS "libSystem.dylib" depends on basically everything in /usr/lib/system which is about 5MB of code. So yeah, I don't think I'd call many real libc libraries "small" by any stretch.
- dottrap 7y ago
  Here is a web page comparing the different C library implementations: http://www.etalabs.net/compare_libcs.html http://www.etalabs.net/compare_libcs.html For the .a size: musl: 426k uClibc: 500k dietlibc: 120k If you statically link to these, your final binary will only pull what is actually needed and be smaller than this.
  - BubRoss 7y ago
    I know they are smaller, but why are they able to only pull in what you use when other libc implementations can't?
    - dottrap 7y ago
      Any that allows static linking will get the same benefit. For the three mentioned in the ancestor thread: Apple doesn't ship a static libc, so developers must dynamically link. glibc does allow static linking, but it is something they don't really go out of their way to promote and I recall some historic pains trying to get it to work right. Maybe that all works better today. MSVC does allow static linking to msvcrt. MSVC is a little different than other platforms because they have permutations (and associated rules) you must be aware of: [debug | release], [static | dynamic], [single-threaded | multithreaded].
antirez 7y ago
I don't see value in keeping the C standard library small in this regard, since everybody is going anyway to implement this kind of code from scratch, it's like not having stdio.h for FILE* operations. However... the library you linked is surely a good pick according to me since I'm the author myself :-D

deleted 7y ago

[deleted]

sdegutis 7y ago

I really love his trick of returning a pointer to the inside of his own allocated memory which is then compatible with C's stdlib string functions, while having his own string functions use pointer arithmetic to get the real struct back out and operate on that. Genius! This is the kind of thing that keeps drawing me to C every once in a while. C is almost like a puzzle, to figure out how you can accomplish what would be trivial in JavaScript, but with all these restrictions caused by an almost extreme simplicity. (Granted, C is not actually simple when you start hitting all the weird edge-cases and surprising UB, etc.)

stkdump 7y ago
I don't love this trick. To get this to work, they had to typedef the string type to char*, which means the API will also accept any normal string and probably crash at runtime. So to get this little convenience feature, type safety was sacrificed.
- tmp0258 7y ago
  Seems to me that it would also prevent tools like the address sanitizer and valgrind from detecting some invalid memory accesses (corresponding to negative indexes). A "more conventional" string library [1] might be safer. [1] for example: https://github.com/websnarf/bstrlib/blob/master/bstrlib.txt https://github.com/websnarf/bstrlib/blob/master/bstrlib.txt

einpoklum 7y ago

> A lot of C developers I know see value in keeping the C standard library small. And what value what that be? ... if you're going to say "easier to fit a small implementation into the memory of some microcontroller" - not a sufficient argument. There's always some smaller system with not enough memory, and larger systems with more than enough.

kjeetgill 7y ago
This is C land. You're not going to make it far by making the case that bloat is acceptable. People have a range of technical and aesthetic reasons for hating bloat and C attracts a lot of them. Off the top of my head: - Smaller memory footprints - Easier reimplementation - Lower attack surface - Less room for breaking changes - Less baggage for when we inevitably wish we could deprecate things
- AceJohnny2 7y ago
  > - Lower attack surface wut?
  - Steven_Vellon 7y ago
    String formatting libraries, and other things that work with buffers are frequent attack surfaces for buffer overflow attacks. Granted, using C means developers often implement these operations themselves which introduces the possibility of creating more attack surfaces. But it's less likely that the standard library presents an attack surface when the standard library is tiny.
    - antirez 7y ago
      Actually not having dynamic strings and buffers in the library is one of the main reasons people do pointer math and reallocations by hand introducing memory corruption bugs.
    - AceJohnny2 7y ago
      I mean, C's attack surface is like that of activated charcoal. I'm not sure that C's small standard library gives it a smaller attack surface, specifically because it means programmers who have better things to do are forced to reinvent the wheel, poorly [1]. But mostly, because C's lack of guardrails means it takes active effort on even trivial operations to be safe. I've been working with it for nearly two decades, and every year I think more that C programs should be confined to a well-guarded quarantined area with hazard trefoils and a "beware of the leopard" sign. [1] https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule edit: people reimplementing their "safe" string library isn't something to brag about, but be ashamed of our entire industry for.

nneonneo 7y ago

Sadly, real C standard libraries are anything but small. glibc is pretty huge; statically linking a simple hello world program produces something like 750KB of output (libc.so itself is nearly 2MB in size). A quick peek at glibc's documentation shows just how much stuff there is: https://www.gnu.org/software/libc/manual/html_mono/libc.html https://www.gnu.org/software/libc/manual/html_mono/libc.html. Unfortunately, I wouldn't call all of it critically useful stuff. For example, "strfry" and "memfrob" are useless beyond toy applications as neither are remotely secure; "l64a" and "a64l" are useless (and possibly even dangerous) because they look like they implement base64 but with a different alphabet (and a much worse API); they have a hilariously large number of random number generators, _none_ of which produce high-quality random numbers for simulation or especially cryptography; etc. etc. Neither MSVC nor macOS will let you truly statically link a binary (because the system interfaces aren't constant), so it's harder to directly compare, but on macOS "libSystem.dylib" depends on basically everything in /usr/lib/system which is about 5MB of code. So yeah, I don't think I'd call many real libc libraries "small" by any stretch.

dottrap 7y ago
Here is a web page comparing the different C library implementations: http://www.etalabs.net/compare_libcs.html http://www.etalabs.net/compare_libcs.html For the .a size: musl: 426k uClibc: 500k dietlibc: 120k If you statically link to these, your final binary will only pull what is actually needed and be smaller than this.
- BubRoss 7y ago
  I know they are smaller, but why are they able to only pull in what you use when other libc implementations can't?
  - dottrap 7y ago
    Any that allows static linking will get the same benefit. For the three mentioned in the ancestor thread: Apple doesn't ship a static libc, so developers must dynamically link. glibc does allow static linking, but it is something they don't really go out of their way to promote and I recall some historic pains trying to get it to work right. Maybe that all works better today. MSVC does allow static linking to msvcrt. MSVC is a little different than other platforms because they have permutations (and associated rules) you must be aware of: [debug | release], [static | dynamic], [single-threaded | multithreaded].

antirez 7y ago

I don't see value in keeping the C standard library small in this regard, since everybody is going anyway to implement this kind of code from scratch, it's like not having stdio.h for FILE* operations. However... the library you linked is surely a good pick according to me since I'm the author myself :-D