5 ms·

I agreed with the general ideas in its original rant, but IMHO the problem is not the POSIX API nor the C language (including "volatile"), those are both well d

by antirez 15y ago

I agreed with the general ideas in its original rant, but IMHO the problem is not the POSIX API nor the C language (including "volatile"), those are both well designed and simple stuff, for the most part (the C standard library is horrid unfortunately). IMHO most of the problems are about the other layers: tricks you need to know about the operating system implementation of POSIX, or dynamic library loading, all the subtle things with different binaries formats, and so forth. A few of this things are easy to solve, for me it is impossible to understand how the libC can be in this sad state, and how the replacements and improvements to it like glib are also a mess. If one day I'll not hack on Redis anymore my mission will be, assuming I'll have another way to pay my bills, to create a replacement for the C standard library. While we are at it, not C++ nor Objective C are the final words on making C better and more comfortable to use (but I think the latter is much better than the former at it). This is surely an area where there is a lot to do. Unfortunately "D" is diverging so much form C that it should completely replacing it to get mainstream: very unlikely given the role C is playing today and the code base. A backward compatible improvement is still what we need I think.

kmm 15y ago
What is so bad about the C standard library? What would you change? I find it lacking a lot of essential features -- can you believe strdup isn't part of the C standard? -- but simplicity and minimalism has always been C's strongest point.
- zeugma 15y ago
  Lack of real string type. String manipulation is a pain and you have to allocate everything by yourself which result in inefficient and dangerous code.
  - apaprocki 15y ago
    Any code can be inefficient if performance wasn't a concern while writing it. I'd say in that case C string manipulation is more efficient by design because you see how many times you are copying something around. When things are abstracted away in something like std::string then you can wind up with code that creates multiple unnecessary copies by accident (e.g. forgetting a '&' to take a reference). Yes, it is more dangerous, but that can be a tradeoff with performance. For example, assume you have a std::hash_map<std::string, int>. There is no way to insert into this without one string copy (C++11 changes this to make zero-copy possible).
    - barrkel 15y ago
      You're imagining a false dichotomy. Having a proper string type doesn't stop you from representing strings as pointers to null-terminated buffers when you need it for the flexibility. Meanwhile, more or less encouraging all software written in the language to use an unsafe idiom (so unsafe that the library itself includes functions that should never be used, like gets()), definitely is responsible for a huge amount of harm.
      - apaprocki 15y ago
        Well what are your thoughts on the example I gave for hash_map? There is always a tradeoff hidden away somewhere. I guess I'd like to see an example of a concrete string type that doesn't incur performance penalties when used in more complex structures. C++11 move semantics / emplace() on containers will fix some of the performance issues with std::string, but support for that everywhere is a ways off. But in C, you'd be left with something like glib's GString which isn't more than API over a struct.
        barrkel 15y ago
        I think your example is irrelevant - to the general case, which is where a string type is useful. If you need to avoid copies that badly in a specific case, don't use std::string or anything like it. For example, I know of one compiler architecture which scanned strings from the source, all the while calculating a hash, and basically interned the string (turning it into an index per unique string) without ever actually copying it. Thereafter, the program used the index (an integer) to represent the string, making for fast lookups and comparisons.
        apaprocki 15y ago
        I guess I just take issue with saying the "general case" usage of strings does not have to avoid copies. That is how we generally wind up with slow, bloated software.
        15y ago
  - kmm 15y ago
    Shouldn't string manipulation be more efficient if you have to everything by hand? Besides, using a recursive memory allocator takes away most of the pain and it's more efficient than a garbage collector.
    - munin 15y ago
      surely every time you re-invent an algorithm you make it more efficient? part of the problem is that there isn't a standard of "maximized efficiency" algorithms for programmers to use, so anything they come up with could be better or worse and for no real reason ..
    - barrkel 15y ago
      It's quite easy to overuse strlen or strcat or otherwise turn an O(n) algorithm into O(n^2). But the efficiency I'd be more concerned with is programmer efficiency finding and fixing security bugs caused by off-by-one buffer lengths and input length checking, and the like (e.g. copying k characters into a char buf[k]).
    - deleted 15y ago
      [deleted]
apaprocki 15y ago
> operating system implementation of POSIX, or dynamic library loading, all the subtle things with different binaries formats, and so forth. That is the real pain point. I embed Spidermonkey on not just Linux, but also Solaris, AIX, and HP-UX. While they are all "POSIX", they all have their peculiar issues which make it harder to dev with or write OSS. Autoconf, while hardly the best option, usually allows enough tweaking to get by in these strange worlds. Solaris is usually the nicest of the pack, while AIX is usually the odd ball. Submitting patches to OSS is easy enough to support these systems most of the time, but the annoying part is that things usually regress again quickly and so you're left constantly bug-fixing when things should "just work". Part of the mess could be avoided if vendors all rallied around GCC/Clang/gdb and redirected their compiler efforts to optimizing open compilers for their hardware. Maybe POSIX needs to evolve faster -- if you view an autoconf test for something as an inconsistency that POSIX could standardize away. EDIT: Here's a perfect example of something which should be "easy" and standard in POSIX. Getting the stack base address of your process. (search for XP_UNIX) http://hg.mozilla.org/mozilla-central/file/90575e23ea93/js/src/jsnativestack.cpp http://hg.mozilla.org/mozilla-central/file/90575e23ea93/js/s...
cageface 15y ago
Objective C is certainly a cleaner object layer over C, but I still find myself writing core data structures and algorithms in C++ on iOS a lot of the time. The Obj-C data structures are so awkward to use and the lack of operating overloading results in a lot of really verbose code for things like intensive string manipulation and matrix math.

by antirez 15y ago

kmm 15y ago

What is so bad about the C standard library? What would you change? I find it lacking a lot of essential features -- can you believe strdup isn't part of the C standard? -- but simplicity and minimalism has always been C's strongest point.

zeugma 15y ago
Lack of real string type. String manipulation is a pain and you have to allocate everything by yourself which result in inefficient and dangerous code.
- apaprocki 15y ago
  Any code can be inefficient if performance wasn't a concern while writing it. I'd say in that case C string manipulation is more efficient by design because you see how many times you are copying something around. When things are abstracted away in something like std::string then you can wind up with code that creates multiple unnecessary copies by accident (e.g. forgetting a '&' to take a reference). Yes, it is more dangerous, but that can be a tradeoff with performance. For example, assume you have a std::hash_map<std::string, int>. There is no way to insert into this without one string copy (C++11 changes this to make zero-copy possible).
  - barrkel 15y ago
    You're imagining a false dichotomy. Having a proper string type doesn't stop you from representing strings as pointers to null-terminated buffers when you need it for the flexibility. Meanwhile, more or less encouraging all software written in the language to use an unsafe idiom (so unsafe that the library itself includes functions that should never be used, like gets()), definitely is responsible for a huge amount of harm.
    - apaprocki 15y ago
      Well what are your thoughts on the example I gave for hash_map? There is always a tradeoff hidden away somewhere. I guess I'd like to see an example of a concrete string type that doesn't incur performance penalties when used in more complex structures. C++11 move semantics / emplace() on containers will fix some of the performance issues with std::string, but support for that everywhere is a ways off. But in C, you'd be left with something like glib's GString which isn't more than API over a struct.
      - barrkel 15y ago
        I think your example is irrelevant - to the general case, which is where a string type is useful. If you need to avoid copies that badly in a specific case, don't use std::string or anything like it. For example, I know of one compiler architecture which scanned strings from the source, all the while calculating a hash, and basically interned the string (turning it into an index per unique string) without ever actually copying it. Thereafter, the program used the index (an integer) to represent the string, making for fast lookups and comparisons.
        apaprocki 15y ago
        I guess I just take issue with saying the "general case" usage of strings does not have to avoid copies. That is how we generally wind up with slow, bloated software.
        15y ago
- kmm 15y ago
  Shouldn't string manipulation be more efficient if you have to everything by hand? Besides, using a recursive memory allocator takes away most of the pain and it's more efficient than a garbage collector.
  - munin 15y ago
    surely every time you re-invent an algorithm you make it more efficient? part of the problem is that there isn't a standard of "maximized efficiency" algorithms for programmers to use, so anything they come up with could be better or worse and for no real reason ..
  - barrkel 15y ago
    It's quite easy to overuse strlen or strcat or otherwise turn an O(n) algorithm into O(n^2). But the efficiency I'd be more concerned with is programmer efficiency finding and fixing security bugs caused by off-by-one buffer lengths and input length checking, and the like (e.g. copying k characters into a char buf[k]).
  - deleted 15y ago
    [deleted]

apaprocki 15y ago

> operating system implementation of POSIX, or dynamic library loading, all the subtle things with different binaries formats, and so forth. That is the real pain point. I embed Spidermonkey on not just Linux, but also Solaris, AIX, and HP-UX. While they are all "POSIX", they all have their peculiar issues which make it harder to dev with or write OSS. Autoconf, while hardly the best option, usually allows enough tweaking to get by in these strange worlds. Solaris is usually the nicest of the pack, while AIX is usually the odd ball. Submitting patches to OSS is easy enough to support these systems most of the time, but the annoying part is that things usually regress again quickly and so you're left constantly bug-fixing when things should "just work". Part of the mess could be avoided if vendors all rallied around GCC/Clang/gdb and redirected their compiler efforts to optimizing open compilers for their hardware. Maybe POSIX needs to evolve faster -- if you view an autoconf test for something as an inconsistency that POSIX could standardize away. EDIT: Here's a perfect example of something which should be "easy" and standard in POSIX. Getting the stack base address of your process. (search for XP_UNIX) http://hg.mozilla.org/mozilla-central/file/90575e23ea93/js/src/jsnativestack.cpp http://hg.mozilla.org/mozilla-central/file/90575e23ea93/js/s...

cageface 15y ago

Objective C is certainly a cleaner object layer over C, but I still find myself writing core data structures and algorithms in C++ on iOS a lot of the time. The Obj-C data structures are so awkward to use and the lack of operating overloading results in a lot of really verbose code for things like intensive string manipulation and matrix math.