3 ms·

There are and have been many techniques and projects for making C more memory-safe. The crucial question it always comes down to is what performance hit do you

There are and have been many techniques and projects for making C more memory-safe. The crucial question it always comes down to is what performance hit do you take using them? That's why C has been on top for so long. Seat-of-the-pants hand-crafted C has always been the fastest high-level language.

WalterBright 2y ago
I found that C programs rarely evolve beyond their initial design. The trouble is, it's hard to refactor C programs. For example, struct S { int a; }; struct S s; s.a = 3; struct S *p; p->a = 3; I.e. a . is for direct access, -> for indirect access. Let's say you want to change passing S by value to passing S by pointer. Now you have to update every use, instead of just the declaration. This is how it would work in D: struct S { int a; } S s; s.a = 3; S* p; p.a = 3; ref S q; q.a = 3; And so refactoring becomes much easier, and so happens more often.
- WalterBright 2y ago
  > C has always been the fastest high-level language. C has another big speed problem. Strings are 0 terminated, rather than length terminated. This means constant scanning of strings to find their length. Even worse, the scanning of the string reloads the cache with the string contents, which is pretty bad for performance. Of course, you could use `struct String { char *p; size_t length; };` but since every library you want to connect to uses 0 terminated strings, you're out on your island all alone, so pragmatically it does not work. Another speed-destroying problem with C strings is you cannot take a substring that does not require allocating a new string and then copying the data. (Unless the substring is right-justified.) This is not fast in any universe. D uses length-denoted strings as a basic data type, and with string processing code, it is much faster than C. Substrings are quick and easy. You can still interface with C because D string literals implicitly convert to C string literals, as the literals are 0 terminated. So this works in D: printf("hello world!\n"); (People sometimes rag on me for still using printf, but printf is the most optimized and debugged library function in the world, so I take advantage!)
  - pizlonator 2y ago
    Yeah totally. It's a perfect example of C being optimized for simple mapping onto linear memory, rather than some kind of performance optimum
    - Dylan16807 2y ago
      I don't know, a pointer with a size is also quite simple.
pizlonator 2y ago
> There are and have been many techniques and projects for making C more memory-safe. Sort of. None of them got all the way to safety, or they never got all the way to compatibility with C. Fil-C is novel in that it achieves both safety and compatibility. > The crucial question it always comes down to is what performance hit do you take using them? Is that really the crucial question? I don't think you would have even gotten to asking that question with most attempts to make C memory safe, because they involved experimental academic compilers that could only compile a subset of the language and only worked for a tiny corpus of benchmarks. Lots of C/C++ code is not written with a perf mindset. Most of the UNIX utilities are like that, for example. > That's why C has been on top for so long. Seat-of-the-pants hand-crafted C has always been the fastest high-level language. I don't think that's the reason. C rose to where it is today even when it was much slower than assembly. C was slower than FORTRAN for a long time (maybe still is?) but people preferred C over FORTRAN anyway. C's biggest superpower is how easy it makes it to talk to system ABI (syscalls, dynamic linking, etc).
- blacksqr 2y ago
  >> There are and have been many techniques and projects for making C more memory-safe. > Sort of. Yes. That's why I used the qualifier "more." Our statements are not in conflict. > Fil-C is novel in that it achieves both safety and compatibility. How does it affect performance? >> The crucial question it always comes down to is what performance hit do you take using them? > Is that really the crucial question? Yes, because it's the factor that industry leaders use to decide on which language to use. For example, Apple switching from Pascal to C way back in the Stone Age. The fact that it's the crucial question doesn't mean that lots of people don't consider other factors for their own reasons. > I don't think you would have even gotten to asking that question with most attempts to make C memory safe. Yes, most. But for example, Microsoft's Checked C comes with a performance penalty of almost 10% for a partial solution. Not academic. Very commercial. > C rose to where it is today even when it was much slower than assembly Yes, that's why I said "high-level language." I don't consider assembly high-level, do you? > people preferred C over FORTRAN anyway People preferred C in the 1970s/80s because at the time you could allocate memory dynamically in C but not in FORTRAN. FORTRAN fixed that in the 1990s, but by then there were too few FORTRAN programmers to compete. Since then C has serially defeated all newcomers. Maybe Go or Rust are poised to take it on. When a major operating system switches from C, we'll know.
WalterBright 2y ago
C's memory safety could be drastically improved with the addition of bounds-checked arrays (which is an extension, and does not change existing code): https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html 25 years of experience with D has shown this to be a huge improvement. D also has references as an alternative to pointers. References cannot have arithmetic done on them. Hence, by replacing pointers with references, and with array bounds checking, the incidence of memory corruption is hugely reduced.
- pizlonator 2y ago
  > C's memory safety could be drastically improved with the addition of bounds-checked arrays (which is an extension, and does not change existing code): If you solved that problem then you'd still have a dumpster fire of memory safety issues from bad casts, use after free, etc
  - WalterBright 2y ago
    Array overflows is consistently the number one memory safety bug in shipped code, by a wide margin.
    - pizlonator 2y ago
      Citation needed. (I would have guessed similar to what you said, minus the "by a wide margin" bit.)
      - WalterBright 2y ago
        A few minutes of googling: https://cwe.mitre.org/top25/archive/2024/2024_cwe_top25.html https://cwe.mitre.org/top25/archive/2024/2024_cwe_top25.html https://runsafesecurity.com/blog/memory-safety-vulnerabilities/ https://runsafesecurity.com/blog/memory-safety-vulnerabiliti...
        2y ago

WalterBright 2y ago

I found that C programs rarely evolve beyond their initial design. The trouble is, it's hard to refactor C programs. For example, struct S { int a; }; struct S s; s.a = 3; struct S *p; p->a = 3; I.e. a . is for direct access, -> for indirect access. Let's say you want to change passing S by value to passing S by pointer. Now you have to update every use, instead of just the declaration. This is how it would work in D: struct S { int a; } S s; s.a = 3; S* p; p.a = 3; ref S q; q.a = 3; And so refactoring becomes much easier, and so happens more often.

WalterBright 2y ago
> C has always been the fastest high-level language. C has another big speed problem. Strings are 0 terminated, rather than length terminated. This means constant scanning of strings to find their length. Even worse, the scanning of the string reloads the cache with the string contents, which is pretty bad for performance. Of course, you could use `struct String { char *p; size_t length; };` but since every library you want to connect to uses 0 terminated strings, you're out on your island all alone, so pragmatically it does not work. Another speed-destroying problem with C strings is you cannot take a substring that does not require allocating a new string and then copying the data. (Unless the substring is right-justified.) This is not fast in any universe. D uses length-denoted strings as a basic data type, and with string processing code, it is much faster than C. Substrings are quick and easy. You can still interface with C because D string literals implicitly convert to C string literals, as the literals are 0 terminated. So this works in D: printf("hello world!\n"); (People sometimes rag on me for still using printf, but printf is the most optimized and debugged library function in the world, so I take advantage!)
- pizlonator 2y ago
  Yeah totally. It's a perfect example of C being optimized for simple mapping onto linear memory, rather than some kind of performance optimum
  - Dylan16807 2y ago
    I don't know, a pointer with a size is also quite simple.

pizlonator 2y ago

> There are and have been many techniques and projects for making C more memory-safe. Sort of. None of them got all the way to safety, or they never got all the way to compatibility with C. Fil-C is novel in that it achieves both safety and compatibility. > The crucial question it always comes down to is what performance hit do you take using them? Is that really the crucial question? I don't think you would have even gotten to asking that question with most attempts to make C memory safe, because they involved experimental academic compilers that could only compile a subset of the language and only worked for a tiny corpus of benchmarks. Lots of C/C++ code is not written with a perf mindset. Most of the UNIX utilities are like that, for example. > That's why C has been on top for so long. Seat-of-the-pants hand-crafted C has always been the fastest high-level language. I don't think that's the reason. C rose to where it is today even when it was much slower than assembly. C was slower than FORTRAN for a long time (maybe still is?) but people preferred C over FORTRAN anyway. C's biggest superpower is how easy it makes it to talk to system ABI (syscalls, dynamic linking, etc).

blacksqr 2y ago
>> There are and have been many techniques and projects for making C more memory-safe. > Sort of. Yes. That's why I used the qualifier "more." Our statements are not in conflict. > Fil-C is novel in that it achieves both safety and compatibility. How does it affect performance? >> The crucial question it always comes down to is what performance hit do you take using them? > Is that really the crucial question? Yes, because it's the factor that industry leaders use to decide on which language to use. For example, Apple switching from Pascal to C way back in the Stone Age. The fact that it's the crucial question doesn't mean that lots of people don't consider other factors for their own reasons. > I don't think you would have even gotten to asking that question with most attempts to make C memory safe. Yes, most. But for example, Microsoft's Checked C comes with a performance penalty of almost 10% for a partial solution. Not academic. Very commercial. > C rose to where it is today even when it was much slower than assembly Yes, that's why I said "high-level language." I don't consider assembly high-level, do you? > people preferred C over FORTRAN anyway People preferred C in the 1970s/80s because at the time you could allocate memory dynamically in C but not in FORTRAN. FORTRAN fixed that in the 1990s, but by then there were too few FORTRAN programmers to compete. Since then C has serially defeated all newcomers. Maybe Go or Rust are poised to take it on. When a major operating system switches from C, we'll know.

WalterBright 2y ago

C's memory safety could be drastically improved with the addition of bounds-checked arrays (which is an extension, and does not change existing code): https://www.digitalmars.com/articles/C-biggest-mistake.html https://www.digitalmars.com/articles/C-biggest-mistake.html 25 years of experience with D has shown this to be a huge improvement. D also has references as an alternative to pointers. References cannot have arithmetic done on them. Hence, by replacing pointers with references, and with array bounds checking, the incidence of memory corruption is hugely reduced.

pizlonator 2y ago
> C's memory safety could be drastically improved with the addition of bounds-checked arrays (which is an extension, and does not change existing code): If you solved that problem then you'd still have a dumpster fire of memory safety issues from bad casts, use after free, etc
- WalterBright 2y ago
  Array overflows is consistently the number one memory safety bug in shipped code, by a wide margin.
  - pizlonator 2y ago
    Citation needed. (I would have guessed similar to what you said, minus the "by a wide margin" bit.)
    - WalterBright 2y ago
      A few minutes of googling: https://cwe.mitre.org/top25/archive/2024/2024_cwe_top25.html https://cwe.mitre.org/top25/archive/2024/2024_cwe_top25.html https://runsafesecurity.com/blog/memory-safety-vulnerabilities/ https://runsafesecurity.com/blog/memory-safety-vulnerabiliti...
      - 2y ago