9 ms·

This is a long paper and the author has 2 main claims: 1) C Language popularity is more to do with cognitive ease of memory addresses as a conceptual model for

by jasode 9y ago

This is a long paper and the author has 2 main claims: 1) C Language popularity is more to do with cognitive ease of memory addresses as a conceptual model for inspection and change. Author claims memory address mental model overshadows runtime performance. 2) switching to "safe" languages like Java/C#/Rust is not necessary. With no changes/violations to existing C Language specification, a new/different implementation (compiler) can add more runtime safety checks similar to managed languages. An example from the paper: >Consider unchecked array accesses. Nowhere does C define that array accesses are unchecked. It just happens that implementations don’t check them. This is an implementation norm, not a fact of the language. Those 2 ideas look orthogonal but he ties them together at the end. I'll take some poetic license (e.g. a little exaggeration) to reword the author's idea to help spur discussion... Consider the idea of the Sufficiently Smart Compiler[1] that claims that a "slow" and "high-level" language like Python/Ruby could be theoretically analyzed and compiled to be as fast as C or handcrafted assembly. In a way, the author is coming from the opposite direction. If you had a "Sufficiently Smart Runtime" for a new C Language compiler implementation, it could (theoretically) do all sorts of extra checks and bookkeeping that wouldn't require any changes to C source code and wouldn't violate the existing C Language standard. (E.g. Imagine a new C runtime that did many checks similar to Valgrind + UBSAN + ASAN + debugger memory fences, etc.) Would the program execution be slower? Well yes, but that's not really an issue because according to author's claim #1, what programmers really like about C is the mental ease of accessing memory addresses. The performance is important, but it's a secondary benefit -- according to the author. [1] http://wiki.c2.com/?SufficientlySmartCompiler http://wiki.c2.com/?SufficientlySmartCompiler

svachalek 9y ago
I've been out of the C world for a long, long time but it seems to me that anywhere that C's pointer arithmetic and ability to cast pointers to/from other types is objectively appealing, that's going to be one of the cases that a compiler can't understand. Of course there's always the subjective "everything looks like a nail" usage as well, which makes every problem seem like a pointer problem because you've never tried to think of them as anything other than a pointer problem. I'm sure you could cater to that usage with a proper runtime but really, it doesn't hurt to try new things sometimes...
- dbcurtis 9y ago
  In my case, nearly 100% of the C code I write is for embedded systems. Casting a hex literal to a pointer type that is a volatile hardware register is better than dropping into asm.... So yes, compilers will always have a hard time understanding device drivers and such unless you turn hardware device concepts into language primitives.
  - swiley 9y ago
    A more correct thing would probably be to create linker scripts that expose symbols for the registers. It's probably not worth the trouble now but the hypothetical compiler would understand it better.
    - planteen 9y ago
      There still needs to be a description of the underlying hardware behavior somehow. The hardware engineers often give you a somewhat correct Excel sheet or force you to look at the HDL to figure it out.
tyingq 9y ago
"Consider the idea of the Sufficiently Smart Compiler[1] that claims that a "slow" and "high-level" language like Python/Ruby could be theoretically analyzed and compiled to be as fast as C or handcrafted assembly." Nim seems to be trying to fit that space.
ChuckMcM 9y ago
Excellent, I think the author would do well to re-frame the question as you have. If nothing else to put it more clearly into the space of provable compilation. When I transferred into the "Oak" group that later became the "Java" organization, the team I was on was looking at whether or not you could write an OS in Java sort of in spite of its safety rules. This sort of concept has been revisited by Rust with its safe/unsafe modal operation. What both of those efforts have in common is that determining safety may be impossible at the construct level but provable if you were to exhaustively search all possible outcomes. What the paper and your comment add to the discussion is the intriguing idea that you could create a 'safe' backend (say the equivalent of the JVM) as a target for a C compiler. And code that could not be compiled would be flagged for later analysis. Much like VHDL can express hardware that cannot by synthesized you might end up with a C compiler that could compile code that could not be executed. I could be fun to spend a bit of time poking around that rabbit hole.
- marktangotango 9y ago
  The idea you can have a safe c compiler or runtime seems totally absurd to me. Why is any of this even considered seriously?
  - Koshkin 9y ago
    Indeed. Even the "obvious" example of the compiler inserting bound checks in the generated code does not work with the well-known method of marking the beginning of a variable-length memory block at the end of a struct using an array of some fixed size, say, 1 (or even 0).
    - deleted 9y ago
      [deleted]
    - ktRolster 9y ago
      For that situation, you would need to mark the end of the variable-length memory block at run-time.
- 9y ago
ktRolster 9y ago
It's not just the mental ease, it's also the physical typing ease (and in some cases, the possibility). For example, he points out that to connect C to existing parts of the system (which is the OS and OS level tools), all you have to do is call the functions. If you want to call a C library from a Java program, it's a lot more work. Furthermore, C has the capability of understanding Java structures (although it's awkward), but Java has no way of understanding C structures from within the language. There is no way to model a driver I/O port in Java, but in C there is. The paper is worth thinking about. If you are creating a language, take interoperability between already existing languages into consideration. JNI is ok, but think how much better it could be if it did auto-marshalling of objects!
- mike_hearn 9y ago
  That's not inherent to the Java language. You can implement garbage collectors and kernels in Java if you extend the JIT compiler: http://jnode.org/ http://jnode.org/
- deleted 9y ago
  [deleted]
nickpsecurity 9y ago
The problem is that it isn't a new idea. People keep trying it as shown below. Unfortunately, C wasn't designed so much as a modified version of something (BCPL) that was the only collection of features Richards could get to compile on his crappy hardware. It's not designed for easy analysis or safety. So, all the attempts are going to hit problems in what legacy code they can support, their performance, or even effectiveness if about reliability/security in a pointer-heavy language. Compare that to Ada, Wirth's stuff, or Modula-3 to find they don't have that problem or have much less of it because they were carefully designed balancing the various tradeoffs. Ada even meets author's criteria for safe language with explicit memory representation despite him saying safe languages don't have that. To back that up with references, first is a bunch of attempts at safer C's or C-like languages with performance issues. The next two are among most recent and practical at memory safety for C apps far as CompSci goes. The last one is an Ada book that lists by chapter each technique its designer used to systematically mitigate bugs or vulnerabilities in systems code. https://pdfs.semanticscholar.org/a890/a850dc78e65e26f8f4def435b17094ce08cf.pdf https://pdfs.semanticscholar.org/a890/a850dc78e65e26f8f4def4... https://llvm.org/pubs/2006-06-12-PLDI-SAFECode.html https://llvm.org/pubs/2006-06-12-PLDI-SAFECode.html https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/ https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/ http://www.adacore.com/uploads/technical-papers/SafeSecureAdav2015-covered.pdf http://www.adacore.com/uploads/technical-papers/SafeSecureAd...
- jstewartmobile 9y ago
  1% inspiration, 99% perspiration. Needs more sweat.
WalterBright 9y ago
> mental ease I'm a long time C programmer, and I was struck by how clumsy and error-prone any manipulation of C strings turns into. It's really hard to look at a mass of strlen/strcpy/memcpy/etc. and see just what is happening. Contrast that with, say, BASIC or Javascript, where string manipulation is easy, natural, and bug-free. I'm going to disagree about the mental ease of programming in C, and a large part of that is difficulty in building useful abstractions around the pointer model.
- ktRolster 9y ago
  That particular problem (strlen/strcpy/memcpy) comes from the problems of the standard library string functions. It can be solved by creating your own string library. Then string manipulation is easy.
  - WalterBright 9y ago
    That falls over as soon as you integrate with anybody else's C code, including the operating system APIs, and with C string literals :-( If it was as easy as you say, it would have happened. And heaven knows I wrote my own string packages, one after the other, and so did everyone else. I eventually abandoned all of them. C's abstraction abilities are simply not good enough to do a decent string encapsulation.
    - ktRolster 9y ago
      shrug It doesn't fall over. I've done it, the openBSD team has done it. DJB has done it. Maybe something is wrong with your implementation that I can help you with?
      - WalterBright 9y ago
        I'm curious. Got links?
        ktRolster 9y ago
bsder 9y ago
C's popularity is due to the fact that it is predictable within certain bounds (single thread or limited concurrency). No GC pauses, no weird runtime crashes due to a strange constructor, no gigantic exception chains, etc. The only languages in the TIOBE index that can even try to make that claim are: C at #2, C++(if you subset it) at #3, Objective-C/Swift(#18/#11), Assembly at #14, Ada at #29, and maybe FORTRAN(#35). That's not a lot of options if you need runtime predictability. Basically C, C(with additions), C(with additions), assembly(hack, spit), Ada (okay), and FORTRAN (God help you). Even now, that means C or Ada--and the first free Ada compiler was 1992. Yes, Rust is coming. But it's got a way to go yet.
- Volt 9y ago
  The fact that there are these other languages with the same properties means that predictability isn't the real reason, right? It's that it also is sparse in its specification and easy to implement a compiler for.
  - bsder 9y ago
    Please name those languages. I'm serious.
    - pjmlp 9y ago
      Ada, Modula-2, Pascal dialects, Algol 68, PL/M, PL/8, PL/S, NEWP...
- benlorenzetti 9y ago
  C's popularity is due to the fact that it is predictable within certain bounds (single thread or limited concurrency). Your post, and reading a discussion further down about Rust's reference counting, has made me realize something primitive that Rust is getting right--a real move forward--which even those who don't enjoy the default "safety switch" being flipped from C (like me) may agree. The C model for memory in time and space is so clean for heap data and the function call stack for one thread (plus global registers), but C has no community-understood/concurring model when it comes to concurrency. Rust, older C++ libraries, C malloc implementations, and other are all alluding toward the simple memory model for multiple threads, which is reference counted pointers, IMO. Basically use a separate type of pointer for heap data, where the max size of the heap is divided by whatever binary power of 2^p processors exist. Rust folks or other languages are welcome to add more ownership semantics or whatever, but the whole family of languages could benefit by this extension to the lingua Franca of C. We may not even need to add a new nominal pointer type to C, just by fiat understand and expect shared, free store objects to always live inside the lowest 1/p th portion of the word address space.

by jasode 9y ago

svachalek 9y ago

I've been out of the C world for a long, long time but it seems to me that anywhere that C's pointer arithmetic and ability to cast pointers to/from other types is objectively appealing, that's going to be one of the cases that a compiler can't understand. Of course there's always the subjective "everything looks like a nail" usage as well, which makes every problem seem like a pointer problem because you've never tried to think of them as anything other than a pointer problem. I'm sure you could cater to that usage with a proper runtime but really, it doesn't hurt to try new things sometimes...

dbcurtis 9y ago
In my case, nearly 100% of the C code I write is for embedded systems. Casting a hex literal to a pointer type that is a volatile hardware register is better than dropping into asm.... So yes, compilers will always have a hard time understanding device drivers and such unless you turn hardware device concepts into language primitives.
- swiley 9y ago
  A more correct thing would probably be to create linker scripts that expose symbols for the registers. It's probably not worth the trouble now but the hypothetical compiler would understand it better.
  - planteen 9y ago
    There still needs to be a description of the underlying hardware behavior somehow. The hardware engineers often give you a somewhat correct Excel sheet or force you to look at the HDL to figure it out.

tyingq 9y ago

"Consider the idea of the Sufficiently Smart Compiler[1] that claims that a "slow" and "high-level" language like Python/Ruby could be theoretically analyzed and compiled to be as fast as C or handcrafted assembly." Nim seems to be trying to fit that space.

ChuckMcM 9y ago

Excellent, I think the author would do well to re-frame the question as you have. If nothing else to put it more clearly into the space of provable compilation. When I transferred into the "Oak" group that later became the "Java" organization, the team I was on was looking at whether or not you could write an OS in Java sort of in spite of its safety rules. This sort of concept has been revisited by Rust with its safe/unsafe modal operation. What both of those efforts have in common is that determining safety may be impossible at the construct level but provable if you were to exhaustively search all possible outcomes. What the paper and your comment add to the discussion is the intriguing idea that you could create a 'safe' backend (say the equivalent of the JVM) as a target for a C compiler. And code that could not be compiled would be flagged for later analysis. Much like VHDL can express hardware that cannot by synthesized you might end up with a C compiler that could compile code that could not be executed. I could be fun to spend a bit of time poking around that rabbit hole.

marktangotango 9y ago
The idea you can have a safe c compiler or runtime seems totally absurd to me. Why is any of this even considered seriously?
- Koshkin 9y ago
  Indeed. Even the "obvious" example of the compiler inserting bound checks in the generated code does not work with the well-known method of marking the beginning of a variable-length memory block at the end of a struct using an array of some fixed size, say, 1 (or even 0).
  - deleted 9y ago
    [deleted]
  - ktRolster 9y ago
    For that situation, you would need to mark the end of the variable-length memory block at run-time.
9y ago

ktRolster 9y ago

It's not just the mental ease, it's also the physical typing ease (and in some cases, the possibility). For example, he points out that to connect C to existing parts of the system (which is the OS and OS level tools), all you have to do is call the functions. If you want to call a C library from a Java program, it's a lot more work. Furthermore, C has the capability of understanding Java structures (although it's awkward), but Java has no way of understanding C structures from within the language. There is no way to model a driver I/O port in Java, but in C there is. The paper is worth thinking about. If you are creating a language, take interoperability between already existing languages into consideration. JNI is ok, but think how much better it could be if it did auto-marshalling of objects!

mike_hearn 9y ago
That's not inherent to the Java language. You can implement garbage collectors and kernels in Java if you extend the JIT compiler: http://jnode.org/ http://jnode.org/
deleted 9y ago
[deleted]

nickpsecurity 9y ago

The problem is that it isn't a new idea. People keep trying it as shown below. Unfortunately, C wasn't designed so much as a modified version of something (BCPL) that was the only collection of features Richards could get to compile on his crappy hardware. It's not designed for easy analysis or safety. So, all the attempts are going to hit problems in what legacy code they can support, their performance, or even effectiveness if about reliability/security in a pointer-heavy language. Compare that to Ada, Wirth's stuff, or Modula-3 to find they don't have that problem or have much less of it because they were carefully designed balancing the various tradeoffs. Ada even meets author's criteria for safe language with explicit memory representation despite him saying safe languages don't have that. To back that up with references, first is a bunch of attempts at safer C's or C-like languages with performance issues. The next two are among most recent and practical at memory safety for C apps far as CompSci goes. The last one is an Ada book that lists by chapter each technique its designer used to systematically mitigate bugs or vulnerabilities in systems code. https://pdfs.semanticscholar.org/a890/a850dc78e65e26f8f4def435b17094ce08cf.pdf https://pdfs.semanticscholar.org/a890/a850dc78e65e26f8f4def4... https://llvm.org/pubs/2006-06-12-PLDI-SAFECode.html https://llvm.org/pubs/2006-06-12-PLDI-SAFECode.html https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/ https://www.cs.rutgers.edu/~santosh.nagarakatte/softbound/ http://www.adacore.com/uploads/technical-papers/SafeSecureAdav2015-covered.pdf http://www.adacore.com/uploads/technical-papers/SafeSecureAd...

jstewartmobile 9y ago
1% inspiration, 99% perspiration. Needs more sweat.

WalterBright 9y ago

> mental ease I'm a long time C programmer, and I was struck by how clumsy and error-prone any manipulation of C strings turns into. It's really hard to look at a mass of strlen/strcpy/memcpy/etc. and see just what is happening. Contrast that with, say, BASIC or Javascript, where string manipulation is easy, natural, and bug-free. I'm going to disagree about the mental ease of programming in C, and a large part of that is difficulty in building useful abstractions around the pointer model.

ktRolster 9y ago
That particular problem (strlen/strcpy/memcpy) comes from the problems of the standard library string functions. It can be solved by creating your own string library. Then string manipulation is easy.
- WalterBright 9y ago
  That falls over as soon as you integrate with anybody else's C code, including the operating system APIs, and with C string literals :-( If it was as easy as you say, it would have happened. And heaven knows I wrote my own string packages, one after the other, and so did everyone else. I eventually abandoned all of them. C's abstraction abilities are simply not good enough to do a decent string encapsulation.
  - ktRolster 9y ago
    shrug It doesn't fall over. I've done it, the openBSD team has done it. DJB has done it. Maybe something is wrong with your implementation that I can help you with?
    - WalterBright 9y ago
      I'm curious. Got links?
      - ktRolster 9y ago

bsder 9y ago

C's popularity is due to the fact that it is predictable within certain bounds (single thread or limited concurrency). No GC pauses, no weird runtime crashes due to a strange constructor, no gigantic exception chains, etc. The only languages in the TIOBE index that can even try to make that claim are: C at #2, C++(if you subset it) at #3, Objective-C/Swift(#18/#11), Assembly at #14, Ada at #29, and maybe FORTRAN(#35). That's not a lot of options if you need runtime predictability. Basically C, C(with additions), C(with additions), assembly(hack, spit), Ada (okay), and FORTRAN (God help you). Even now, that means C or Ada--and the first free Ada compiler was 1992. Yes, Rust is coming. But it's got a way to go yet.

Volt 9y ago
The fact that there are these other languages with the same properties means that predictability isn't the real reason, right? It's that it also is sparse in its specification and easy to implement a compiler for.
- bsder 9y ago
  Please name those languages. I'm serious.
  - pjmlp 9y ago
    Ada, Modula-2, Pascal dialects, Algol 68, PL/M, PL/8, PL/S, NEWP...
benlorenzetti 9y ago
C's popularity is due to the fact that it is predictable within certain bounds (single thread or limited concurrency). Your post, and reading a discussion further down about Rust's reference counting, has made me realize something primitive that Rust is getting right--a real move forward--which even those who don't enjoy the default "safety switch" being flipped from C (like me) may agree. The C model for memory in time and space is so clean for heap data and the function call stack for one thread (plus global registers), but C has no community-understood/concurring model when it comes to concurrency. Rust, older C++ libraries, C malloc implementations, and other are all alluding toward the simple memory model for multiple threads, which is reference counted pointers, IMO. Basically use a separate type of pointer for heap data, where the max size of the heap is divided by whatever binary power of 2^p processors exist. Rust folks or other languages are welcome to add more ownership semantics or whatever, but the whole family of languages could benefit by this extension to the lingua Franca of C. We may not even need to add a new nominal pointer type to C, just by fiat understand and expect shared, free store objects to always live inside the lowest 1/p th portion of the word address space.