5 ms·

Thanks for the illuminating link. Nevertheless, I unfortunately do not see an easy solution in the near future that avoids C/C++ much as I would love to ditch

Thanks for the illuminating link. Nevertheless, I unfortunately do not see an easy solution in the near future that avoids C/C++ much as I would love to ditch them. Let us leave aside the question of legacy code and focus on future systems projects. General issue: C and C++ have an advantage in the sense that we understand their many problems well due to their long history, and they have formal specs that are far more carefully written than most others, where documentation is spotty and formal specs are often lacking. Example: I have not studied Rust closely, but it seems to me that there are sections that essentially must be placed under "unsafe" for performance or low level access. The oft-repeated claim is that it minimizes the surface, since it localizes the unsafe sections. I have not seen any hard, formal guarantees in either the spec or implementation as to what is precisely meant by this. I am very interested to know how I can reason that the unsafe section has zero side effects outside of it. Such a statement in general must be false, since the unsafe block clearly communicates with the rest of the program, else it is dead code. And if the interface is sandboxed (an imperfect solution), then it implies some kind of performance penalty across the barrier, implying additional cognitive load for a developer since he/she has to reason about the possible performance impacts for such things, among others like array bounds checks. Lastly, if the guarantee regarding the extent of the "unsafe" block is as complicated as I suspect it to be, the load is increased further. Of course, the load comment is really relevant to Rust vs C and not Rust vs C++. Specifics: 1. Go - garbage collector, large runtime/binary sizes, portability issues (AIX: https://groups.google.com/forum/#!topic/golang-nuts/IBi9wqn__FY https://groups.google.com/forum/#!topic/golang-nuts/IBi9wqn_...). 2. Rust - too new, general points above. 3. D - lacks the same energy as the above projects, so likely fares worse than the above. Furthermore, due to its closer ties to C++, its rather incremental nature resulted in a significant loss when C++11/C++14 came around. 4. Others - too many to list, focused on above due to their frequency on Hacker News.

Manishearth 11y ago
> I can reason that the unsafe section has zero side effects outside of it There are basically a core set of invariants that `unsafe` code must hold, and you can reason about it in those terms. http://doc.rust-lang.org/nightly/nomicon/ http://doc.rust-lang.org/nightly/nomicon/ talks a lot about this.
pcwalton 11y ago
Focusing on Rust "unsafe" as some sort of way to ding its safety drives me crazy. "unsafe" blocks in Rust is designed to be just like the compiler backend. We could have built things like Vec::new and vector indexing into the code generator, like Go or Java did. Then we wouldn't need the "unsafe" construct in the language (except to interface with kernel32.dll and/or syscalls). But that would make our lives harder for absolutely no extra safety gain. It's precisely the same thing as implementing things in the compiler backend, but it's safer than doing that, because we get to write code in a stronger type system than that of raw LLVM. More succinctly: What makes the compiler backend safe while unsafe blocks are not? Can you name one thing that makes the compiler backend safer than code written in unsafe blocks?
- gajjanag 11y ago
  I did not "ding" it, but asked for an honest, transparent response if available. It can only help the Rust community by making such things clear. I focused on it as it is what I got probing Rust since after all there can be "no free lunch". The compiler backend may be viewed as a smaller component that needs to be trusted. General code lies on top of it, and there is a big difference due to code volume.
  - pcwalton 11y ago
    I don't understand what "there is a big difference due to code volume" means.
    - gajjanag 11y ago
      I was referring to all usage of Rust by clients across the world versus a single compiler and reference standard library. Many of these clients will need "unsafe" blocks, and the combined length of these "unsafe" blocks will exceed that of the standard library and compiler assuming Rust adoption is high. This is what concerns me: I wished Rust's "unsafe" blocks could have been exclusively confined to some things in the reference language compiler and standard library. Unfortunately, it seems like many reasonable systems applications still need access to unsafe blocks for reasons of performance, low level access, etc and such needs are not uncommon given the target systems audience.
nickpsecurity 11y ago
"General issue: C and C++ have an advantage in the sense that we understand their many problems well due to their long history, and they have formal specs that are far more carefully written than most others, where documentation is spotty and formal specs are often lacking. " See, even that is misleading. The C specs have a ridiculous amount of undefined behavior and dark corners that can mess up developers. C++ probably does too more than I know. Their specs are actually so hard to formalize that people got Master's degrees, etc for pulling off part of that after decades of people trying. Whereas, many LISP's, ML's, and Wirth languages (eg Oberon/Modula lines) were described quite succinctly and including code. C & C++ specs are so horrible you can't even be sure what your code will do even if you write it to spec. !? "Example: I have not studied Rust closely, but it seems to me that there are sections that essentially must be placed under "unsafe" for performance or low level access." Others are addressing the point about Rust, which I'm ignorant about. What I can say is the generic rule on this issue. Most safe-by-default type languages wrap unsafe behavior behind function calls which still have type/interface checks. If that code behaves correctly, you get to leverage type system of safer language to make sure it works with the rest correctly. If it behaves incorrectly, one of two things happen: local damage where an incorrect result is obtained; application crash or hack. You wouldn't be using the unsafe code unless you thought it was necessary. So, doing safe + a little unsafe adds no risk vs going all unsafe in C, etc. Yet, it counters many risks. So, it's a good tradeoff even if unsafe code can have unpredictable effects on the rest. Note that one can attempt to further mitigate without sandboxing by writing that unsafe code in a way amendable to static analysis or through testing of what it does. One can use something like Frama-C or SPARK Ada to model that one part to make it as bulletproof as possible. Only then include that algorithm into the otherwise safe program. And again wrapped with interface safety to counter even more risk. C and C++ will always have an advantage over sheer momentum. However, there's always alternatives available that lack the problems of the above. They don't have to have the GC, portability, and size issues as Ada and Wirth's languages always showed. I mean, Wirth's Pascal/P was safer than C, efficient (although not max), and more portable given backend was simple, stack-machine anyone could implement. It was ported to 70+ architectures that differed a lot. The ports of Oberon compiler and OS to each new hardware took 1-2 undergrads under 2 years each time due to good design, modularity, and simplicity. What kind of effort have the UNIX's and C compilers took to put on each new architecture? ;) Anyway, I agree if you want max contributions by existing programmers that C is likely best choice for OS development. Or at least the kernel of something like JX Operating System or VerveOS where rest is done safely. If you want best results, then avoiding C and using safer alternatives is best. You'll get so much more done w/ higher robustness in the time you save debugging. :)

Manishearth 11y ago

> I can reason that the unsafe section has zero side effects outside of it There are basically a core set of invariants that `unsafe` code must hold, and you can reason about it in those terms. http://doc.rust-lang.org/nightly/nomicon/ http://doc.rust-lang.org/nightly/nomicon/ talks a lot about this.

pcwalton 11y ago

Focusing on Rust "unsafe" as some sort of way to ding its safety drives me crazy. "unsafe" blocks in Rust is designed to be just like the compiler backend. We could have built things like Vec::new and vector indexing into the code generator, like Go or Java did. Then we wouldn't need the "unsafe" construct in the language (except to interface with kernel32.dll and/or syscalls). But that would make our lives harder for absolutely no extra safety gain. It's precisely the same thing as implementing things in the compiler backend, but it's safer than doing that, because we get to write code in a stronger type system than that of raw LLVM. More succinctly: What makes the compiler backend safe while unsafe blocks are not? Can you name one thing that makes the compiler backend safer than code written in unsafe blocks?

gajjanag 11y ago
I did not "ding" it, but asked for an honest, transparent response if available. It can only help the Rust community by making such things clear. I focused on it as it is what I got probing Rust since after all there can be "no free lunch". The compiler backend may be viewed as a smaller component that needs to be trusted. General code lies on top of it, and there is a big difference due to code volume.
- pcwalton 11y ago
  I don't understand what "there is a big difference due to code volume" means.
  - gajjanag 11y ago
    I was referring to all usage of Rust by clients across the world versus a single compiler and reference standard library. Many of these clients will need "unsafe" blocks, and the combined length of these "unsafe" blocks will exceed that of the standard library and compiler assuming Rust adoption is high. This is what concerns me: I wished Rust's "unsafe" blocks could have been exclusively confined to some things in the reference language compiler and standard library. Unfortunately, it seems like many reasonable systems applications still need access to unsafe blocks for reasons of performance, low level access, etc and such needs are not uncommon given the target systems audience.

nickpsecurity 11y ago

"General issue: C and C++ have an advantage in the sense that we understand their many problems well due to their long history, and they have formal specs that are far more carefully written than most others, where documentation is spotty and formal specs are often lacking. " See, even that is misleading. The C specs have a ridiculous amount of undefined behavior and dark corners that can mess up developers. C++ probably does too more than I know. Their specs are actually so hard to formalize that people got Master's degrees, etc for pulling off part of that after decades of people trying. Whereas, many LISP's, ML's, and Wirth languages (eg Oberon/Modula lines) were described quite succinctly and including code. C & C++ specs are so horrible you can't even be sure what your code will do even if you write it to spec. !? "Example: I have not studied Rust closely, but it seems to me that there are sections that essentially must be placed under "unsafe" for performance or low level access." Others are addressing the point about Rust, which I'm ignorant about. What I can say is the generic rule on this issue. Most safe-by-default type languages wrap unsafe behavior behind function calls which still have type/interface checks. If that code behaves correctly, you get to leverage type system of safer language to make sure it works with the rest correctly. If it behaves incorrectly, one of two things happen: local damage where an incorrect result is obtained; application crash or hack. You wouldn't be using the unsafe code unless you thought it was necessary. So, doing safe + a little unsafe adds no risk vs going all unsafe in C, etc. Yet, it counters many risks. So, it's a good tradeoff even if unsafe code can have unpredictable effects on the rest. Note that one can attempt to further mitigate without sandboxing by writing that unsafe code in a way amendable to static analysis or through testing of what it does. One can use something like Frama-C or SPARK Ada to model that one part to make it as bulletproof as possible. Only then include that algorithm into the otherwise safe program. And again wrapped with interface safety to counter even more risk. C and C++ will always have an advantage over sheer momentum. However, there's always alternatives available that lack the problems of the above. They don't have to have the GC, portability, and size issues as Ada and Wirth's languages always showed. I mean, Wirth's Pascal/P was safer than C, efficient (although not max), and more portable given backend was simple, stack-machine anyone could implement. It was ported to 70+ architectures that differed a lot. The ports of Oberon compiler and OS to each new hardware took 1-2 undergrads under 2 years each time due to good design, modularity, and simplicity. What kind of effort have the UNIX's and C compilers took to put on each new architecture? ;) Anyway, I agree if you want max contributions by existing programmers that C is likely best choice for OS development. Or at least the kernel of something like JX Operating System or VerveOS where rest is done safely. If you want best results, then avoiding C and using safer alternatives is best. You'll get so much more done w/ higher robustness in the time you save debugging. :)