5 ms·

> Moreover, we shouldn't assume that Zig pointers are the only feature that breaks spatial memory safety. It was simply the first one I found after like 2 minut

by azakai 4y ago

> Moreover, we shouldn't assume that Zig pointers are the only feature that breaks spatial memory safety. It was simply the first one I found after like 2 minutes of looking through the docs. After like 5 more minutes I found another: extern unions. Just now I found another: sentinel-terminated pointers, since you could delete the sentinel. You're right on each of those points. But (AFAIK) all those types are explicitly there for C interop. When interfacing with C, yes, Zig will inherit much of C's unsafety. But the big picture is that 1. Idiomatic Zig should not use those things, and 2. Those things are (IIUC) mechanically-identifiable at the type level. A compiler flag could warn about them. You can check if a codebase uses them, and find out exactly where. No such thing is possible for C. Maybe you'd prefer that an unsafe block be required to be placed around each of them, but in the end that's a matter of notation. In theory, Zig could decide to mandate such explicit unsafe blocks, but C couldn't - there isn't a useful safe subset to delineated. I could be wrong, however. Possibly the Zig community will decide that say sentinel-terminated pointers are good to use in general (and not just for C interop). And maybe I've missed something and those types cannot be discovered mechanically for auditing purposes. If either of those is the case then I'd agree Zig lacks spatial memory safety. (Btw, you seem downvoted atm. Not me, of course - there's probably no one else I respect more on the topic of memory safety than you! - and it's disappointing that people downvote comments just because they disagree.)

pcwalton 4y ago
Well, let's turn this around. Is C++ spatially memory safe? C++20 has slices (ranges). They're bounds checked, via the .at() method. You can get the integer overflow semantics of Zig with "-fsanitize=signed-integer-overflow -fsanitize=unsigned-integer-overflow -fsanitize=float-cast-overflow". You could write a checker that enforces that only these features are used (in fact, this checker basically exists--ISO Core C++ Guidelines). I don't see any reason why modern C++ wouldn't be just as spatially memory safe as Zig is, if we're allowed to subset the language in ways more trivial than "disable the unsafe keyword". The main thing that distinguishes Zig at that point would be that it's a simpler language. That's true. But I don't actually think that makes for safer programs, empirically speaking--otherwise, C programs would be safer than C++ programs, and they generally aren't.
- azakai 4y ago
  Yes, technically there might exist a subset of C++ as safe as idiomatic Zig. But in practice inertia and legacy code and lack of ergonomics etc. work against that subset of C++ becoming popular - does anyone constantly type .at()? Zig can do better, if it doesn't squander the opportunity. In other words, this isn't about theoretical subsets of a language. It's that there is an opportunity when designing a new language to use better defaults and have better idiomatic patterns. If Zig and Carbon do it properly, I think they can get to a place where they have practically no spatial memory errors in the real world. That is measurable, in principle, so we'll see how it plays out I suppose.
  - burntsushi 4y ago
    I think it's worthwhile to point out that there are two threads of argument here. One is what it means to be spatially memory safe. Another is whether and how effective various mitigations are.
  - pjmlp 4y ago
    Instead of typing .at() you do #define _ITERATOR_DEBUG_LEVEL 1, done now operator[]() does bounds checking as well.
- user45 4y ago
  Note that STL containers allow you to bounds check "[]" indexing even in release mode if you set the right preprocessor directives. The flag differs between compilers and stdlib implementations, though. For libstdc++ (default on linux), use _GLIBCXX_ASSERTIONS. For example, the following code aborts on release builds too: // g++ -O3 -D_GLIBCXX_ASSERTIONS=1 file.cpp #include <iostream> #include <vector> int main() { std::vector<int> vec{1, 2, 3, 4}; std::cout << vec[4]; } MSVC provides similar functionality using the _ITERATOR_DEBUG_LEVEL directive. The point is that there is no inherent limitation that one has to use the .at() method for bounds checking STL container access.
  - azakai 4y ago
    Good to know, thanks!
burntsushi 4y ago
I do think the 'unsafe' notation is the crucial bit here. Rust has stuff that is heavily (although not completely) motivated by C interop. For example, 'union' in Rust is something that is rarely used outside of C interop. (With some notable exceptions in the standard library.) But, crucially, in order to actually use a Rust 'union', you do have to utter the 'unsafe' notation. I'm not sure I have an opinion as strong as pcwalton here, but I do think that you can't call yourself memory safe (even on a particular dimension) if you have to delineate all of the various language features that are unsafe without having to somehow explicitly annotate it as such in the source. Or at the very least, I don't think you can say that Zig's spatial memory safety is on even footing with Rust's (assuming pcwalton's characterization of Zig's unsafe features is correct). The annotation really is the point here, because you can pin everything (modulo bugs and OS features that permit subversion) with regard to memory safety down to an unsafe annotation somewhere. That's a powerful tool that crystalizes what it means to be a "safe API." As I said, I don't necessarily share pcwalton's strong opinion (although I do weakly agree with him) that having an annotation like 'unsafe' is the only way to go. I do think it's possible to go the Zig route and reduce memory safety bugs. But we should be very clear eyed about the claims being made and how comparisons are drawn.
- azakai 4y ago
  I don't think the annotation itself is actually the point. The point is being able to find all sources of unsafety and reason about them. Annotations do that, but other things can too. Imagine if you could run a tool on a codebase, and it statically found all the unsafe locations, and in idiomatic code there were very few of those, and you could reason about them. That would be equivalent to writing "unsafe" in the code, except you need to run the tool to "see" the annotations. That would be as good as annotations - it could prevent the same number of bugs. (Though I guess an argument could be, maybe some people forget to run the tool; fair enough.) No such tool can exist for C or C++. But such a tool could exist for a new language like Zig and Carbon, if they design themselves in certain ways, at least for spatial memory safety. Concretely, unsafe things like raw pointers in Zig have different types than safe slices, so a tool can actually find them; and unsafe things are (AFAIK) not idiomatic either, so they'd be rare.

by azakai 4y ago

pcwalton 4y ago

Well, let's turn this around. Is C++ spatially memory safe? C++20 has slices (ranges). They're bounds checked, via the .at() method. You can get the integer overflow semantics of Zig with "-fsanitize=signed-integer-overflow -fsanitize=unsigned-integer-overflow -fsanitize=float-cast-overflow". You could write a checker that enforces that only these features are used (in fact, this checker basically exists--ISO Core C++ Guidelines). I don't see any reason why modern C++ wouldn't be just as spatially memory safe as Zig is, if we're allowed to subset the language in ways more trivial than "disable the unsafe keyword". The main thing that distinguishes Zig at that point would be that it's a simpler language. That's true. But I don't actually think that makes for safer programs, empirically speaking--otherwise, C programs would be safer than C++ programs, and they generally aren't.

azakai 4y ago
Yes, technically there might exist a subset of C++ as safe as idiomatic Zig. But in practice inertia and legacy code and lack of ergonomics etc. work against that subset of C++ becoming popular - does anyone constantly type .at()? Zig can do better, if it doesn't squander the opportunity. In other words, this isn't about theoretical subsets of a language. It's that there is an opportunity when designing a new language to use better defaults and have better idiomatic patterns. If Zig and Carbon do it properly, I think they can get to a place where they have practically no spatial memory errors in the real world. That is measurable, in principle, so we'll see how it plays out I suppose.
- burntsushi 4y ago
  I think it's worthwhile to point out that there are two threads of argument here. One is what it means to be spatially memory safe. Another is whether and how effective various mitigations are.
- pjmlp 4y ago
  Instead of typing .at() you do #define _ITERATOR_DEBUG_LEVEL 1, done now operator[]() does bounds checking as well.
user45 4y ago
Note that STL containers allow you to bounds check "[]" indexing even in release mode if you set the right preprocessor directives. The flag differs between compilers and stdlib implementations, though. For libstdc++ (default on linux), use _GLIBCXX_ASSERTIONS. For example, the following code aborts on release builds too: // g++ -O3 -D_GLIBCXX_ASSERTIONS=1 file.cpp #include <iostream> #include <vector> int main() { std::vector<int> vec{1, 2, 3, 4}; std::cout << vec[4]; } MSVC provides similar functionality using the _ITERATOR_DEBUG_LEVEL directive. The point is that there is no inherent limitation that one has to use the .at() method for bounds checking STL container access.
- azakai 4y ago
  Good to know, thanks!

burntsushi 4y ago

I do think the 'unsafe' notation is the crucial bit here. Rust has stuff that is heavily (although not completely) motivated by C interop. For example, 'union' in Rust is something that is rarely used outside of C interop. (With some notable exceptions in the standard library.) But, crucially, in order to actually use a Rust 'union', you do have to utter the 'unsafe' notation. I'm not sure I have an opinion as strong as pcwalton here, but I do think that you can't call yourself memory safe (even on a particular dimension) if you have to delineate all of the various language features that are unsafe without having to somehow explicitly annotate it as such in the source. Or at the very least, I don't think you can say that Zig's spatial memory safety is on even footing with Rust's (assuming pcwalton's characterization of Zig's unsafe features is correct). The annotation really is the point here, because you can pin everything (modulo bugs and OS features that permit subversion) with regard to memory safety down to an unsafe annotation somewhere. That's a powerful tool that crystalizes what it means to be a "safe API." As I said, I don't necessarily share pcwalton's strong opinion (although I do weakly agree with him) that having an annotation like 'unsafe' is the only way to go. I do think it's possible to go the Zig route and reduce memory safety bugs. But we should be very clear eyed about the claims being made and how comparisons are drawn.

azakai 4y ago
I don't think the annotation itself is actually the point. The point is being able to find all sources of unsafety and reason about them. Annotations do that, but other things can too. Imagine if you could run a tool on a codebase, and it statically found all the unsafe locations, and in idiomatic code there were very few of those, and you could reason about them. That would be equivalent to writing "unsafe" in the code, except you need to run the tool to "see" the annotations. That would be as good as annotations - it could prevent the same number of bugs. (Though I guess an argument could be, maybe some people forget to run the tool; fair enough.) No such tool can exist for C or C++. But such a tool could exist for a new language like Zig and Carbon, if they design themselves in certain ways, at least for spatial memory safety. Concretely, unsafe things like raw pointers in Zig have different types than safe slices, so a tool can actually find them; and unsafe things are (AFAIK) not idiomatic either, so they'd be rare.