4 ms·

Not a comment on the book but I hope a slightly off topic thread is OK... As a systems programmer, what is the selling point of Zig? For me, memory safety is t

Not a comment on the book but I hope a slightly off topic thread is OK... As a systems programmer, what is the selling point of Zig? For me, memory safety is the number 1, 2, 3, 4 and 5 problem with C. A new language that doesn't solve those problems seems very strange to me. And as a counterpoint, I will tolerate massive number of issues with Rust because it gives us memory safety. I work in security so that may give me a bias. But maybe instead of "bias" it's more like awareness. If you don't deal with it day in, day out, it seems impossible to really internalise how completely fucking broken something like the Linux kernel is. Continuing to write e.g. driver code in an unsafe language is just not acceptable for our civilisation. But, also maybe you don't need full complete memory safety, if a language has well designed abstractions maybe it can just be "safe enough in practice". I've worked on C++ code that feels that way. Is this the deal with Zig?

hansvm 1y ago
You touched on the idea in your last paragraph. Comparing Zig to C using memory safety as a binary threshold instead of a continuous scale is apples to oranges. For one, Rust isn't even at the far right of the memory safety scale, so any argument against Zig due to some epsilon of safety has to reckon with the epsilon Rust is missing as well. That scale exists, and in choosing a language we're choosing (among other things) where on that scale we want to be. Maybe Rust is "good enough", and maybe not. For two, there's a lot more to software design than memory safety _even when your goals revolve around the safety and stability of the end product_. So long as you get the basics right (like how Zig uses slices by default and has defer/errdefer statements), you won't have an RCE from a memory safety bug, or at least not one that you wouldn't likely see in Rust anyway (e.g., suppose you're writing a new data structure in both languages; suppose that it involves back-references or something else where the common pattern in Rust would involve creating backing memory and passing indices around, combined with an unsafe block on the actual cell access to satisfy the borrow checker; the fact that you had to do something in an unsafe block opens you up to safety issues like index wraparound causing OOB reads). If memory safety is "good enough" from an RCE perspective, what are we buying with the rest of Rust's memory safety? Why is it so important? For everything else, you're just preventing normal application bugs -- leaking user data, incorrect computations, random crashes, etc. The importance of normal application bugs varies from project to project, but in the context of systems programming of any kind I'll argue that these are absolutely as critical as RCEs and the other "big" things you're trying to prevent via memory safety. The whole reason you try to prevent an RCE is to ensure that you have access to your files/data/tools and that an attacker doesn't, but if logic bugs corrupt data, crash your tools, and leak PII then you're still fucked. Memory safety didn't save you. That shift in perspective is important. If our goal is preventing ordinary application bugs on top of memory-safety bugs it becomes blatantly obvious that you'd be willing to trade a little safety for some bug prevention. Does Zig actually help reduce bugs though? IME, yes. The big thing it has going for it is that it's a small, cohesive language that gives you the power you need to do the things you're trying to do. Being small makes it possible to create features that compose well and are correct by default, and having the power to do what you're trying to do means you don't have to create convoluted (expensive, bug-prone) workarounds. Some examples: 1. `let` rebindings in Rust are a great feature, and I like them a lot. Better than 90% of the time I use them correctly. The other 10% I'm instead trying to create a new variable and accidentally shadowing an existing one. That's often not the end of the world, but if there exists any code after my `let` rebinding expecting the old value then it's totally incorrect (assuming compatible types so that you don't have a compilation failure). Zig yells loudly about the error. 2. The error-handling system in Zig is easy to get right because of largely compatible return types. By contrast, I see an awful lot of `unwrap` in Rust code in places that really shouldn't have it, and in codebases that try to use `?` or other more-likely-to-be-correct strategies the largely incompatible return types force most people into slapping `anyhow` around everything and calling it a day. 3. Going back to error-handling, Rust allows intentional discards of return values, but if you start by discarding the return value of a non-error function and the signature is later updated to potentially return errors Rust still lets you discard the value. Zig forces you to do something explicitly acknowledging that there was an error. 100% of the time the compiler has yelled at me for a potential logic bug it was correct to do so. It really is just a bunch of little things, but those little things add up and make for a pleasant language with shockingly few bugs. Maybe my team is just even more exceptional than I already think or something and I'm over-indexing on the language, but in the last couple years I haven't seen a single memory safety issue, or much in the way of other bugs in the Zig part of the company.
- throwawaymaths 1y ago
  the far right of the security scale is C (+ isabelle?) again (seL4)
- tialaramex 1y ago
  1. I guess by "let rebinding" you mean shadowing. Clippy has lints for shadowing which are default off, you might find that some patterns of shadowing which are legal in Rust are too often problematic in your software and so you should tell Clippy to warn you about them or even deny them by default. For example you might decide to warn for clippy::shadow_unrelated which says if I have a variable named "goose" and then later in the function I just make another variable that's not obviously derived from the first variable but it is called "goose" anyway, warn me I did that. 2. I don't see any advantage of "largely compatible" error handling over anyhow, maybe you could be more explicit about what you want here or give an example. 3. What you're claiming for Zig here is a semantic check. This could exist but be fallible (it can't be infallible because Rice's Theorem), but I suspect the reality is that it's not a semantic check and you've misattributed your experience since I don't see any such check in Zig. Maybe an example would clarify.
  - hansvm 1y ago
    I really didn't mean to make this about Zig vs Rust. That was a mistake. My real point is that Zig is also a great language, even when you care about safety. That said, the things you asked about: 1. I suppose you might get away with linting all of shadow_reuse, shadow_same, and shadow_unrelated. Would that sufficiently disable the feature? 2. The biggest problems with overusing `anyhow` to circumvent the language's error system are performance and type safety. Perf is probably fine enough for most applications (though, we _are_ talking about systems programming, and this is yet another way in which allocation-free Rust is hard to actually do), but not being able to match on the returned error makes appropriately handling errors clunkier than it should be, leading to devs instead bubbling things up the call stack even more than normal or "handling" the error by ignoring any context and using some cudgel like killing connections regardless of what the underlying error was. 3. Rice's Theorem is only tangentially related. Your goal, and one that Rust also strives for, is to make it so that changes to the type signature of a function require compatible changes at call sites, which you can often handle syntactically. Rust allows `let _ = foo()` to also silence errors, and where Zig normally allows `_ = foo()` to silence unhandled return values it additionally requires `try`, `catch`, or `return` (or similar control flow). You have options like `_ = foo() catch {}` or `_ = foo() catch @panic("WTF")` if you really do want to silence the error, but you'll never be caught off-guard by an error being added to a function's return type.
pron 1y ago
1. Lack of memory safety is a big problem particularly because it's a common cause of dangerous vulnerabilities, but spatial safety is more important than temporal safety for that reason, and Zig does offer spatial safety. 2. Other important issues of concern when it comes to security/correctness are language complexity and compilation speed. A complicated language with lots of implicitness can obscure bugs and make reviews slower. Slow compilation reduces the iteration speed and harms testing. Zig focuses on these two. The language is simple, with no implicitness - overloading of any kind is not allowed and neither are any kind of hidden calls. Even if security were the only concern, it's not clear at all how much complexity and compilation speed should be sacrificed for temporal safety. Remember that it's easy to classify bugs by technical causes, but more diffuse aspects, like testing and review, are also very important, and Zig tries to find a good balance. 3. Nevertheless, the language is still very expressive [1]. In any language, you want the algorithm to be clear and with neither extraneous nor important hidden details for that domain. I think Zig gets that about right for low-level programming (and C++ doesn't). For me, the #1 problem with C is lack of safety and the #2 problem is lack of expressivity. With C++, the main problem for me is language complexity and implicitness, with slow compilation and lack of safety tied for number 2. So Zig is as expressive as C++. but not only safer, but also much simpler, and compiles faster (and improving compilation speed further is an important goal). [1]: I say that two languages are equally expressive if, over all algorithms, idiomatic implementations in both languages (that are roughly equally clear) differ in length by no more than a linear relation with a small constant.
- mi_lk 1y ago
  Can you say more about spatial security v.s. temporal security and how Zig does them?
  - pron 1y ago
    Spatial memory safety means that a language (at least in its subset that's designated "safe") doesn't allow you to manufacture pointers into memory that may contain data of a different type than what's expected by the pointer (we'll call such pointers "invalid"). The classic examples of spatial memory safety is guaranteeing that arrays are never accessed out of bounds (hence "spatial", as in pointers are safely constrained in the address space). Zig guarantees (except when using delineated "unsafe" code) such spatial safety. Temporal memory safety is the guarantee that you never access pointers that have been valid at some point in time after they've become invalid due to reallocation of memory; we call such pointers "dangling" (hence "temporal", as in pointers are safely constrained in time). The classic example of this is use-after-free. Zig does not guarantee temporal safety, and you can accidentally have a dangling pointer (i.e. access a one-time valid pointer after it's become invalid). Invalid pointers are especially dangerous because, in languages where they can occur, they've been a very common source of exploitable security vulnerabilities. However, violating spatial memory bounds is more dangerous, as the result is more easily exploited by attackers in the case of a vulnerability, as it's more predictable.
ptrwis 1y ago
Maybe in the future, if the language has very good AI support, security guarantees of the language won't be as important, as it (ai) will find potential bugs well enough. This may be the case with Zig, as the language is simple and consistent, and the lack of macros will make it easier for LLMs to understand the code.
- bigstrat2003 1y ago
  AI is not remotely capable enough to be trusted with that. Perhaps in the future it will be, but I'm not betting on it with the lack of improvement we have seen thus far.
throwawaymaths 1y ago
1. you get spatial memory safety which iirc is more common for security problems than temporal. 2. honestly for high performance applications you might reach for an ECS type system anyways (i think dom engines do this) at which point you would be getting around the borrow checker anyways.
- bjackman 1y ago
  > you get spatial memory safety which iirc is more common for security problems than temporal. No, at least in the Linux kernel use-after-free is the biggest memory safety issue by a long way. Maybe this isn't true in some other system software? Like if you are dealing with parsing. But, you don't really need memory safety for that usecase, fuzzing is unreasonably effective. (Obviously for greenfield dev, safe languages are the way to go. But setting up a dank fuzzing pipeline is easier than migrating to a new language).

hansvm 1y ago

You touched on the idea in your last paragraph. Comparing Zig to C using memory safety as a binary threshold instead of a continuous scale is apples to oranges. For one, Rust isn't even at the far right of the memory safety scale, so any argument against Zig due to some epsilon of safety has to reckon with the epsilon Rust is missing as well. That scale exists, and in choosing a language we're choosing (among other things) where on that scale we want to be. Maybe Rust is "good enough", and maybe not. For two, there's a lot more to software design than memory safety _even when your goals revolve around the safety and stability of the end product_. So long as you get the basics right (like how Zig uses slices by default and has defer/errdefer statements), you won't have an RCE from a memory safety bug, or at least not one that you wouldn't likely see in Rust anyway (e.g., suppose you're writing a new data structure in both languages; suppose that it involves back-references or something else where the common pattern in Rust would involve creating backing memory and passing indices around, combined with an unsafe block on the actual cell access to satisfy the borrow checker; the fact that you had to do something in an unsafe block opens you up to safety issues like index wraparound causing OOB reads). If memory safety is "good enough" from an RCE perspective, what are we buying with the rest of Rust's memory safety? Why is it so important? For everything else, you're just preventing normal application bugs -- leaking user data, incorrect computations, random crashes, etc. The importance of normal application bugs varies from project to project, but in the context of systems programming of any kind I'll argue that these are absolutely as critical as RCEs and the other "big" things you're trying to prevent via memory safety. The whole reason you try to prevent an RCE is to ensure that you have access to your files/data/tools and that an attacker doesn't, but if logic bugs corrupt data, crash your tools, and leak PII then you're still fucked. Memory safety didn't save you. That shift in perspective is important. If our goal is preventing ordinary application bugs on top of memory-safety bugs it becomes blatantly obvious that you'd be willing to trade a little safety for some bug prevention. Does Zig actually help reduce bugs though? IME, yes. The big thing it has going for it is that it's a small, cohesive language that gives you the power you need to do the things you're trying to do. Being small makes it possible to create features that compose well and are correct by default, and having the power to do what you're trying to do means you don't have to create convoluted (expensive, bug-prone) workarounds. Some examples: 1. `let` rebindings in Rust are a great feature, and I like them a lot. Better than 90% of the time I use them correctly. The other 10% I'm instead trying to create a new variable and accidentally shadowing an existing one. That's often not the end of the world, but if there exists any code after my `let` rebinding expecting the old value then it's totally incorrect (assuming compatible types so that you don't have a compilation failure). Zig yells loudly about the error. 2. The error-handling system in Zig is easy to get right because of largely compatible return types. By contrast, I see an awful lot of `unwrap` in Rust code in places that really shouldn't have it, and in codebases that try to use `?` or other more-likely-to-be-correct strategies the largely incompatible return types force most people into slapping `anyhow` around everything and calling it a day. 3. Going back to error-handling, Rust allows intentional discards of return values, but if you start by discarding the return value of a non-error function and the signature is later updated to potentially return errors Rust still lets you discard the value. Zig forces you to do something explicitly acknowledging that there was an error. 100% of the time the compiler has yelled at me for a potential logic bug it was correct to do so. It really is just a bunch of little things, but those little things add up and make for a pleasant language with shockingly few bugs. Maybe my team is just even more exceptional than I already think or something and I'm over-indexing on the language, but in the last couple years I haven't seen a single memory safety issue, or much in the way of other bugs in the Zig part of the company.

throwawaymaths 1y ago
the far right of the security scale is C (+ isabelle?) again (seL4)
tialaramex 1y ago
1. I guess by "let rebinding" you mean shadowing. Clippy has lints for shadowing which are default off, you might find that some patterns of shadowing which are legal in Rust are too often problematic in your software and so you should tell Clippy to warn you about them or even deny them by default. For example you might decide to warn for clippy::shadow_unrelated which says if I have a variable named "goose" and then later in the function I just make another variable that's not obviously derived from the first variable but it is called "goose" anyway, warn me I did that. 2. I don't see any advantage of "largely compatible" error handling over anyhow, maybe you could be more explicit about what you want here or give an example. 3. What you're claiming for Zig here is a semantic check. This could exist but be fallible (it can't be infallible because Rice's Theorem), but I suspect the reality is that it's not a semantic check and you've misattributed your experience since I don't see any such check in Zig. Maybe an example would clarify.
- hansvm 1y ago
  I really didn't mean to make this about Zig vs Rust. That was a mistake. My real point is that Zig is also a great language, even when you care about safety. That said, the things you asked about: 1. I suppose you might get away with linting all of shadow_reuse, shadow_same, and shadow_unrelated. Would that sufficiently disable the feature? 2. The biggest problems with overusing `anyhow` to circumvent the language's error system are performance and type safety. Perf is probably fine enough for most applications (though, we _are_ talking about systems programming, and this is yet another way in which allocation-free Rust is hard to actually do), but not being able to match on the returned error makes appropriately handling errors clunkier than it should be, leading to devs instead bubbling things up the call stack even more than normal or "handling" the error by ignoring any context and using some cudgel like killing connections regardless of what the underlying error was. 3. Rice's Theorem is only tangentially related. Your goal, and one that Rust also strives for, is to make it so that changes to the type signature of a function require compatible changes at call sites, which you can often handle syntactically. Rust allows `let _ = foo()` to also silence errors, and where Zig normally allows `_ = foo()` to silence unhandled return values it additionally requires `try`, `catch`, or `return` (or similar control flow). You have options like `_ = foo() catch {}` or `_ = foo() catch @panic("WTF")` if you really do want to silence the error, but you'll never be caught off-guard by an error being added to a function's return type.

pron 1y ago

1. Lack of memory safety is a big problem particularly because it's a common cause of dangerous vulnerabilities, but spatial safety is more important than temporal safety for that reason, and Zig does offer spatial safety. 2. Other important issues of concern when it comes to security/correctness are language complexity and compilation speed. A complicated language with lots of implicitness can obscure bugs and make reviews slower. Slow compilation reduces the iteration speed and harms testing. Zig focuses on these two. The language is simple, with no implicitness - overloading of any kind is not allowed and neither are any kind of hidden calls. Even if security were the only concern, it's not clear at all how much complexity and compilation speed should be sacrificed for temporal safety. Remember that it's easy to classify bugs by technical causes, but more diffuse aspects, like testing and review, are also very important, and Zig tries to find a good balance. 3. Nevertheless, the language is still very expressive [1]. In any language, you want the algorithm to be clear and with neither extraneous nor important hidden details for that domain. I think Zig gets that about right for low-level programming (and C++ doesn't). For me, the #1 problem with C is lack of safety and the #2 problem is lack of expressivity. With C++, the main problem for me is language complexity and implicitness, with slow compilation and lack of safety tied for number 2. So Zig is as expressive as C++. but not only safer, but also much simpler, and compiles faster (and improving compilation speed further is an important goal). [1]: I say that two languages are equally expressive if, over all algorithms, idiomatic implementations in both languages (that are roughly equally clear) differ in length by no more than a linear relation with a small constant.

mi_lk 1y ago
Can you say more about spatial security v.s. temporal security and how Zig does them?
- pron 1y ago
  Spatial memory safety means that a language (at least in its subset that's designated "safe") doesn't allow you to manufacture pointers into memory that may contain data of a different type than what's expected by the pointer (we'll call such pointers "invalid"). The classic examples of spatial memory safety is guaranteeing that arrays are never accessed out of bounds (hence "spatial", as in pointers are safely constrained in the address space). Zig guarantees (except when using delineated "unsafe" code) such spatial safety. Temporal memory safety is the guarantee that you never access pointers that have been valid at some point in time after they've become invalid due to reallocation of memory; we call such pointers "dangling" (hence "temporal", as in pointers are safely constrained in time). The classic example of this is use-after-free. Zig does not guarantee temporal safety, and you can accidentally have a dangling pointer (i.e. access a one-time valid pointer after it's become invalid). Invalid pointers are especially dangerous because, in languages where they can occur, they've been a very common source of exploitable security vulnerabilities. However, violating spatial memory bounds is more dangerous, as the result is more easily exploited by attackers in the case of a vulnerability, as it's more predictable.

ptrwis 1y ago

Maybe in the future, if the language has very good AI support, security guarantees of the language won't be as important, as it (ai) will find potential bugs well enough. This may be the case with Zig, as the language is simple and consistent, and the lack of macros will make it easier for LLMs to understand the code.

bigstrat2003 1y ago
AI is not remotely capable enough to be trusted with that. Perhaps in the future it will be, but I'm not betting on it with the lack of improvement we have seen thus far.

throwawaymaths 1y ago

1. you get spatial memory safety which iirc is more common for security problems than temporal. 2. honestly for high performance applications you might reach for an ECS type system anyways (i think dom engines do this) at which point you would be getting around the borrow checker anyways.

bjackman 1y ago
> you get spatial memory safety which iirc is more common for security problems than temporal. No, at least in the Linux kernel use-after-free is the biggest memory safety issue by a long way. Maybe this isn't true in some other system software? Like if you are dealing with parsing. But, you don't really need memory safety for that usecase, fuzzing is unreasonably effective. (Obviously for greenfield dev, safe languages are the way to go. But setting up a dank fuzzing pipeline is easier than migrating to a new language).