7 ms·

The recent bug in the Linux kernel Rust code, based on my understanding, was in unsafe code, and related to interop with C. So I wouldn't really classify it as

by tasn 9mo ago

The recent bug in the Linux kernel Rust code, based on my understanding, was in unsafe code, and related to interop with C. So I wouldn't really classify it as a Rust bug. In fact, under normal circumstances (no interop), people rarely use unsafe in Rust, and the use is very isolated. I think the idea of developers developing a "bugs antenna" is good in theory, though in practice the kernel, Redis, and many other projects suffer from these classes of bugs consistently. Additionally, that's why people use linters and code formatters even though developers can develop a sensitivity to coding conventions (in fact, these tools used to be unpopular in C-land). Trusting humans develop sensibility is just not enough. Specifically, about the concurrency: Redis is (mostly) single-threaded, and I guess that's at least in part because of the difficulty of building safe, fast and highly-concurrent C applications (please correct me if I'm wrong). Can people write safer C (e.g. by using sds.c and the likes)? For sure! Though we've been writing C for 50+ years at this point, at some point "people can just do X" is no longer a valid argument. As while we could, in fact we don't.

antirez 9mo ago
In Rust you can avoid "unsafe" when you use Rust like it was Go or Python. If you write low level code, that is where C is in theory replaceable only by Rust (and not by Go), then you find yourself in need of writing many unsafe sections. And to lower the amount of unsafe sections, you have to build unnatural abstractions, often, in order to group such unsafe sections into common patterns. Is is a tradeoff, not a silver bullet.
- tasn 9mo ago
  I think this framing is a bit backwards. Many C programs (and many parts of C programs) would benefit from being more like Go or Python as evident by your very own sds.c. Now, if what you're saying is that with super highly optimized sections of a codebase, or extremely specific circumstances (some kernel drivers) you'd need a bit of unsafe rust: then sure. Though all of a sudden you flipped the script, and the unsafe becomes the exception, not the rule; and you can keep those pieces of code contained. Similarly to how C programmers use inline assembly in some scenarios. Funny enough, this is similar to something that Rust did the opposite of C, and is much better for it: immutable by default (let mut vs. const in C) and non-nullable by default (and even being able to define something as non-null). Flipping the script so that GOOD is default and BAD is rare was a huge win. I definitely don't think Rust is a silver bullet, though I'd definitely say it's at least a silver alloy bullet. At least when it comes to the above topics.
- burntsushi 9mo ago
  Not necessarily at all. Go peruse the `regex` crate source code, including its dependencies. The biggest `unsafe` sections are probably for SIMD accelerated search. There's no "unnatural abstractions" there. Just a memmem-like interface. There's some `unsafe` for eliding bounds checks in the main DFA search loops. No unnatural abstractions there either. There's also some `unsafe` for some synchronization primitives for managing mutable scratch space to use during a search. A C library (e.g., PCRE2) makes the caller handle this. The `regex` crate does it for you. But not for unnatural reasons. To make using regexes simpler. There are lower level APIs that provide the control of C if you need it. That's pretty much it. All told, this is a teeny tiny fraction of the code in the `regex` crate (and all of its dependencies). Finally, a demonstration of C-like speed: https://github.com/BurntSushi/rebar?tab=readme-ov-file#summary-of-search-time-benchmarks https://github.com/BurntSushi/rebar?tab=readme-ov-file#summa... > Is is a tradeoff, not a silver bullet. Uncontroversial.
- josephg 9mo ago
  In my experience (several years of writing high performance rust code), there’s only really 2 instances where you need unsafe blocks: - C interop - Low level machine code (eg inline assembly) Most programs don’t need to do either of those things. I think you could directly port redis to entirely safe rust, and it would be just as fast. (Though there will need be unsafe code somewhere to wrap epoll). And even when you need a bit of unsafe, it’s usually a tiny minority of any given program. I used to think you needed unsafe for custom container types, but now I write custom container types in purely safe rust on top of Vec. The code is simpler, and easier to debug. And I’m shocked to find performance has mostly improved as a result.
pclmulqdq 9mo ago
I hear "people rarely use unsafe rust" quite a lot, but every time I see a project or library with C-like performance, there's a _lot_ of unsafe code in there. Treating bugs in unsafe code as not being bugs in rust code is kind of silly, also.
- antirez 9mo ago
  Exactly. You don't need much unsafe if you use Rust to replace a Python project, for instance. If there is lower level code, high performances needs, things change.
  - kibwen 9mo ago
    For replacing a Python project with Rust, unsafe blocks will comprise 0% of your code. For replacing a C project with Rust, unsafe blocks will comprise about 5% of your code. The fact that the percentage is higher in the latter case doesn't change the fact that 95% of your codebase is just as safe as the Python project would be.
    - antirez 9mo ago
      A big amount of C code does not do anything unsafe as well, it calls other stuff, do loops, logic business, and so forth. It is also wrong to believe 100% of the C code is basically unsafe.
      - staticassertion 9mo ago
        You're swapping definitions of unsafe. Earlier you were referring to the `unsafe` keyword. Now you're using `unsafe` to refer to a property of code. This makes it easy to say things like "It is also wrong to believe 100% of the C code is basically unsafe" but you're just swapping definitions partway through the conversation.
        broken-kebab 9mo ago
        What I see is that antirez claims that absence of "safe" (as syntax) in C lang doesn't automatically mean that all of C code is unsafe (as property). There's no swapping of definitions as I see it.
pjdesno 9mo ago
> was in unsafe code, and related to interop with C 1) "interop with C" is part of the fundamental requirements specification for any code running in the Linux kernel. If Rust can't handle that safely (not Rust "safe", but safely), it isn't appropriate for the job. 2) I believe the problem was related to the fact that Rust can't implement a doubly-linked list in safe code. This is a fundamental limitation, and again is an issue when the fundamental requirement for the task is to interface to data structures implemented as doubly-linked lists. No matter how good a language is, if it doesn't have support for floating point types, it's not a good language for implementing math libraries. For most applications, the inability to safely express doubly-linked lists and difficulty in interfacing with C aren't fundamental problems - just don't use doubly-linked lists or interface with C code. (well, you still have to call system libraries, but these are slow-moving APIs that can be wrapped by Rust experts) For this particular example, however, C interop and doubly-linked lists are fundamental parts of the problem to be solved by the code.
- kibwen 9mo ago
  > If Rust can't handle that safely (not Rust "safe", but safely), it isn't appropriate for the job. Rust is no less safe at C interop than using C directly.
  - hirvi74 9mo ago
    (Not the user you were replying to) If Rust is no less safe than C in such a regard, then what benefit is Rust providing that C could not? I am genuinely curious because OS development is not my forte. I assume the justification to implement Rust must be contingent on more than Rust just being 'newer = better', right?
    - gf000 9mo ago
      It's not less safe in C interop. It is significantly safer at everything else.
  - smj-edison 9mo ago
    As long as you keep C pointers as pointers. The mutable aliasing rules can bite you though.
nec4b 9mo ago
>> I guess that's at least in part because of the difficulty of building safe, fast and highly-concurrent C applications (please correct me if I'm wrong). You wrote that question in a browser mostly written in C++ language, running on an OS most likely written in C language.
- gf000 9mo ago
  Just because the pyramids exist, it means they were easy to build? OS and browser development are seriously hard and took countless expert man hours.
  - nec4b 9mo ago
    OS can be actually pretty simple to make. Sometimes it's a part of a CS curriculum to make one. If it were so much easier to do it in other languages (e.g. in Rust), don't you think we would already be using them?
    - gf000 9mo ago
      https://github.com/flosse/rust-os-comparison https://github.com/flosse/rust-os-comparison Writing a toy one? Sure. Writing a real one? Who's gonna write all the drivers and the myriad other things? And the claim was not that it's "so much easier", but that it is so much easier to write it in a secure way. Which claim is true. But it's still a complex and hard program. (And don't even get started on browsers, it's no accident that even Microsoft dropped maintaining their own browser).
      - nec4b 9mo ago
        The toy one can still be as highly concurrent as the the real one. The amount of drivers written for it doesn't matter. The point is if it were much easier, then they would overtake existing ones easily, just by adding features and iterating so much faster and that is clearly not the case. >>difficulty of building safe, fast and highly-concurrent C This was the original claim. The answer is, there is a tonne of C code out there that is safe, fast and concurrent. Isn't it logical? We have been using C for the last 50 years to build stuff with it and there is a lot of it. There doesn't seem to be a big jump in productivity with the newer generation of low level languages, even though they have many improvements over C. This is anecdotal, I used to do a lot of low level C and C++ development. And C++ is a much bigger language then C. And honestly I don't think I was ever more productive with it. Maybe the code looked more organized and extendable, but it took the same or larger amount of time to write it. On the other hand when I develop with Javascript or C#, I'm easily 10 times more productive then I would be with either C or C++. This is a bit of apples and oranges comparison, but what I'm trying to say is that new low level languages don't bring huge gains in productivity.

by tasn 9mo ago

antirez 9mo ago

In Rust you can avoid "unsafe" when you use Rust like it was Go or Python. If you write low level code, that is where C is in theory replaceable only by Rust (and not by Go), then you find yourself in need of writing many unsafe sections. And to lower the amount of unsafe sections, you have to build unnatural abstractions, often, in order to group such unsafe sections into common patterns. Is is a tradeoff, not a silver bullet.

tasn 9mo ago
I think this framing is a bit backwards. Many C programs (and many parts of C programs) would benefit from being more like Go or Python as evident by your very own sds.c. Now, if what you're saying is that with super highly optimized sections of a codebase, or extremely specific circumstances (some kernel drivers) you'd need a bit of unsafe rust: then sure. Though all of a sudden you flipped the script, and the unsafe becomes the exception, not the rule; and you can keep those pieces of code contained. Similarly to how C programmers use inline assembly in some scenarios. Funny enough, this is similar to something that Rust did the opposite of C, and is much better for it: immutable by default (let mut vs. const in C) and non-nullable by default (and even being able to define something as non-null). Flipping the script so that GOOD is default and BAD is rare was a huge win. I definitely don't think Rust is a silver bullet, though I'd definitely say it's at least a silver alloy bullet. At least when it comes to the above topics.
burntsushi 9mo ago
Not necessarily at all. Go peruse the `regex` crate source code, including its dependencies. The biggest `unsafe` sections are probably for SIMD accelerated search. There's no "unnatural abstractions" there. Just a memmem-like interface. There's some `unsafe` for eliding bounds checks in the main DFA search loops. No unnatural abstractions there either. There's also some `unsafe` for some synchronization primitives for managing mutable scratch space to use during a search. A C library (e.g., PCRE2) makes the caller handle this. The `regex` crate does it for you. But not for unnatural reasons. To make using regexes simpler. There are lower level APIs that provide the control of C if you need it. That's pretty much it. All told, this is a teeny tiny fraction of the code in the `regex` crate (and all of its dependencies). Finally, a demonstration of C-like speed: https://github.com/BurntSushi/rebar?tab=readme-ov-file#summary-of-search-time-benchmarks https://github.com/BurntSushi/rebar?tab=readme-ov-file#summa... > Is is a tradeoff, not a silver bullet. Uncontroversial.
josephg 9mo ago
In my experience (several years of writing high performance rust code), there’s only really 2 instances where you need unsafe blocks: - C interop - Low level machine code (eg inline assembly) Most programs don’t need to do either of those things. I think you could directly port redis to entirely safe rust, and it would be just as fast. (Though there will need be unsafe code somewhere to wrap epoll). And even when you need a bit of unsafe, it’s usually a tiny minority of any given program. I used to think you needed unsafe for custom container types, but now I write custom container types in purely safe rust on top of Vec. The code is simpler, and easier to debug. And I’m shocked to find performance has mostly improved as a result.

pclmulqdq 9mo ago

I hear "people rarely use unsafe rust" quite a lot, but every time I see a project or library with C-like performance, there's a _lot_ of unsafe code in there. Treating bugs in unsafe code as not being bugs in rust code is kind of silly, also.

antirez 9mo ago
Exactly. You don't need much unsafe if you use Rust to replace a Python project, for instance. If there is lower level code, high performances needs, things change.
- kibwen 9mo ago
  For replacing a Python project with Rust, unsafe blocks will comprise 0% of your code. For replacing a C project with Rust, unsafe blocks will comprise about 5% of your code. The fact that the percentage is higher in the latter case doesn't change the fact that 95% of your codebase is just as safe as the Python project would be.
  - antirez 9mo ago
    A big amount of C code does not do anything unsafe as well, it calls other stuff, do loops, logic business, and so forth. It is also wrong to believe 100% of the C code is basically unsafe.
    - staticassertion 9mo ago
      You're swapping definitions of unsafe. Earlier you were referring to the `unsafe` keyword. Now you're using `unsafe` to refer to a property of code. This makes it easy to say things like "It is also wrong to believe 100% of the C code is basically unsafe" but you're just swapping definitions partway through the conversation.
      - broken-kebab 9mo ago
        What I see is that antirez claims that absence of "safe" (as syntax) in C lang doesn't automatically mean that all of C code is unsafe (as property). There's no swapping of definitions as I see it.

pjdesno 9mo ago

> was in unsafe code, and related to interop with C 1) "interop with C" is part of the fundamental requirements specification for any code running in the Linux kernel. If Rust can't handle that safely (not Rust "safe", but safely), it isn't appropriate for the job. 2) I believe the problem was related to the fact that Rust can't implement a doubly-linked list in safe code. This is a fundamental limitation, and again is an issue when the fundamental requirement for the task is to interface to data structures implemented as doubly-linked lists. No matter how good a language is, if it doesn't have support for floating point types, it's not a good language for implementing math libraries. For most applications, the inability to safely express doubly-linked lists and difficulty in interfacing with C aren't fundamental problems - just don't use doubly-linked lists or interface with C code. (well, you still have to call system libraries, but these are slow-moving APIs that can be wrapped by Rust experts) For this particular example, however, C interop and doubly-linked lists are fundamental parts of the problem to be solved by the code.

kibwen 9mo ago
> If Rust can't handle that safely (not Rust "safe", but safely), it isn't appropriate for the job. Rust is no less safe at C interop than using C directly.
- hirvi74 9mo ago
  (Not the user you were replying to) If Rust is no less safe than C in such a regard, then what benefit is Rust providing that C could not? I am genuinely curious because OS development is not my forte. I assume the justification to implement Rust must be contingent on more than Rust just being 'newer = better', right?
  - gf000 9mo ago
    It's not less safe in C interop. It is significantly safer at everything else.
- smj-edison 9mo ago
  As long as you keep C pointers as pointers. The mutable aliasing rules can bite you though.

nec4b 9mo ago

>> I guess that's at least in part because of the difficulty of building safe, fast and highly-concurrent C applications (please correct me if I'm wrong). You wrote that question in a browser mostly written in C++ language, running on an OS most likely written in C language.

gf000 9mo ago
Just because the pyramids exist, it means they were easy to build? OS and browser development are seriously hard and took countless expert man hours.
- nec4b 9mo ago
  OS can be actually pretty simple to make. Sometimes it's a part of a CS curriculum to make one. If it were so much easier to do it in other languages (e.g. in Rust), don't you think we would already be using them?
  - gf000 9mo ago
    https://github.com/flosse/rust-os-comparison https://github.com/flosse/rust-os-comparison Writing a toy one? Sure. Writing a real one? Who's gonna write all the drivers and the myriad other things? And the claim was not that it's "so much easier", but that it is so much easier to write it in a secure way. Which claim is true. But it's still a complex and hard program. (And don't even get started on browsers, it's no accident that even Microsoft dropped maintaining their own browser).
    - nec4b 9mo ago
      The toy one can still be as highly concurrent as the the real one. The amount of drivers written for it doesn't matter. The point is if it were much easier, then they would overtake existing ones easily, just by adding features and iterating so much faster and that is clearly not the case. >>difficulty of building safe, fast and highly-concurrent C This was the original claim. The answer is, there is a tonne of C code out there that is safe, fast and concurrent. Isn't it logical? We have been using C for the last 50 years to build stuff with it and there is a lot of it. There doesn't seem to be a big jump in productivity with the newer generation of low level languages, even though they have many improvements over C. This is anecdotal, I used to do a lot of low level C and C++ development. And C++ is a much bigger language then C. And honestly I don't think I was ever more productive with it. Maybe the code looked more organized and extendable, but it took the same or larger amount of time to write it. On the other hand when I develop with Javascript or C#, I'm easily 10 times more productive then I would be with either C or C++. This is a bit of apples and oranges comparison, but what I'm trying to say is that new low level languages don't bring huge gains in productivity.