4 ms·

It's... basically written in C. I'm no expert on zlib/deflate or related algorithms, but digging around https://github.com/trifectatechfoundation/zlib-rs/ http

by ajross 2y ago

It's... basically written in C. I'm no expert on zlib/deflate or related algorithms, but digging around https://github.com/trifectatechfoundation/zlib-rs/ https://github.com/trifectatechfoundation/zlib-rs/ almost every block with meaningful logic is marked unsafe. There's raw allocation management, raw slicing of arrays, etc... This code looks and smells like C, and very much not like rust. I don't know that this is a direct transcription of the C code, but if you were to try something like that this is sort of what it would look like. I think there's lots of value in wrapping a raw/unsafe implementation with a rust API, but that's not quite what most people think of when writing code "in rust".

hermanradtke 2y ago
> basically written in C Unsafe Rust still has to conform to many of Rust’s rules. It is meaningfully different than C.
- est31 2y ago
  It has also way less tooling available than C to analyze its safety.
  - nindalf 2y ago
    The number of tools matters less than the quality of the tools. Rust’s inherent guarantees + miri + software verification tools mean that in practice Rust code, even with unsafe, ends up being higher quality.
  - wyager 2y ago
    Miri is better than any C tool I'm aware of for runtime UB detection.
    - est31 2y ago
      Miri is the closest to a UB specification for Rust that there is, coming in the form of a tool so you can run it. It's really cool but Valgrind, which is a C tool that also supports Rust, also supports Rust code that calls to C and that does I/O, both pretty common things for programs to do.
  - vlovich123 2y ago
    The things I’ve seen broadly adopted in the industry (i.e. sanitizers) are equally available in Rust. & Rust’s testing infrastructure is standardized so tests are actually common to see in every library.
- ajross 2y ago
  Are there examples you're thinking about? The only good ones I can think of are bits about undefined behavior semantics, which frankly are very well covered in modern C code via tools like ubsan, etc...
  - sedatk 2y ago
    This comment summarizes the difference of unsafe Rust quite well. Basically, mostly safe Rust, but with few exceptions, fewer than one would imagine: https://news.ycombinator.com/item?id=43382176 https://news.ycombinator.com/item?id=43382176
  - steveklabnik 2y ago
    They're just fundamentally different languages. There's semantics that exist in all four of these quadrants: * defined in C, undefined in Rust * undefined in C, undefined in Rust * defined in Rust, undefined in C * defined in Rust, defined in C
    - ajross 2y ago
      That doesn't seem responsive. The question wasn't whether Rust and C are literally the same language ("duh", as it were), it was effectively "are there meaningful safety features provided to the unsafe zlib-rs code in question in that aren't already available in C toolchains/ecosystems?" And there really aren't. The abbreviated/limited safety environment being exploited by this non-idiomatic Rust code seems to me to be basically isomorphic to the way you'd solve the problem in C.
      - steveklabnik 2y ago
        > it was effectively "are there meaningful safety features provided to the unsafe zlib-rs code in question in that aren't already available in C toolchains/ecosystems?" Ah, so that was like, not in your comment, but in a parent. > And there really aren't. I mean, not all of the code is unsafe. From a cursory glance, there's surely way more here than I see in most Rust packages, but that doesn't mean that you get no advantages. I picked a random file, and chose some random code out of it, and see this: pub fn copy<'a>( dest: &mut MaybeUninit<DeflateStream<'a>>, source: &mut DeflateStream<'a>, ) -> ReturnCode { // SAFETY: source and dest are both mutable references, so guaranteed not to overlap. // dest being a reference to maybe uninitialized memory makes a copy of 1 DeflateStream valid. unsafe { core::ptr::copy_nonoverlapping(source, dest.as_mut_ptr(), 1); } The semantics of safe code, `&mut T`, provide the justification for why the unsafe code is okay. Heck, this code wouldn't even be legal in C, thanks to strict aliasing. (Well, I guess you could argue that in C code they'd be of the same type, since you don't have "might be uninitialized" in C's typesystem, but again, this is an invariant encoded in the type system that C can't do, so it's not possible to express in C for that reason either.)
xxs 2y ago
I mentioned in under another comment - and while I consider myself versed enough in deflate - comparing the library to zlib-ng is quite weird as the latter is generally hand written assembly. In order to beat it'd take some oddity in the test itself
oneshtein 2y ago
Cannot understand your complain. It written in Rust, but for you it looks like C. So what?
- Alifatisk 2y ago
  So, it is basically like it was written in C.
  - oneshtein 2y ago
    Yes, it's possible to write in Rust like in C. This code is example of that. You can even use automatic code converter to convert C into Rust.
- ajross 2y ago
  It doesn't exploit (and in fact deliberately evades) Rust's signature memory safety features. The impression from the headline is "Rust is as fast as C now!", but in fact the subset of the language that has been shown to be as fast as C is the subset that is basically isomorphic to C. The impression a naive reader might take is that idiomatic/safe/best-practices Rust has now closed the performance gap. But clearly that's not happening here.
  - sedatk 2y ago
    Rust's many memory safety features (including the borrow checker) are still enabled in unsafe Rust blocks. For more information: https://news.ycombinator.com/item?id=43382176 https://news.ycombinator.com/item?id=43382176
    - ajross 2y ago
      But again, not exploited by the code in question. This isn't using the Rust runtime heap, it's doing its own thing with raw pointers/indexing, and even seems to have its own allocator.
      - 2y ago
johnisgood 2y ago
It does actually seem like what a C -> Rust transpiler would spit out.
gf000 2y ago
C is not assembly, nor is it portable assembly at all in this century, so your phrasing is very off. C code will go through a huge amounts of transformations by the compiler, and unless you are a compiler expert you will have no idea how the resulting code looks. It's not targeting the PDP-11 anymore.
solidsnack9000 2y ago
I'm not sure why people say this about certain languages (it is sometimes said about Haskell, as well). The code has a C style to it, but that doesn't mean it wasn't actually written in Rust -- Rust deliberately has features to support writing this kind of code, in concert with safer, stricter code. Imagine if we applied this standard to C code. "Zlib-NG is basically written in assembler, not C..." https://github.com/zlib-ng/zlib-ng/blob/50e9ca06e29867a9014ea187eb0f46fff5125ed2/arch/arm/acle_intrins.h#L22 https://github.com/zlib-ng/zlib-ng/blob/50e9ca06e29867a9014e...
- ajross 2y ago
  > Imagine if we applied this standard to C code. "Zlib-NG is basically written in assembler, not C..." We absolutely should, if someone claimed/implied-via-headline that naive C was natively as fast as hand-tuned assembly! This kind of context matters. FWIW: I'm not talking about the assembly in zlib-rs, I was specifically limiting my analysis to the rust layers doing memory organization, etc... Discussing Rust is just exhausting. It's one digression after another, like the community can't just take a reasonable point ("zlib-rs isn't a good example of idiomatic rust performance") on its face.
  - solidsnack9000 2y ago
    I'm not sure anyone really believes `zlib-rs` is a good example of idiomatic Rust performance, though Maybe the reason I think that is because I've written Rust for a variety of purposes (web application, database bindings, high performance parser) so I account for the "register" of Rust that is appropriate without thinking about it. https://en.wikipedia.org/wiki/Register_(sociolinguistics) https://en.wikipedia.org/wiki/Register_(sociolinguistics) It might be that a simple description like the headline leads some people to believe they could write Rust the easy way and get code that's as fast as writing "Rust the hard way". However, that is different than what you earlier said -- "It's... basically written in C.". I have actually written Rust programs where some parts were literally written in C and linked in -- in order to build functioning plugins -- and there is a world of difference with that. Regarding Discussing Rust is just exhausting. It's one digression after another, like the community can't just take a reasonable point ("zlib-rs isn't a good example of idiomatic rust performance") on its face. I'm just not sure what to say to this. What do you expect from me, here?

by ajross 2y ago

hermanradtke 2y ago

> basically written in C Unsafe Rust still has to conform to many of Rust’s rules. It is meaningfully different than C.

est31 2y ago
It has also way less tooling available than C to analyze its safety.
- nindalf 2y ago
  The number of tools matters less than the quality of the tools. Rust’s inherent guarantees + miri + software verification tools mean that in practice Rust code, even with unsafe, ends up being higher quality.
- wyager 2y ago
  Miri is better than any C tool I'm aware of for runtime UB detection.
  - est31 2y ago
    Miri is the closest to a UB specification for Rust that there is, coming in the form of a tool so you can run it. It's really cool but Valgrind, which is a C tool that also supports Rust, also supports Rust code that calls to C and that does I/O, both pretty common things for programs to do.
- vlovich123 2y ago
  The things I’ve seen broadly adopted in the industry (i.e. sanitizers) are equally available in Rust. & Rust’s testing infrastructure is standardized so tests are actually common to see in every library.
ajross 2y ago
Are there examples you're thinking about? The only good ones I can think of are bits about undefined behavior semantics, which frankly are very well covered in modern C code via tools like ubsan, etc...
- sedatk 2y ago
  This comment summarizes the difference of unsafe Rust quite well. Basically, mostly safe Rust, but with few exceptions, fewer than one would imagine: https://news.ycombinator.com/item?id=43382176 https://news.ycombinator.com/item?id=43382176
- steveklabnik 2y ago
  They're just fundamentally different languages. There's semantics that exist in all four of these quadrants: * defined in C, undefined in Rust * undefined in C, undefined in Rust * defined in Rust, undefined in C * defined in Rust, defined in C
  - ajross 2y ago
    That doesn't seem responsive. The question wasn't whether Rust and C are literally the same language ("duh", as it were), it was effectively "are there meaningful safety features provided to the unsafe zlib-rs code in question in that aren't already available in C toolchains/ecosystems?" And there really aren't. The abbreviated/limited safety environment being exploited by this non-idiomatic Rust code seems to me to be basically isomorphic to the way you'd solve the problem in C.
    - steveklabnik 2y ago
      > it was effectively "are there meaningful safety features provided to the unsafe zlib-rs code in question in that aren't already available in C toolchains/ecosystems?" Ah, so that was like, not in your comment, but in a parent. > And there really aren't. I mean, not all of the code is unsafe. From a cursory glance, there's surely way more here than I see in most Rust packages, but that doesn't mean that you get no advantages. I picked a random file, and chose some random code out of it, and see this: pub fn copy<'a>( dest: &mut MaybeUninit<DeflateStream<'a>>, source: &mut DeflateStream<'a>, ) -> ReturnCode { // SAFETY: source and dest are both mutable references, so guaranteed not to overlap. // dest being a reference to maybe uninitialized memory makes a copy of 1 DeflateStream valid. unsafe { core::ptr::copy_nonoverlapping(source, dest.as_mut_ptr(), 1); } The semantics of safe code, `&mut T`, provide the justification for why the unsafe code is okay. Heck, this code wouldn't even be legal in C, thanks to strict aliasing. (Well, I guess you could argue that in C code they'd be of the same type, since you don't have "might be uninitialized" in C's typesystem, but again, this is an invariant encoded in the type system that C can't do, so it's not possible to express in C for that reason either.)

xxs 2y ago

I mentioned in under another comment - and while I consider myself versed enough in deflate - comparing the library to zlib-ng is quite weird as the latter is generally hand written assembly. In order to beat it'd take some oddity in the test itself

oneshtein 2y ago

Cannot understand your complain. It written in Rust, but for you it looks like C. So what?

Alifatisk 2y ago
So, it is basically like it was written in C.
- oneshtein 2y ago
  Yes, it's possible to write in Rust like in C. This code is example of that. You can even use automatic code converter to convert C into Rust.
ajross 2y ago
It doesn't exploit (and in fact deliberately evades) Rust's signature memory safety features. The impression from the headline is "Rust is as fast as C now!", but in fact the subset of the language that has been shown to be as fast as C is the subset that is basically isomorphic to C. The impression a naive reader might take is that idiomatic/safe/best-practices Rust has now closed the performance gap. But clearly that's not happening here.
- sedatk 2y ago
  Rust's many memory safety features (including the borrow checker) are still enabled in unsafe Rust blocks. For more information: https://news.ycombinator.com/item?id=43382176 https://news.ycombinator.com/item?id=43382176
  - ajross 2y ago
    But again, not exploited by the code in question. This isn't using the Rust runtime heap, it's doing its own thing with raw pointers/indexing, and even seems to have its own allocator.
    - 2y ago

johnisgood 2y ago

It does actually seem like what a C -> Rust transpiler would spit out.

gf000 2y ago

C is not assembly, nor is it portable assembly at all in this century, so your phrasing is very off. C code will go through a huge amounts of transformations by the compiler, and unless you are a compiler expert you will have no idea how the resulting code looks. It's not targeting the PDP-11 anymore.

solidsnack9000 2y ago

I'm not sure why people say this about certain languages (it is sometimes said about Haskell, as well). The code has a C style to it, but that doesn't mean it wasn't actually written in Rust -- Rust deliberately has features to support writing this kind of code, in concert with safer, stricter code. Imagine if we applied this standard to C code. "Zlib-NG is basically written in assembler, not C..." https://github.com/zlib-ng/zlib-ng/blob/50e9ca06e29867a9014ea187eb0f46fff5125ed2/arch/arm/acle_intrins.h#L22 https://github.com/zlib-ng/zlib-ng/blob/50e9ca06e29867a9014e...

ajross 2y ago
> Imagine if we applied this standard to C code. "Zlib-NG is basically written in assembler, not C..." We absolutely should, if someone claimed/implied-via-headline that naive C was natively as fast as hand-tuned assembly! This kind of context matters. FWIW: I'm not talking about the assembly in zlib-rs, I was specifically limiting my analysis to the rust layers doing memory organization, etc... Discussing Rust is just exhausting. It's one digression after another, like the community can't just take a reasonable point ("zlib-rs isn't a good example of idiomatic rust performance") on its face.
- solidsnack9000 2y ago
  I'm not sure anyone really believes `zlib-rs` is a good example of idiomatic Rust performance, though Maybe the reason I think that is because I've written Rust for a variety of purposes (web application, database bindings, high performance parser) so I account for the "register" of Rust that is appropriate without thinking about it. https://en.wikipedia.org/wiki/Register_(sociolinguistics) https://en.wikipedia.org/wiki/Register_(sociolinguistics) It might be that a simple description like the headline leads some people to believe they could write Rust the easy way and get code that's as fast as writing "Rust the hard way". However, that is different than what you earlier said -- "It's... basically written in C.". I have actually written Rust programs where some parts were literally written in C and linked in -- in order to build functioning plugins -- and there is a world of difference with that. Regarding Discussing Rust is just exhausting. It's one digression after another, like the community can't just take a reasonable point ("zlib-rs isn't a good example of idiomatic rust performance") on its face. I'm just not sure what to say to this. What do you expect from me, here?