8 ms·

Rust inadequate for text compression codecs?

• ydjje 2y ago
  [dead]
• jgord 2y ago
  The first section on rust protections static vs dynamic is thought provoking and informative... tbf, author admits he needed to dive into ASM to work around the C compilers sensible optimisation defaults. tbf to the author, he does explain that the domain of codecs implementation is niche. great read ... in the best traditions of the Mike Abrash articles of yesteryear.
  • jgord 2y ago
    Glad to be proven wrong in my subconscious assumptions about authors age and sex .. I assumed some bearded 50yo polyglot who came up in the 80s writing TSRs in A86. About page does not disappoint.
• JJOmeo 2y ago
  I had a similar experience writing image decompressors. There wasn't enough to internally manage for rust to provide any benefit, and the gritty fast algorithmic code is just simpler to do with C++. C wasn't going to get me the flexibility and generic patterns I was going for.
  • jgord 2y ago
    what %ge of C++ features did you use for that ?
• hlieberman 2y ago
  This kind of hyper-specific need (codecs) is probably better served by a specialist language, like Whuffs (https://github.com/google/wuffs https://github.com/google/wuffs). You don't need, or want, the level of expressiveness that comes with something like Rust, but on the other hand, it's a compact enough problem set that you're willing to spend extra development work to eke out every bit of speed.
  • adastra22 2y ago
    Thanks for linking to Wuffs! Hadn’t seen it before.
  • vlovich123 2y ago
    Was going to make the same comment. And in the article they note that they have to go own to assembly to do this anyway and I find the inline assembly with Rust to be more ergonomic and safer than the assembly facilities you get with C++ (not that I’ve done that much assembly to be fair).
    • jchw 2y ago
      I agree, Rust seems a lot better for inline assembly. For C and C++, there's too much variability between compilers for what you can actually use with inline assembly. Right now, I'm desperately waiting for #[naked] to be stable[1][2]. It's not always necessary, but it's incredibly useful for a lot of low level fuckery. Until it's there, there aren't many good alternatives; you could have a custom build script that calls out to an external assembler, I suppose. [1]: https://github.com/rust-lang/reference/pull/1689 https://github.com/rust-lang/reference/pull/1689 [2]: https://github.com/rust-lang/rust/pull/134213 https://github.com/rust-lang/rust/pull/134213
      • wakawaka28 2y ago
        Can you really complain about differences between compilers, when Rust basically has ONE compiler? You would most likely have differences if there were more Rust compilers out there. I mean, at least with C++ there are choices, and you can choose to just use one compiler like clang if the differences bother you.
        • dmoy 2y ago
          I read it as > I agree, Rust seems a lot better for inline assembly [because there's basically only one compiler]. [Compared to] C and C++, [where] there's too much variability between compilers for what you can actually use with inline assembly
• maartenscholl 2y ago
  The author claims to be an expert in C++ but begins the article with a detour that incorrectly states that static and dynamic dispatch are necessarily orthogonal in C++. In reality you can do both within the same hierarchy using CRTP combined with a virtual base method. Compile-time inlining eliminates the overhead associated with the virtual call, so you can match Rusts flexibility with the same performance.
  • wakawaka28 2y ago
    The cases where you can inline a virtual method seem narrow. So narrow in fact, that if it is possible then you should probably only be doing CRTP or simple static dispatch. See: https://stackoverflow.com/questions/733737/are-inline-virtual-functions-really-a-non-sense https://stackoverflow.com/questions/733737/are-inline-virtua... If you try to use CRTP + virtual on polymorphic types then one has to wonder if it will work as intended for both use cases (when used as a static object or a polymorphic one). I'm not the absolute most expert C++ programmer, but I'm no noob. The idea of deliberately introducing dynamic polymorphism only to try to optimize it out seems like a bad idea. It's unnecessarily complicated and confusing. If you want to go fast just use CRTP straight up and forget all about dynamic dispatch and potential cute optimizations. Edit: I think this explains my objection: https://www.codeproject.com/Tips/537606/Cplusplus-Prefer-Curiously-Recurring-Template-Patt https://www.codeproject.com/Tips/537606/Cplusplus-Prefer-Cur... (I found that in the StackOverflow post I linked to above.)
  • klysm 2y ago
    I feel like expert in C++ is a wide range. There are corners of C++ that probably 5 people know about total
• patrick451 2y ago
  > We have observed a ~13.3% slow-down, which is significant enough to warrant a second thought on whether the bounds checks were a good idea in the first place. Wow. I will think twice the next time I reach for .at().
  • DevelopingElk 2y ago
    Across an application the penalty is normally 1-5%. Most business code benefits from the increased safety. Parsers are an exception, but the large attack surface sometimes makes it a good idea there too.
    • TinkersW 2y ago
      If you are quoting the numbers google put out, recall that those numbers were after they profiled and manually disabled bounds checks in cases where it caused significant slowdown.. so in other words the numbers were garbage.
    • int_19h 2y ago
      The bigger problem is that std::vector::at() is kinda useless in any case since it's very rare in idiomatic C++ to be indexing into a vector using an integer. You're much more likely to be using an iterator, since that's what you'll get from standard algorithms like std::find() etc. And there's no API for "checked iterators" that is equivalent to at(). For a debug build, your STL implementation might use checked iterators, but that's a quality of implementation matter that you can't rely on. MSVC does have a documented checked iterator facility that can be enabled even in release builds: https://learn.microsoft.com/en-us/cpp/standard-library/checked-iterators https://learn.microsoft.com/en-us/cpp/standard-library/check....
  • imtringued 2y ago
    The antivirus packages I've been forced to use probably wasted more time than those 13% will ever save.
• unrealhoang 2y ago
  I think tfa hasn’t considered the advantages of enforcing the correct usage of the codecs library. Most of the time memory safety issues are caused not in the codecs code itself but the user code using the codecs incorrectly. Unsafe Rust is still improving on its ergonomics (handling uninitialized memory) but the current capacity should be enough to implement anything.
  • jchw 2y ago
    To be fair, you definitely don't need a Rust implementation to get that advantage: a simple wrapper will do the trick.
• wrs 2y ago
  Not to detract from the good points made here, but it does confuse me to see "panic!()’s are hiding even in the most innocuous places like unwrap() and expect() calls". They're not innocuous at all -- they should stand out just as much as writing panic!(). Panicking is literally unwrap and expect's only job.
• pdpi 2y ago
  > Moreover, panic!()’s are hiding even in the most innocious places like unwrap() and expect() calls unwrap and expect aren't "innocuous places where panic is hiding". The whole point of unwrap and expects is that you're effectively saying "Hitting a None/Err here is impossible, so seeing them should be treated as a correctness error".
  • zamalek 2y ago
    Furthermore > a panic!() or an abort() are much more annoying to catch and handle. By design. If you're hoping to catch a panic, then you're doing it woefully wrong. Catching panics is correct only in extremely niche scenarios. Panic-free Rust code can sometimes be challenging, if a dependency incorrectly asserts an unwrap/expect, but the Rust community tends to hold itself to a higher standard than that. Other things, > Most of it is the low-level nitty-gritty, which Rust is not particularly good at. I don't feel like this was motivated anywhere, just asserted at the end. My opinion is that Rust gives you more tools to make invalid states unrepresentable - this can drastically lower cognitive burden when working on the code. > Bounds checks are rarely performed in the compiletime and we have to pay their price at the runtime. This is a significant performance hit for coders. Rust is extremely good at eliding bounds checks. If you attempt to write C++ in Rust (e.g. a moving pointer/slice) then you're probably going to trip these heuristics up, idiomatic Rust (heavy iterator usage) generally results in fewer bounds checks. > The ownership model is not particularly useful to coders. Maybe not useful, but certainly unlikely to be a hindrance. The ownership model generally maps extremely well to the CS101 concept of Input -> Processing -> Output, which coders are (and UIs aren't, if you've ever wondered why its still mostly an unsolved problem in Rust). You can do raw assembly in Rust too.
    • SAI_Peregrinus 2y ago
      > Catching panics is correct only in extremely niche scenarios. Catching panics is correct when the panic would cause unwinding across a FFI boundary. I can't think of anywhere else where I wouldn't be very suspicious seeing a catch_unwind().
      • 2y ago
• devit 2y ago
  The article is wrong on the claim that Rust panic are harder to catch than C++ exceptions: as long as you don't configure panic=abort you can catch them easily with catch_panic and they are generally implemented using the same runtime mechanism (i.e. Rust panics usually effectively are C++ exceptions for most purposes).
  • 01HNNWZ0MV43FF 2y ago
    Hm, it just feels wrong though. Panics feel bigger than exceptions
    • int_19h 2y ago
      That's a cultural thing (and a good one, too). The one big difference tho is that in Rust, the end user of the library - i.e. the person compiling the binary of which this library is a part - can decide at that point whether panics unwind like C++ exceptions, or just abort immediately. Conversely, this means that the library should never assume that it can catch panics, even its own internal ones, because it may be compiled with panic=abort. So it's kinda like C++ exceptions, but libraries can only throw, never catch.
• waltercool 2y ago
  [dead]
• jclulow 2y ago
  > But such behaviour is still unacceptable from a library perspective: a library should never, ever call abort or otherwise terminate the program. It's true that libraries should not abort for a regular, foreseeable error. I fundamentally disagree that they should never abort. If an invariant (something we believe absolutely must be true) is violated, the only sensible thing to do is abort the process. For example, if you set a pointer to NULL and then half an hour later it's time to store a value in that pointer but somehow it's not NULL anymore: clearly something has gone terribly wrong, either in your logic, or potentially in totally unrelated code in some other library which has scribbled on the heap. If execution is allowed to continue, the resultant behaviour is totally undefined: you might corrupt data, or you might allow a permissions check to pass that should have failed, etc. We can't always detect when something has gone wrong, but when we do, it's irresponsible to drive on or try to return an error.
  • anon-3988 2y ago
    That behavior is up to the user. The library should only report the error.
    • burntsushi 2y ago
      This is terrible advice, and people suggesting it should put their money where their mouth it and show real world examples. Which libraries in widespread use know how to detect all of their possible bugs due to invariant violations and report them as explicit error values? I'd love to see the API docs for them. "This error value is impossible and this library will never return it. If it does, then there is a bug in the library. Since there are no known bugs related to this invariant violation, this cannot happen." Nevermind the fact that you're now potentially introducing an error channel into operations that should never actually error. That potentially has performance implications. Nevermind the fact that now your implementation details (internal runtime invariants are implementation details) have now leaked out into your API. Need a new internal runtime invariant? Now you need to add that fact to the API. Need to remove an invariant? Ah well, you still need to leave that possible error value in your API to avoid breaking your users.
• haberman 2y ago
  > But such behaviour is still unacceptable from a library perspective: a library should never, ever call abort or otherwise terminate the program. I tend to agree with this, which is why I was happy to discover that No-Panic Rust does appear to be practical: https://blog.reverberate.org/2025/02/03/no-panic-rust.html https://blog.reverberate.org/2025/02/03/no-panic-rust.html
• davidhyde 2y ago
  When writing codecs in Rust I found the Iterator trait (and all its provided methods) to be very useful for avoiding panics relating to out of bounds indexing into arrays. It also tends to generate very efficient code since bounds checks are done on the order of once per loop rather than once per loop iteration. And then there is the powerful itertools crate that gives you even more capabilities. Highly recommended!
• koakuma-chan 2y ago
  Ariel listen to me: compile-time memory safety and robust type system? It's a mess. Programming in C is better than anything they got over there. The syntax might seem much sweeter where reference lifetimes play, but frills like the borrow checker will only get in the way.
  • burjui 2y ago
    ... in the way of writing code with undefined behaviour. "You don't tell me how to live! Maybe I want to shoot myself in the foot sometimes! I already shot myself in the head, and it was awesome, instantly made me an old fart that refuses to learn. Ignorance is bliss!"
• oneup2 2y ago
  I feel like this is kind of obvious. Plenty of people write C or C++ and then optimize some low-level function using hand coded assembly. Even the author seemed to suggest they couldn't get their C compiler to do what they wanted so they bailed and hand wrote assembly. How is this any different from rust? I'd expect the same for rust. You make a safe higher level API. It validates all the inputs and then calls into the private "unsafe parts" implemented in C or assembly. Hopefully you've put enough validation in the high level API so that it will be unlikely to call the low-level API in a way that breaks.
• aldanor 2y ago
  The author seems clearly inexperienced with Rust (see the paragraph about unwraps and panics) and unfamiliar with common paradigms and ways of doing things in low-level Rust; it's unclear why this post is gaining so much attention on hn. Rust is no worse than C/C++ in writing all sorts of low level codecs, be it with or without using unsafe, with performance matching that of C/C++ and readability/safety usually being much better. // personal experience: participated in Rust core lib float parser implementation, part of which involved porting it directly from c++; implemented safe rust qoi image encoder, bpe text encoder, etc; in all cases they turned out to be the fastest existent out there; in all cases, though, good prior experience with low-level rust was required, especially if doing it without unsafe and trying to avoid most bound checks, so there's that.
• pjmlp 2y ago
  While it won't fix all the security issues that C++ might have, given its C subset, the provided examples can be improved by enabling the hardened runtime configuration, built-in compiler analysis, which also validate use before initialisation (finally fixed in C++26). Rewriting is not always an option, so improving one's code security is also something to be aware of. Having said this and since the thread is about text handling, Microsoft has rewriten DWriteCore in Rust.
• zanellato19 2y ago
  >In my mind, a more advanced memory-safe system will likely overtake Rust and become adequate for the task of implementing codecs and other low level tools (perhaps integrated with a theorem prover and a proof assistant that lets the programmer clarify the assertions, preconditions, postconditions and invariants). However, I will stick with C for now. Ultimately, a lot of this boils down to personal preference, so try to not harass strangers online for their choices. I seen this in basically every "I prefer C for this use case" blog post and its always unconvincing, because C is much worse for safety then Rust, so I always think the author would find other problems with these next tools because its just goal posts moving.