7 ms·

Whilst unrelated to the article, my complaint about codecs in Rust is that they seem to be slow. Whilst the reason for this might be the immaturity of the libra

Whilst unrelated to the article, my complaint about codecs in Rust is that they seem to be slow. Whilst the reason for this might be the immaturity of the libraries that I've used, but they've always been slower than their C counter-parts. The native JPEG decoder spins up 4 threads to decode the same amount of frames at 3x the time as the libjpeg-turbo does. There's a similar story for the FLAC decoder. I don't think that any of this has anything to do with the language itself, it's just that it takes time for things to mature. As for the issues outlined in the article - the language is different enough from C and C++ that one just has to accept the fact that you cannot write idiomatic C and C++ in Rust and expect it to be comfortable, performant or safe. However, with Rust, I'd say, that one can achieve 99% of what one can achieve in C today. The only thing that Rust is missing currently is the ability to arbitrarily jump around the call stack due to the way destructors are implemented, but there's a way to mitigate this and it's being worked as far as I was aware.

pornel 9y ago
Decoding in libjpeg-turbo is mostly implemented in ASM. Rust, like C, is slower than hand-optimized SIMD ASM.
- mrec 9y ago
  If anyone's interested, this is the (epic!) discussion thread for "Getting explicit SIMD on stable Rust": https://internals.rust-lang.org/t/getting-explicit-simd-on-stable-rust/4380/334 https://internals.rust-lang.org/t/getting-explicit-simd-on-s... It still wouldn't compete with really good hand-tuned asm, but it might help reduce the perf/usability tradeoff a bit.
  - phkahler 9y ago
    I'd just like to see 2,3, and 4 element vectors as a first class citizen in C, C++, or Rust. These are incredibly common for so many things it's hard for me to understand this omission. I want to be able to pass them by value and as return values from function. I want to do operations like a=b+c with vectors without creating classes or overloading operators. For a lot of people the SIMD instructions are about parallel computation, but for me they represent vector primitives.
    - Zelizz 9y ago
      Look into the Clang vector extensions. They're very similar to GLSL/OpenCL vectors. They implement some very basic operator overloading too, so they're much less annoying that calling a function for everything. I made a simple linear algebra library using them, if you want to see an example: https://github.com/GavinHigham/glla https://github.com/GavinHigham/glla
    - mrec 9y ago
      What would baking them into the language add that a library solution couldn't? I'm not keeping track of Rust very closely these days, but it supports arithmetic operator overloading and the Copy trait. What's missing? I agree that this needs to be standardized for vector-ey crates to be able to talk to each other seamlessly. Otherwise we'll end up with a rerun of the C++ strings fiasco, with char* and wchar_t* and std::string and std::wstring and BSTR and _bstr_t and CString and CComBSTR and QString and GString and...
      - phkahler 9y ago
        OOP often involves overhead. If someone defined a really clean vector type/object in a library that let me write expressions the natural way, that could be added to the language I suppose. And that's what I want.
        dbaupp 9y ago
        Rust, C and C++ all possess ways (it's the even default/only way in C and Rust, and almost so in C++) to write types like this that don't involve the typical pointer-soup/dynamic-call overhead of typical OOP. People can and do write vector in libraries right now, usually using existing compiler support that gives them guarantees about SIMD.
        StillBored 9y ago
        What I find odd about your reply, is that you downgrade C++, yet its really the only language that has sufficient operator overloading to do what the poster was requesting (transparent support without OO overhead). AKA, its completely possible, and there are quite a number of C++ libraries that make vectors look like native types complete with long lists of global operator overloads for interaction with other base types. Generally these libraries are just thin wrappers around inline assembly or intel intrinsics for SSE/AVX and generally don't bring any form of OO syntax into the picture. That said, even with C++ the libraries tend to fall down a bit when it comes to individual element operation/masking because the closest method is generally using array syntax for the elements which limits the operations to a single element in the vector at a time. Which means you end up with OO method calling syntax for element masking (or creative solutions with operator() )
bigdubs 9y ago
~fin~
- knieveltech 9y ago
  While hijacking a thread for idle pedantry is something of a sport around here your comment is unusually pointless for the form. Go eat something, you'll feel better.
- spatulon 9y ago
  It's still widely used and not considered archaic in British and Australian English.
  - bigdubs 9y ago
    https://www.merriam-webster.com/dictionary/whilst https://www.merriam-webster.com/dictionary/whilst The definition is singularly, "while", with no extra connotation, other than it is as you said chiefly British. https://trends.google.com/trends/explore?q=while,whilst https://trends.google.com/trends/explore?q=while,whilst "While" is, not surprisingly, much more widely used, "whilst" being more affected.
daniel_rh 9y ago
I'm the author of https://github.com/dropbox/rust-brotli https://github.com/dropbox/rust-brotli and I certainly ran into the issues mentioned in the article, and the initial version of my brotli decoder and encoder were each almost 10x slower. I also worked around each and every one of them, and the result is something that performs at 80-90% of the speed of the original on average, and some files compress faster with rust than with the original brotli codec. allocator: I ran into the same situation and abstracted it with a generic allocator: https://github.com/dropbox/rust-alloc-no-stdlib https://github.com/dropbox/rust-alloc-no-stdlib it's an ugly solution, but it works and can allow for significant perf improvements down the line by bundling all same types together benchmarks: I simply made a test that printed the time primitive types: it's easy to write generic functions that do this macro system: I found to be amazing, but I never used it to compact data types borrow checker: split_at_mut was super helpful...also core::cmp::replace was useful to taking away pieces of a structure and manipulating them, then putting them back Though it wasn't all roses: to try to get SSE vectorization I had to convert this 6 line short string matching function https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d980e0a50795752c45dd3359e/src/enc/static_dict.rs#L122 https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d... into this multi hundred line monster with macros and so forth https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d980e0a50795752c45dd3359e/src/enc/static_dict.rs#L131 https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d... but in the end it was as fast as hand-tuned intrinsics in C I also found myself scared by dependencies, including onto the std library, so I tried to get everything to remain within the core library. This should allow something as low-level as a codec to be used in kernel space or in another place where a custom allocator is needed. I also found that "rewriting in safe rust from C" was made easy by corrode https://github.com/jameysharp/corrode https://github.com/jameysharp/corrode since C and rust can be interface compatible, it's easy to go one file at a time and turn it into working rust, then safe rust.
jeffdavis 9y ago
How can they allow jumping around the call stack while also destructing/freeing objects?

pornel 9y ago

Decoding in libjpeg-turbo is mostly implemented in ASM. Rust, like C, is slower than hand-optimized SIMD ASM.

mrec 9y ago
If anyone's interested, this is the (epic!) discussion thread for "Getting explicit SIMD on stable Rust": https://internals.rust-lang.org/t/getting-explicit-simd-on-stable-rust/4380/334 https://internals.rust-lang.org/t/getting-explicit-simd-on-s... It still wouldn't compete with really good hand-tuned asm, but it might help reduce the perf/usability tradeoff a bit.
- phkahler 9y ago
  I'd just like to see 2,3, and 4 element vectors as a first class citizen in C, C++, or Rust. These are incredibly common for so many things it's hard for me to understand this omission. I want to be able to pass them by value and as return values from function. I want to do operations like a=b+c with vectors without creating classes or overloading operators. For a lot of people the SIMD instructions are about parallel computation, but for me they represent vector primitives.
  - Zelizz 9y ago
    Look into the Clang vector extensions. They're very similar to GLSL/OpenCL vectors. They implement some very basic operator overloading too, so they're much less annoying that calling a function for everything. I made a simple linear algebra library using them, if you want to see an example: https://github.com/GavinHigham/glla https://github.com/GavinHigham/glla
  - mrec 9y ago
    What would baking them into the language add that a library solution couldn't? I'm not keeping track of Rust very closely these days, but it supports arithmetic operator overloading and the Copy trait. What's missing? I agree that this needs to be standardized for vector-ey crates to be able to talk to each other seamlessly. Otherwise we'll end up with a rerun of the C++ strings fiasco, with char* and wchar_t* and std::string and std::wstring and BSTR and _bstr_t and CString and CComBSTR and QString and GString and...
    - phkahler 9y ago
      OOP often involves overhead. If someone defined a really clean vector type/object in a library that let me write expressions the natural way, that could be added to the language I suppose. And that's what I want.
      - dbaupp 9y ago
        Rust, C and C++ all possess ways (it's the even default/only way in C and Rust, and almost so in C++) to write types like this that don't involve the typical pointer-soup/dynamic-call overhead of typical OOP. People can and do write vector in libraries right now, usually using existing compiler support that gives them guarantees about SIMD.
        StillBored 9y ago
        What I find odd about your reply, is that you downgrade C++, yet its really the only language that has sufficient operator overloading to do what the poster was requesting (transparent support without OO overhead). AKA, its completely possible, and there are quite a number of C++ libraries that make vectors look like native types complete with long lists of global operator overloads for interaction with other base types. Generally these libraries are just thin wrappers around inline assembly or intel intrinsics for SSE/AVX and generally don't bring any form of OO syntax into the picture. That said, even with C++ the libraries tend to fall down a bit when it comes to individual element operation/masking because the closest method is generally using array syntax for the elements which limits the operations to a single element in the vector at a time. Which means you end up with OO method calling syntax for element masking (or creative solutions with operator() )

bigdubs 9y ago

~fin~

knieveltech 9y ago
While hijacking a thread for idle pedantry is something of a sport around here your comment is unusually pointless for the form. Go eat something, you'll feel better.
spatulon 9y ago
It's still widely used and not considered archaic in British and Australian English.
- bigdubs 9y ago
  https://www.merriam-webster.com/dictionary/whilst https://www.merriam-webster.com/dictionary/whilst The definition is singularly, "while", with no extra connotation, other than it is as you said chiefly British. https://trends.google.com/trends/explore?q=while,whilst https://trends.google.com/trends/explore?q=while,whilst "While" is, not surprisingly, much more widely used, "whilst" being more affected.

daniel_rh 9y ago

I'm the author of https://github.com/dropbox/rust-brotli https://github.com/dropbox/rust-brotli and I certainly ran into the issues mentioned in the article, and the initial version of my brotli decoder and encoder were each almost 10x slower. I also worked around each and every one of them, and the result is something that performs at 80-90% of the speed of the original on average, and some files compress faster with rust than with the original brotli codec. allocator: I ran into the same situation and abstracted it with a generic allocator: https://github.com/dropbox/rust-alloc-no-stdlib https://github.com/dropbox/rust-alloc-no-stdlib it's an ugly solution, but it works and can allow for significant perf improvements down the line by bundling all same types together benchmarks: I simply made a test that printed the time primitive types: it's easy to write generic functions that do this macro system: I found to be amazing, but I never used it to compact data types borrow checker: split_at_mut was super helpful...also core::cmp::replace was useful to taking away pieces of a structure and manipulating them, then putting them back Though it wasn't all roses: to try to get SSE vectorization I had to convert this 6 line short string matching function https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d980e0a50795752c45dd3359e/src/enc/static_dict.rs#L122 https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d... into this multi hundred line monster with macros and so forth https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d980e0a50795752c45dd3359e/src/enc/static_dict.rs#L131 https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d... but in the end it was as fast as hand-tuned intrinsics in C I also found myself scared by dependencies, including onto the std library, so I tried to get everything to remain within the core library. This should allow something as low-level as a codec to be used in kernel space or in another place where a custom allocator is needed. I also found that "rewriting in safe rust from C" was made easy by corrode https://github.com/jameysharp/corrode https://github.com/jameysharp/corrode since C and rust can be interface compatible, it's easy to go one file at a time and turn it into working rust, then safe rust.

jeffdavis 9y ago

How can they allow jumping around the call stack while also destructing/freeing objects?