26 ms·

Rust: Not So Great For Codec Implementing

215 points by lossolo 9y ago

steveklabnik 9y ago
A lot of good comments on Reddit, with answers/solutions/discussion of the points brought up here: https://www.reddit.com/r/rust/comments/6qv2s5/rust_not_so_great_for_codec_implementing/ https://www.reddit.com/r/rust/comments/6qv2s5/rust_not_so_gr... These kinds of experience reports are so valuable. (I've been following the whole series and they're very interesting.) Even if solutions to these issues do exist, if people can't find them, well that's a problem too.
- mook 9y ago
  Seems like (from that Reddit thread) a lot of the things are in progress in the nightlies. I wonder if that means the nightlies are holding Rust back (because it relieves the pressure to actually ship features for use by normal developers).
  - steveklabnik 9y ago
    Trust me, being available only on nightly does not reduce pressure :) Most of our users use stable, and want to stick to stable. There's just some use-cases that need features we haven't finished designing yet, such is life.
    - mook 9y ago
      Might very well be; I have the luxury of not looking a nightlies at all :) it's just been frustrating how often problem reports get a response that the solutions are in the nightlies. I suppose it's possible that it's more meant as "it's being worked on" rather than "this is not a real issue, stop complaining", but the acknowledgement that it's actually an issue for people using release builds seem to be missing whenever nightlies are brought up in these sorts of responses. Contrast this with other languages (C++, Python, Ruby, Go) where I have no idea what features are under development until they hit a stable release. Compare to JavaScript, where stable releases are so slow people built transpilers ;) This could also be because I'm interested enough in Rust to occasionally land on the first-party discussion forums, of course (probably helped by how open the language development is). I'd say it's also young, except that it now has a release number much larger than 1.0 and doesn't really get to claim that anymore.
      - kbenson 9y ago
        > I suppose it's possible that it's more meant as "it's being worked on" rather than "this is not a real issue, stop complaining", but the acknowledgement that it's actually an issue for people using release builds seem to be missing whenever nightlies are brought up in these sorts of responses. I think that's the right assumption, because it's an acknowledgement that the issue actively being worked on to the degree that they have at least a partial solution implemented, as opposed to the "someone is working on that" response language developers sometimes give which usually predates that and could mean someone is reading literature and surveying the field and no code has actually been written. As you note, because development is so open, nightlies are easily accessible, so people might incorrectly infer that they are being recommended to use them, or that the problem is solved. I think what the devs are attempting to communicate is that it's close, and if you want you can test it out to see what it's like and give feedback. This might be a case of similar groups with slightly different context (lang devs / lang users) interpret a statement differently. From what I've seen, the Rust devs see fairly responsive to communication issues when pointed out, so maybe this will result in some change (or perhaps they are acutely aware of the issue already and instituted changes have not shown benefit, at least yet).
        mook 9y ago
        Yeah, I appreciate that nightlies are an option. It's just frustrating how often that option is trotted out without the caveat that it isn't a real solution. (This is again basically reacting to a particular Reddit comment linked above, but those sorts of comments are things that are more memorable.) Actually interacting with rust-related people, whatever their capacity (developer, user, etc.) tend to go pretty well; it's just that once in a while a random comment shows up that seems dismissive of actual problems people are reporting. You're probably right though, it's just that people have different contexts and I'm reading more into things than intended. Again, rust development is extremely open (which is great) and these things just tend to not be visible for other languages, rather than not happening.
      - steveklabnik 9y ago
        > I suppose it's possible that it's more meant as "it's being worked on" rather than "this is not a real issue, stop complaining", Yup, that's absolutely it. I'm not sure of a good, succinct way to imply the former rather than the later... > it now has a release number much larger than 1.0 We do releases every six weeks, so while it is in some sense, it's also just going to be higher than other languages.
  - Manishearth 9y ago
    I think that was a minority of the points; the majority of the points was in the space of the author not realizing something was possible in Rust. Out of these points they were all for features which are steadily on their way to stable (or future things which are actively being worked on right now). Very few folks use nightly -- many use it locally for improved developer tooling, but mostly everyone wants their crate to work on stable. So I wouldn't say it relieves pressure that way.
ainar-g 9y ago
The link doesn't seem to work for me at the moment. Here is Google Cache: http://webcache.googleusercontent.com/search?q=cache:qOUhWkjzMY0J:https://codecs.multimedia.cx/%3Fp%3D1246&num=1&lr=lang_de%7Clang_en%7Clang_ru&hl=ru&gl=ru&tbs=lr:lang_1de%7Clang_1en%7Clang_1ru&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:qOUhWkj...
modalduality 9y ago
Not a Rust expert, but some thoughts on the negatives. > Compilation time is too large Can you try compiling incrementally? https://blog.rust-lang.org/2016/09/08/incremental.html https://blog.rust-lang.org/2016/09/08/incremental.html. Might still only be on nightly. > And, on the similar note, benchmarks. I agree, profiling as well isn't as full featured as in more mature languages. Clojure, incidentally has great benchmarking due to being on the JVM. > Also the tuple assignments. Can't you just do fn main() { let (a, b) = (5, 2); println!("{}, {}", b, a); } >There are many cases where compiler could do the stuff automatically. I think this will be solved with the new non-lexical lifetimes RFC. Also a problem I had when starting, I generally assume referential transparency.
- masklinn 9y ago
  > Can't you just do That's a declaration not an assignment, TFA is (somewhat oddly) using the language precisely and fittingly: https://doc.rust-lang.org/reference/expressions.html#assignment-expressions https://doc.rust-lang.org/reference/expressions.html#assignm... And there are cases where shadowing is not acceptable e.g. when updating bindings within a loop, you usually want to see the updates from outside the loop.
- kllrnohj 9y ago
  > Clojure, incidentally has great benchmarking due to being on the JVM. Eh? Benchmarking on the JVM is notoriously difficult bordering on impossible. There's things like Google Caliper but test runs take forever due to attempting to force JIT warmup and doing GCs after every run. And the project's own wiki tells you the results are basically meaningless for a variety of reasons. Benchmarking things like C++ or Rust are trivial by comparison since when you call method foo() it's gonna do pretty much the same instructions every time. Highly consistent, highly repeatable, highly benchmarkable. Call method foo() in a JVM language and there's not a single person on the planet that can reliably tell you what's going to get executed on the metal. That's why you typically profile JVM languages rather than benchmarking them.
  - mcguire 9y ago
simias 9y ago
Some of the complaints are perfectly fair (compile time, powerful but unwieldy macros). Other complaints seem a bit odder to me: >While overall built-in testing capabilities in Rust are good (file it under good things too), the fact that benchmarking is available only for limbo nightly Rust is annoying; Okay, but what are you comparing it against? Neither C or C++ have builtin benchmarking or even tests. >If you care about systems programming and safety you’d have at least one or two functions to convert type into a smaller one (e.g. i16/u16 -> u8) and/or check whether the result fits. I don't understand this one at all. What's wrong with "as"? And if you need a checked conversion just write a small wrapper function? Again, if you're comparing to C or C++ I'd argue that Rust's semantic are friendlier and a lot less error prone at the cost of increased verbosity. C will implicitly cast and promote integer types without asking any questions, Rust makes it explicit. I understand C promotion rules and I've still managed to get it wrong on occasions, to the point where I now force myself to make all casts explicit in C like I would in Rust. At least this way the intent is obvious, as I think it should be. >Also the tuple assignments. I’d like to be able to assign multiple variables from a tuple but it’s not possible now. And maybe it would be nice to be able to declare several variables with one let; So unless I'm missing something something like: let (a, _, b) = (1, 2, 3) >Same for function calling—why does bitread.seek(bitread.tell() - 42); fail borrow check while let pos = bitread.tell() - 42; bitread.seek(pos); Yeah, that's always been a minor annoyance of mine too. I intuitively think that a(b()); should be treated like let tmp = b(); a(tmp); That is b() would terminate and release its borrows before a is even considered and "tmp" would live until the end of the block. I'm guessing that there must be a good reason why it isn't so however. Note that in C and C++ parameter evaluation order is unspecified so calling functions with potential side effects in parameter lists should be done very carefully. The compiler won't ever stop you though, don't worry. I'm sure the foot will grow back eventually. >Borrow checker and arrays. [...] Now this whole section feels a lot like "I'm trying to code in Rust like in C and it doesn't work and it frustrates me". Which is fair, messing around with arrays in C is a lot easier than Rust, there's no doubt about that. C also makes it massively easier to spectacularly shoot yourself in the foot if you mess up. Yes, if you need uninitialized arrays and these sorts of things you need to use unsafe. But that's mostly optimization and you probably don't want to start implementing your codec with that type of code. After all maybe the compiler will be clever enough to see what you're doing with the buffer and not actually run the init code. And if it doesn't you're free to add the unsafe code later, once your codec works, you have good tests and it's time to optimize more aggressively. >And that’s why C is still the best language for systems programming—it still lets you to do what you mean (the problem is that most programmers don’t really know what they mean) Ah, so after the Sufficiently Smart Compiler we have the Sufficiently Smart Coder. That seems like quite a definitive claim for somebody who didn't know about "split_at_mut" in Rust a few lines earlier. Maybe the author should experiment a bit more with the language before making bold statements like these? It's not rare to find security vulnerabilities in codecs and they tend to be extremely exposed. Maybe it's worth the hassle of making array manipulation slightly less convenient for the sake of security? >type keyword. Since it’s a keyword it can’t be used as a variable name and many objects have type, you know. All languages have reserved keywords, that's strictly in bikeshed territory. Of course when you get to a new language with a new set of reserved keyword you have to learn new habits. You never name anything "struct", "class", "switch" or "break" in C or C++ because you're used to have these keywords removed. Rust lets me name variables "class" but not "type". Oh well. >Not being able to combine if let with some other condition (those nested conditions tend to accumulate rather fast) I tend to agree with this one, although I guess it could become messy quickly if you mix refutable lets with regular boolean conditions.
StavrosK 9y ago
Since the link is a bit slow, here's an IPFS version: https://www.eternum.io/ipfs/QmYrdKpWbHCuNPUrBd6MPFrcTqPBW55w1AFHdw4toUt8MM/ https://www.eternum.io/ipfs/QmYrdKpWbHCuNPUrBd6MPFrcTqPBW55w...
- tambourine_man 9y ago
  Nice to see IPFS links popping up
  - StavrosK 9y ago
    I agree! Are you running a local node? I was thinking that a browser extension that automatically redirected URLs like ".*/ip[fn]s/\w+" to the local daemon address would be pretty useful.
    - tambourine_man 9y ago
      No, just like the idea behind it. Yeah, even a bookmarklet might be usefull. Or a single purpose site like http://isup.me http://isup.me
    - atbentley 9y ago
      Check out ipfs station if you use chrome. https://chrome.google.com/webstore/detail/ipfs-station/kckhgoigikkadogfdiojcblegfhdnjei https://chrome.google.com/webstore/detail/ipfs-station/kckhg...
      - StavrosK 9y ago
        Fantastic, that's exactly what I wanted, thank you. It doesn't seem to like my remote node, but I'll figure it out.
eptcyka 9y ago
Whilst unrelated to the article, my complaint about codecs in Rust is that they seem to be slow. Whilst the reason for this might be the immaturity of the libraries that I've used, but they've always been slower than their C counter-parts. The native JPEG decoder spins up 4 threads to decode the same amount of frames at 3x the time as the libjpeg-turbo does. There's a similar story for the FLAC decoder. I don't think that any of this has anything to do with the language itself, it's just that it takes time for things to mature. As for the issues outlined in the article - the language is different enough from C and C++ that one just has to accept the fact that you cannot write idiomatic C and C++ in Rust and expect it to be comfortable, performant or safe. However, with Rust, I'd say, that one can achieve 99% of what one can achieve in C today. The only thing that Rust is missing currently is the ability to arbitrarily jump around the call stack due to the way destructors are implemented, but there's a way to mitigate this and it's being worked as far as I was aware.
- pornel 9y ago
  Decoding in libjpeg-turbo is mostly implemented in ASM. Rust, like C, is slower than hand-optimized SIMD ASM.
  - mrec 9y ago
    If anyone's interested, this is the (epic!) discussion thread for "Getting explicit SIMD on stable Rust": https://internals.rust-lang.org/t/getting-explicit-simd-on-stable-rust/4380/334 https://internals.rust-lang.org/t/getting-explicit-simd-on-s... It still wouldn't compete with really good hand-tuned asm, but it might help reduce the perf/usability tradeoff a bit.
    - phkahler 9y ago
      I'd just like to see 2,3, and 4 element vectors as a first class citizen in C, C++, or Rust. These are incredibly common for so many things it's hard for me to understand this omission. I want to be able to pass them by value and as return values from function. I want to do operations like a=b+c with vectors without creating classes or overloading operators. For a lot of people the SIMD instructions are about parallel computation, but for me they represent vector primitives.
Sean1708 9y ago
> And don’t tell me about Bytes crate—it should not be a separate crate I'd be interested to hear the author's reasoning behind this, if it does what they want then why not use it? It's small and well written, so I don't think vetting it should be a problem. The rest of the article seems quite sensible, that comment just strikes me as a little odd.
- zanny 9y ago
  There is a real change in mentality you have to go through if you transition from a fairly strict C/C++/ even Java background to trying out Rust. In the former languages, adding dependencies rapidly becomes a painful experience, whereas Rust does much better dependency management and automatic building than even Python (where you need a requirements file or something similar to go pull down all the deps). With Rust, you really should just use crates. The std is meant to be limited to just the most used code and that which should not change for the sake of keeping the ecosystem stable.
  - ditonal 9y ago
    > even Python Off topic, but it shouldn't be better than "even Python", because Python has a really, really broken dependency system. Far more so than Java, which has Maven/Gradle which are both infinitely better than the pip/virtualenv disaster. People complain about things like shading in Maven being complicated. What they might not realize is that pip doesn't even try to address conflicting dependencies, it will just silently give you the wrong version! You ask for A==0.2 and it will give you A==0.1 if another dependency asked for A==0.1 first. And it won't even warn you even though it's straight up broken behavior. Virtualenv makes packaging annoying since it's almost vendoring but not quite. To totally understand the packaging system forces you into the world of eggs, wheels, disutils, conflicting versions, condas, etc. Sorry for tangent, just thought it was funny you would hold Python up as a standard of dependency excellence when it's probably the worst overall ecosystem of major languages.
    - nolok 9y ago
      I find it super weird how Python is still in this state while even PHP has managed to get something good going. To me, this is a clear case of trying to fix a broken system whereas starting from scratch would be much better.
dgfgfdagasdfgfa 9y ago
None of these seem related to codecs as opposed to the author's preference for how s/he likes to code. Not that this isn't super valuable feedback, but the title is misleading.
Animats 9y ago
The only legitimate complaint there is that it's tough in Rust to get two arbitrary mutable slices into the same array. Rust wants to be sure they're disjoint, to prevent aliasing. For some matrix manipulation, this is inconvenient. It would be easier if Rust had real multidimensional arrays. If the compiler knows about multidimensional arrays, some additional optimizations are possible. For example, if you want to borrow two different rows, that's fine as long as the row subscript is different.
- SilasX 9y ago
  Is it hard to implement such primitives for distinguishing ownership of subsections of an array? Or to extend Rust to do do that?
  - geofft 9y ago
    It's pretty easy to do it in unsafe code, or to use an existing safe function (implemented internally with unsafe code) like the standard library function split_at_mut to do it. Here's a doc on how to write such a function yourself, and includes the (actual) source code for split_at_mut: https://doc.rust-lang.org/nomicon/borrow-splitting.html https://doc.rust-lang.org/nomicon/borrow-splitting.html As others have mentioned, there's nothing special about the standard library in this regard; you can just write the method yourself and it'll work equally well. (The standard library is moderately special in that it gets to use nightly features because dealing with breaking changes is easy for code that literally lives inside Rust's own source repository, but it's not special in terms of its ability to use unsafe code or anything.)
    - SilasX 9y ago
      I meant primitives that preserve Rust's (safe) ownership model.
      - geofft 9y ago
        Well, you can use the existing split_at_mut if you're worried about getting your hand-rolled unsafe implementation wrong. It's definitely reasonable if you're splitting on rows; it may or may not be what you want if you're trying to get a slice-like thing that splits on columns. (There's no reason to avoid safe functions that happen to have unsafe implementations. For instance, the standard-library implementation of indexing a slice, as in b"hello"[3], is just a bounds check plus an unsafe dereference.)
davemp 9y ago
> And that’s why C is still the best language for systems programming—it still lets you to do what you mean (the problem is that most programmers don’t really know what they mean) Honestly after a ~1yr of embedded C co-op/intership experience, I'm familiar enough but not too entrenched to say that C is not that great for embedded/systems. When you're dealing at the hardware architecture level you need more detail than C (or any PL) can reasonably provide. C doesn't "let you do what you mean", it has no knowledge of special registers, interrupts, timers, DMA, etc. Companies have coped with a slew of macros that are just ugly to write with and make testing much more difficult than need be. If the language had actual support for embedded you'd see support for architecture strictly as libraries (which may be possible in C but certainly not ergonomic or supported by the culture around the language). Library architectures would make writing simulators and embedded unit tests __MUCH__ simpler. Not to mention the minefield that is the undefined sections of the C spec. A lot of people "mean" for an integer to rollover, but that's undefined and the C spec doesn't care about what you "mean". [1] Just today I meant for a constant lookup table not to be overwritten by the stack (I had plenty of unused RAM), but unfortunately C doesn't care what I meant. I had to dig around and find an odd macro to jam that data into program space--effectively hiring some muscle (gcc) to break the spec so it behaved the way I wanted. [2] I run into this sort of thing all the time. I know the C spec. I know my hardware. C doesn't care and needs to be beaten up. The only real benefit of C in these situations is that it's a huge sissy and people are really good at beating the piss out of it now. I'm not very familiar with rust, but if it makes an honest effort to be ergonomic for embedded (might need to be forked). It will eventually crush C. [1]: https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [2]: https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Named-Address-Spaces.html https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Named-Address-S...
- legulere 9y ago
  Just a tiny nitpick: Only signed integer overflow is undefined, unsigned integer overflow follows modulo arithmetics. It's pretty much impossible to remember all those details.
  - davemp 9y ago
ekidd 9y ago
Personally, I've been happy with Rust for writing an MPEG-2 subtitle decoder. I posted about this earlier here: https://news.ycombinator.com/item?id=14753201 https://news.ycombinator.com/item?id=14753201 For me, the biggest advantages over C/C++ were: 1. Rust's compile-time checks, run-time checks and fuzzing tools make it much easier to write secure code. Since decoders are a notorious source of bad security bugs, this is a big plus in my book. 2. Rust's dependency management makes it rather pleasant to rely on 3rd-party libraries. The combination of an immutable package repository and semver makes it easy to trust that things won't break. And in my experience, the number of 3rd-party libraries is relatively small compared to more popular languages, but the quality tends to be fairly high (especially relative to npm). 3. The tooling is surprisingly nice for a young language. cargo has solid defaults, Visual Studio Code provides auto-completion and tool tips, and there are good libraries for basic logging, argument parsing, etc. 4. Rust makes it relatively easy to write fast code, as long as you use references and slices when appropriate. Downsides include the learning curve (about a week or two for a C++ programmer to start feeling semi-comfortable), and slower compile times for large modules or ones using lots of parameterized types. My coworkers will write Rust, but they tend to say things like, "Rust is intense."
gwbas1c 9y ago
I tried Rust for the first time a few months ago. One of the things that really impressed me was that it was very easy to just get started. Even though the language is very different than I'm used to, I was able to "just jump in" without jumping through a lot of hoops. I was working with 3rd party crates very quickly. One thing that I think does need improvement is the error handling. Far too many idiomatic examples just panic for error handling. There's no good way to just group errors by a higher-level classification and fall through to a generic error handler. As a result, the choice is between panic on error, or very detailed and verbose error handling. There's no middle ground.
- biokoda 9y ago
  > There's no middle ground. Not at all. This is what the ? operator is for (was try! before). The idiomatic rust way is to have your own Error type that implements From for other Error types that you may encounter (io::Error, Utf8Error, etc.). Then you can simply just write let result = dosomething()?; for any operation that may fail. If an error happens, function will return with the error. I don't think you can do it any cleaner, given the constraints Rust is operating under.
  - moomin 9y ago
    The From is The Right Thing, but it can be hard to convince people used to unchecked exceptions and costless casting of this. (I say costless, you pay for it every time you don't cast...)
- steveklabnik 9y ago
  There's a whole chapter in the second edition of the book devoted to error handling https://doc.rust-lang.org/book/second-edition/ch09-00-error-handling.html https://doc.rust-lang.org/book/second-edition/ch09-00-error-...
  - gwbas1c 9y ago
    Yes, that chapter painfully explains my point. Most of the replies pointed out the new ? operator, which I'll have to check out. (One of the things that I like about the Rust community is that they constantly improve.) The thing is, if you've only handled errors via return codes in C, then Rust's system is a major improvement. The challenge comes once you've programmed with exceptions that have inheritance. It's pretty easy to set filters higher up in the stack and "not care" lower in the stack. This allows you to effectively ignore error handling for functions that only fail in obscure corner cases, because a filter higher up in the stack can handle so many different errors. (Operations that require cleanup can also be written in a way that cleanup code always runs, as some languages support try-finally.) That's probably the hardest thing for me to adjust to in Rust. Lifetimes take care of the try-finally part, but I still can't figure out how to have generic error filters higher in my stack.
AceJohnny2 9y ago
Codecs are actually a wonderful test-case for Rust: they're a frequent source of vulnerabilities because they have to deal with arbitrary input, they usually have to perform fiddly sub-byte and/or array manipulation, and their pressure for maximum performance mean they have to make as efficient use of the hardware and memory as possible (and any mistake lead back to the first point) So even if you don't agree with some of the author's points, his use-case is a good test for the suitability of Rust as a Systems Programming Language.
- Gankro 9y ago
  Yes, let us never forget how many exploits there are in trivial codecs like ICO and BMP, because they're written in C(++): https://bugzilla.mozilla.org/show_bug.cgi?id=775794#c0 https://bugzilla.mozilla.org/show_bug.cgi?id=775794#c0
  - nnethercote 9y ago
    BMP isn't so trivial, largely because it's been extended multiple times while never having a proper spec. http://searchfox.org/mozilla-central/rev/bbc1c59e460a27b20929b56489e2e55438de81fa/image/decoders/nsBMPDecoder.cpp#10-83 http://searchfox.org/mozilla-central/rev/bbc1c59e460a27b2092... has some of the gory details.
    - tinus_hn 9y ago
      And .ico is just .bmp with some extras.
schneems 9y ago
Site is down for me. Mirror?
- steveklabnik 9y ago
  https://news.ycombinator.com/item?id=14901856 https://news.ycombinator.com/item?id=14901856
AndyKelley 9y ago
This article is practically selling Zig. It has all the good stuff mentioned sans traits, and none of the bad stuff. The only new bad thing Zig brings to the table is immaturity. Zig is a much more modest incremental improvement over C whereas rust is trying to introduce this new big idea of the borrow checker.
- hinkley 9y ago
  Watch out for the inevitable copycat product called Zag that steals all your users by saying they zigged when they should have zagged.

215 points by lossolo 9y ago

A lot of good comments on Reddit, with answers/solutions/discussion of the points brought up here: https://www.reddit.com/r/rust/comments/6qv2s5/rust_not_so_great_for_codec_implementing/ https://www.reddit.com/r/rust/comments/6qv2s5/rust_not_so_gr... These kinds of experience reports are so valuable. (I've been following the whole series and they're very interesting.) Even if solutions to these issues do exist, if people can't find them, well that's a problem too.

mook 9y ago
Seems like (from that Reddit thread) a lot of the things are in progress in the nightlies. I wonder if that means the nightlies are holding Rust back (because it relieves the pressure to actually ship features for use by normal developers).
- steveklabnik 9y ago
  Trust me, being available only on nightly does not reduce pressure :) Most of our users use stable, and want to stick to stable. There's just some use-cases that need features we haven't finished designing yet, such is life.
  - mook 9y ago
    Might very well be; I have the luxury of not looking a nightlies at all :) it's just been frustrating how often problem reports get a response that the solutions are in the nightlies. I suppose it's possible that it's more meant as "it's being worked on" rather than "this is not a real issue, stop complaining", but the acknowledgement that it's actually an issue for people using release builds seem to be missing whenever nightlies are brought up in these sorts of responses. Contrast this with other languages (C++, Python, Ruby, Go) where I have no idea what features are under development until they hit a stable release. Compare to JavaScript, where stable releases are so slow people built transpilers ;) This could also be because I'm interested enough in Rust to occasionally land on the first-party discussion forums, of course (probably helped by how open the language development is). I'd say it's also young, except that it now has a release number much larger than 1.0 and doesn't really get to claim that anymore.
    - kbenson 9y ago
      > I suppose it's possible that it's more meant as "it's being worked on" rather than "this is not a real issue, stop complaining", but the acknowledgement that it's actually an issue for people using release builds seem to be missing whenever nightlies are brought up in these sorts of responses. I think that's the right assumption, because it's an acknowledgement that the issue actively being worked on to the degree that they have at least a partial solution implemented, as opposed to the "someone is working on that" response language developers sometimes give which usually predates that and could mean someone is reading literature and surveying the field and no code has actually been written. As you note, because development is so open, nightlies are easily accessible, so people might incorrectly infer that they are being recommended to use them, or that the problem is solved. I think what the devs are attempting to communicate is that it's close, and if you want you can test it out to see what it's like and give feedback. This might be a case of similar groups with slightly different context (lang devs / lang users) interpret a statement differently. From what I've seen, the Rust devs see fairly responsive to communication issues when pointed out, so maybe this will result in some change (or perhaps they are acutely aware of the issue already and instituted changes have not shown benefit, at least yet).
      - mook 9y ago
        Yeah, I appreciate that nightlies are an option. It's just frustrating how often that option is trotted out without the caveat that it isn't a real solution. (This is again basically reacting to a particular Reddit comment linked above, but those sorts of comments are things that are more memorable.) Actually interacting with rust-related people, whatever their capacity (developer, user, etc.) tend to go pretty well; it's just that once in a while a random comment shows up that seems dismissive of actual problems people are reporting. You're probably right though, it's just that people have different contexts and I'm reading more into things than intended. Again, rust development is extremely open (which is great) and these things just tend to not be visible for other languages, rather than not happening.
    - steveklabnik 9y ago
      > I suppose it's possible that it's more meant as "it's being worked on" rather than "this is not a real issue, stop complaining", Yup, that's absolutely it. I'm not sure of a good, succinct way to imply the former rather than the later... > it now has a release number much larger than 1.0 We do releases every six weeks, so while it is in some sense, it's also just going to be higher than other languages.
- Manishearth 9y ago
  I think that was a minority of the points; the majority of the points was in the space of the author not realizing something was possible in Rust. Out of these points they were all for features which are steadily on their way to stable (or future things which are actively being worked on right now). Very few folks use nightly -- many use it locally for improved developer tooling, but mostly everyone wants their crate to work on stable. So I wouldn't say it relieves pressure that way.

ainar-g 9y ago

The link doesn't seem to work for me at the moment. Here is Google Cache: http://webcache.googleusercontent.com/search?q=cache:qOUhWkjzMY0J:https://codecs.multimedia.cx/%3Fp%3D1246&num=1&lr=lang_de%7Clang_en%7Clang_ru&hl=ru&gl=ru&tbs=lr:lang_1de%7Clang_1en%7Clang_1ru&strip=1&vwsrc=0 http://webcache.googleusercontent.com/search?q=cache:qOUhWkj...

modalduality 9y ago

Not a Rust expert, but some thoughts on the negatives. > Compilation time is too large Can you try compiling incrementally? https://blog.rust-lang.org/2016/09/08/incremental.html https://blog.rust-lang.org/2016/09/08/incremental.html. Might still only be on nightly. > And, on the similar note, benchmarks. I agree, profiling as well isn't as full featured as in more mature languages. Clojure, incidentally has great benchmarking due to being on the JVM. > Also the tuple assignments. Can't you just do fn main() { let (a, b) = (5, 2); println!("{}, {}", b, a); } >There are many cases where compiler could do the stuff automatically. I think this will be solved with the new non-lexical lifetimes RFC. Also a problem I had when starting, I generally assume referential transparency.

masklinn 9y ago
> Can't you just do That's a declaration not an assignment, TFA is (somewhat oddly) using the language precisely and fittingly: https://doc.rust-lang.org/reference/expressions.html#assignment-expressions https://doc.rust-lang.org/reference/expressions.html#assignm... And there are cases where shadowing is not acceptable e.g. when updating bindings within a loop, you usually want to see the updates from outside the loop.
kllrnohj 9y ago
> Clojure, incidentally has great benchmarking due to being on the JVM. Eh? Benchmarking on the JVM is notoriously difficult bordering on impossible. There's things like Google Caliper but test runs take forever due to attempting to force JIT warmup and doing GCs after every run. And the project's own wiki tells you the results are basically meaningless for a variety of reasons. Benchmarking things like C++ or Rust are trivial by comparison since when you call method foo() it's gonna do pretty much the same instructions every time. Highly consistent, highly repeatable, highly benchmarkable. Call method foo() in a JVM language and there's not a single person on the planet that can reliably tell you what's going to get executed on the metal. That's why you typically profile JVM languages rather than benchmarking them.
- mcguire 9y ago

simias 9y ago

Some of the complaints are perfectly fair (compile time, powerful but unwieldy macros). Other complaints seem a bit odder to me: >While overall built-in testing capabilities in Rust are good (file it under good things too), the fact that benchmarking is available only for limbo nightly Rust is annoying; Okay, but what are you comparing it against? Neither C or C++ have builtin benchmarking or even tests. >If you care about systems programming and safety you’d have at least one or two functions to convert type into a smaller one (e.g. i16/u16 -> u8) and/or check whether the result fits. I don't understand this one at all. What's wrong with "as"? And if you need a checked conversion just write a small wrapper function? Again, if you're comparing to C or C++ I'd argue that Rust's semantic are friendlier and a lot less error prone at the cost of increased verbosity. C will implicitly cast and promote integer types without asking any questions, Rust makes it explicit. I understand C promotion rules and I've still managed to get it wrong on occasions, to the point where I now force myself to make all casts explicit in C like I would in Rust. At least this way the intent is obvious, as I think it should be. >Also the tuple assignments. I’d like to be able to assign multiple variables from a tuple but it’s not possible now. And maybe it would be nice to be able to declare several variables with one let; So unless I'm missing something something like: let (a, _, b) = (1, 2, 3) >Same for function calling—why does bitread.seek(bitread.tell() - 42); fail borrow check while let pos = bitread.tell() - 42; bitread.seek(pos); Yeah, that's always been a minor annoyance of mine too. I intuitively think that a(b()); should be treated like let tmp = b(); a(tmp); That is b() would terminate and release its borrows before a is even considered and "tmp" would live until the end of the block. I'm guessing that there must be a good reason why it isn't so however. Note that in C and C++ parameter evaluation order is unspecified so calling functions with potential side effects in parameter lists should be done very carefully. The compiler won't ever stop you though, don't worry. I'm sure the foot will grow back eventually. >Borrow checker and arrays. [...] Now this whole section feels a lot like "I'm trying to code in Rust like in C and it doesn't work and it frustrates me". Which is fair, messing around with arrays in C is a lot easier than Rust, there's no doubt about that. C also makes it massively easier to spectacularly shoot yourself in the foot if you mess up. Yes, if you need uninitialized arrays and these sorts of things you need to use unsafe. But that's mostly optimization and you probably don't want to start implementing your codec with that type of code. After all maybe the compiler will be clever enough to see what you're doing with the buffer and not actually run the init code. And if it doesn't you're free to add the unsafe code later, once your codec works, you have good tests and it's time to optimize more aggressively. >And that’s why C is still the best language for systems programming—it still lets you to do what you mean (the problem is that most programmers don’t really know what they mean) Ah, so after the Sufficiently Smart Compiler we have the Sufficiently Smart Coder. That seems like quite a definitive claim for somebody who didn't know about "split_at_mut" in Rust a few lines earlier. Maybe the author should experiment a bit more with the language before making bold statements like these? It's not rare to find security vulnerabilities in codecs and they tend to be extremely exposed. Maybe it's worth the hassle of making array manipulation slightly less convenient for the sake of security? >type keyword. Since it’s a keyword it can’t be used as a variable name and many objects have type, you know. All languages have reserved keywords, that's strictly in bikeshed territory. Of course when you get to a new language with a new set of reserved keyword you have to learn new habits. You never name anything "struct", "class", "switch" or "break" in C or C++ because you're used to have these keywords removed. Rust lets me name variables "class" but not "type". Oh well. >Not being able to combine if let with some other condition (those nested conditions tend to accumulate rather fast) I tend to agree with this one, although I guess it could become messy quickly if you mix refutable lets with regular boolean conditions.

StavrosK 9y ago

Since the link is a bit slow, here's an IPFS version: https://www.eternum.io/ipfs/QmYrdKpWbHCuNPUrBd6MPFrcTqPBW55w1AFHdw4toUt8MM/ https://www.eternum.io/ipfs/QmYrdKpWbHCuNPUrBd6MPFrcTqPBW55w...

tambourine_man 9y ago
Nice to see IPFS links popping up
- StavrosK 9y ago
  I agree! Are you running a local node? I was thinking that a browser extension that automatically redirected URLs like ".*/ip[fn]s/\w+" to the local daemon address would be pretty useful.
  - tambourine_man 9y ago
    No, just like the idea behind it. Yeah, even a bookmarklet might be usefull. Or a single purpose site like http://isup.me http://isup.me
  - atbentley 9y ago
    Check out ipfs station if you use chrome. https://chrome.google.com/webstore/detail/ipfs-station/kckhgoigikkadogfdiojcblegfhdnjei https://chrome.google.com/webstore/detail/ipfs-station/kckhg...
    - StavrosK 9y ago
      Fantastic, that's exactly what I wanted, thank you. It doesn't seem to like my remote node, but I'll figure it out.

eptcyka 9y ago

Whilst unrelated to the article, my complaint about codecs in Rust is that they seem to be slow. Whilst the reason for this might be the immaturity of the libraries that I've used, but they've always been slower than their C counter-parts. The native JPEG decoder spins up 4 threads to decode the same amount of frames at 3x the time as the libjpeg-turbo does. There's a similar story for the FLAC decoder. I don't think that any of this has anything to do with the language itself, it's just that it takes time for things to mature. As for the issues outlined in the article - the language is different enough from C and C++ that one just has to accept the fact that you cannot write idiomatic C and C++ in Rust and expect it to be comfortable, performant or safe. However, with Rust, I'd say, that one can achieve 99% of what one can achieve in C today. The only thing that Rust is missing currently is the ability to arbitrarily jump around the call stack due to the way destructors are implemented, but there's a way to mitigate this and it's being worked as far as I was aware.

pornel 9y ago
Decoding in libjpeg-turbo is mostly implemented in ASM. Rust, like C, is slower than hand-optimized SIMD ASM.
- mrec 9y ago
  If anyone's interested, this is the (epic!) discussion thread for "Getting explicit SIMD on stable Rust": https://internals.rust-lang.org/t/getting-explicit-simd-on-stable-rust/4380/334 https://internals.rust-lang.org/t/getting-explicit-simd-on-s... It still wouldn't compete with really good hand-tuned asm, but it might help reduce the perf/usability tradeoff a bit.
  - phkahler 9y ago
    I'd just like to see 2,3, and 4 element vectors as a first class citizen in C, C++, or Rust. These are incredibly common for so many things it's hard for me to understand this omission. I want to be able to pass them by value and as return values from function. I want to do operations like a=b+c with vectors without creating classes or overloading operators. For a lot of people the SIMD instructions are about parallel computation, but for me they represent vector primitives.

Sean1708 9y ago

> And don’t tell me about Bytes crate—it should not be a separate crate I'd be interested to hear the author's reasoning behind this, if it does what they want then why not use it? It's small and well written, so I don't think vetting it should be a problem. The rest of the article seems quite sensible, that comment just strikes me as a little odd.

zanny 9y ago
There is a real change in mentality you have to go through if you transition from a fairly strict C/C++/ even Java background to trying out Rust. In the former languages, adding dependencies rapidly becomes a painful experience, whereas Rust does much better dependency management and automatic building than even Python (where you need a requirements file or something similar to go pull down all the deps). With Rust, you really should just use crates. The std is meant to be limited to just the most used code and that which should not change for the sake of keeping the ecosystem stable.
- ditonal 9y ago
  > even Python Off topic, but it shouldn't be better than "even Python", because Python has a really, really broken dependency system. Far more so than Java, which has Maven/Gradle which are both infinitely better than the pip/virtualenv disaster. People complain about things like shading in Maven being complicated. What they might not realize is that pip doesn't even try to address conflicting dependencies, it will just silently give you the wrong version! You ask for A==0.2 and it will give you A==0.1 if another dependency asked for A==0.1 first. And it won't even warn you even though it's straight up broken behavior. Virtualenv makes packaging annoying since it's almost vendoring but not quite. To totally understand the packaging system forces you into the world of eggs, wheels, disutils, conflicting versions, condas, etc. Sorry for tangent, just thought it was funny you would hold Python up as a standard of dependency excellence when it's probably the worst overall ecosystem of major languages.
  - nolok 9y ago
    I find it super weird how Python is still in this state while even PHP has managed to get something good going. To me, this is a clear case of trying to fix a broken system whereas starting from scratch would be much better.

dgfgfdagasdfgfa 9y ago

None of these seem related to codecs as opposed to the author's preference for how s/he likes to code. Not that this isn't super valuable feedback, but the title is misleading.

Animats 9y ago

The only legitimate complaint there is that it's tough in Rust to get two arbitrary mutable slices into the same array. Rust wants to be sure they're disjoint, to prevent aliasing. For some matrix manipulation, this is inconvenient. It would be easier if Rust had real multidimensional arrays. If the compiler knows about multidimensional arrays, some additional optimizations are possible. For example, if you want to borrow two different rows, that's fine as long as the row subscript is different.

SilasX 9y ago
Is it hard to implement such primitives for distinguishing ownership of subsections of an array? Or to extend Rust to do do that?
- geofft 9y ago
  It's pretty easy to do it in unsafe code, or to use an existing safe function (implemented internally with unsafe code) like the standard library function split_at_mut to do it. Here's a doc on how to write such a function yourself, and includes the (actual) source code for split_at_mut: https://doc.rust-lang.org/nomicon/borrow-splitting.html https://doc.rust-lang.org/nomicon/borrow-splitting.html As others have mentioned, there's nothing special about the standard library in this regard; you can just write the method yourself and it'll work equally well. (The standard library is moderately special in that it gets to use nightly features because dealing with breaking changes is easy for code that literally lives inside Rust's own source repository, but it's not special in terms of its ability to use unsafe code or anything.)
  - SilasX 9y ago
    I meant primitives that preserve Rust's (safe) ownership model.
    - geofft 9y ago
      Well, you can use the existing split_at_mut if you're worried about getting your hand-rolled unsafe implementation wrong. It's definitely reasonable if you're splitting on rows; it may or may not be what you want if you're trying to get a slice-like thing that splits on columns. (There's no reason to avoid safe functions that happen to have unsafe implementations. For instance, the standard-library implementation of indexing a slice, as in b"hello"[3], is just a bounds check plus an unsafe dereference.)

davemp 9y ago

> And that’s why C is still the best language for systems programming—it still lets you to do what you mean (the problem is that most programmers don’t really know what they mean) Honestly after a ~1yr of embedded C co-op/intership experience, I'm familiar enough but not too entrenched to say that C is not that great for embedded/systems. When you're dealing at the hardware architecture level you need more detail than C (or any PL) can reasonably provide. C doesn't "let you do what you mean", it has no knowledge of special registers, interrupts, timers, DMA, etc. Companies have coped with a slew of macros that are just ugly to write with and make testing much more difficult than need be. If the language had actual support for embedded you'd see support for architecture strictly as libraries (which may be possible in C but certainly not ergonomic or supported by the culture around the language). Library architectures would make writing simulators and embedded unit tests __MUCH__ simpler. Not to mention the minefield that is the undefined sections of the C spec. A lot of people "mean" for an integer to rollover, but that's undefined and the C spec doesn't care about what you "mean". [1] Just today I meant for a constant lookup table not to be overwritten by the stack (I had plenty of unused RAM), but unfortunately C doesn't care what I meant. I had to dig around and find an odd macro to jam that data into program space--effectively hiring some muscle (gcc) to break the spec so it behaved the way I wanted. [2] I run into this sort of thing all the time. I know the C spec. I know my hardware. C doesn't care and needs to be beaten up. The only real benefit of C in these situations is that it's a huge sissy and people are really good at beating the piss out of it now. I'm not very familiar with rust, but if it makes an honest effort to be ergonomic for embedded (might need to be forked). It will eventually crush C. [1]: https://blog.regehr.org/archives/213 https://blog.regehr.org/archives/213 [2]: https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Named-Address-Spaces.html https://gcc.gnu.org/onlinedocs/gcc-4.8.5/gcc/Named-Address-S...

legulere 9y ago
Just a tiny nitpick: Only signed integer overflow is undefined, unsigned integer overflow follows modulo arithmetics. It's pretty much impossible to remember all those details.
- davemp 9y ago

ekidd 9y ago

Personally, I've been happy with Rust for writing an MPEG-2 subtitle decoder. I posted about this earlier here: https://news.ycombinator.com/item?id=14753201 https://news.ycombinator.com/item?id=14753201 For me, the biggest advantages over C/C++ were: 1. Rust's compile-time checks, run-time checks and fuzzing tools make it much easier to write secure code. Since decoders are a notorious source of bad security bugs, this is a big plus in my book. 2. Rust's dependency management makes it rather pleasant to rely on 3rd-party libraries. The combination of an immutable package repository and semver makes it easy to trust that things won't break. And in my experience, the number of 3rd-party libraries is relatively small compared to more popular languages, but the quality tends to be fairly high (especially relative to npm). 3. The tooling is surprisingly nice for a young language. cargo has solid defaults, Visual Studio Code provides auto-completion and tool tips, and there are good libraries for basic logging, argument parsing, etc. 4. Rust makes it relatively easy to write fast code, as long as you use references and slices when appropriate. Downsides include the learning curve (about a week or two for a C++ programmer to start feeling semi-comfortable), and slower compile times for large modules or ones using lots of parameterized types. My coworkers will write Rust, but they tend to say things like, "Rust is intense."

gwbas1c 9y ago

I tried Rust for the first time a few months ago. One of the things that really impressed me was that it was very easy to just get started. Even though the language is very different than I'm used to, I was able to "just jump in" without jumping through a lot of hoops. I was working with 3rd party crates very quickly. One thing that I think does need improvement is the error handling. Far too many idiomatic examples just panic for error handling. There's no good way to just group errors by a higher-level classification and fall through to a generic error handler. As a result, the choice is between panic on error, or very detailed and verbose error handling. There's no middle ground.

biokoda 9y ago
> There's no middle ground. Not at all. This is what the ? operator is for (was try! before). The idiomatic rust way is to have your own Error type that implements From for other Error types that you may encounter (io::Error, Utf8Error, etc.). Then you can simply just write let result = dosomething()?; for any operation that may fail. If an error happens, function will return with the error. I don't think you can do it any cleaner, given the constraints Rust is operating under.
- moomin 9y ago
  The From is The Right Thing, but it can be hard to convince people used to unchecked exceptions and costless casting of this. (I say costless, you pay for it every time you don't cast...)
steveklabnik 9y ago
There's a whole chapter in the second edition of the book devoted to error handling https://doc.rust-lang.org/book/second-edition/ch09-00-error-handling.html https://doc.rust-lang.org/book/second-edition/ch09-00-error-...
- gwbas1c 9y ago
  Yes, that chapter painfully explains my point. Most of the replies pointed out the new ? operator, which I'll have to check out. (One of the things that I like about the Rust community is that they constantly improve.) The thing is, if you've only handled errors via return codes in C, then Rust's system is a major improvement. The challenge comes once you've programmed with exceptions that have inheritance. It's pretty easy to set filters higher up in the stack and "not care" lower in the stack. This allows you to effectively ignore error handling for functions that only fail in obscure corner cases, because a filter higher up in the stack can handle so many different errors. (Operations that require cleanup can also be written in a way that cleanup code always runs, as some languages support try-finally.) That's probably the hardest thing for me to adjust to in Rust. Lifetimes take care of the try-finally part, but I still can't figure out how to have generic error filters higher in my stack.

AceJohnny2 9y ago

Codecs are actually a wonderful test-case for Rust: they're a frequent source of vulnerabilities because they have to deal with arbitrary input, they usually have to perform fiddly sub-byte and/or array manipulation, and their pressure for maximum performance mean they have to make as efficient use of the hardware and memory as possible (and any mistake lead back to the first point) So even if you don't agree with some of the author's points, his use-case is a good test for the suitability of Rust as a Systems Programming Language.

Gankro 9y ago
Yes, let us never forget how many exploits there are in trivial codecs like ICO and BMP, because they're written in C(++): https://bugzilla.mozilla.org/show_bug.cgi?id=775794#c0 https://bugzilla.mozilla.org/show_bug.cgi?id=775794#c0
- nnethercote 9y ago
  BMP isn't so trivial, largely because it's been extended multiple times while never having a proper spec. http://searchfox.org/mozilla-central/rev/bbc1c59e460a27b20929b56489e2e55438de81fa/image/decoders/nsBMPDecoder.cpp#10-83 http://searchfox.org/mozilla-central/rev/bbc1c59e460a27b2092... has some of the gory details.
  - tinus_hn 9y ago
    And .ico is just .bmp with some extras.

schneems 9y ago

Site is down for me. Mirror?

steveklabnik 9y ago
https://news.ycombinator.com/item?id=14901856 https://news.ycombinator.com/item?id=14901856

AndyKelley 9y ago

This article is practically selling Zig. It has all the good stuff mentioned sans traits, and none of the bad stuff. The only new bad thing Zig brings to the table is immaturity. Zig is a much more modest incremental improvement over C whereas rust is trying to introduce this new big idea of the borrow checker.

hinkley 9y ago
Watch out for the inevitable copycat product called Zag that steals all your users by saying they zigged when they should have zagged.