16 ms·
Some mistakes Rust doesn't catch
- athorax 5y agoThis is unbelievably awesome. Learning by breaking things is one of my favorite things and the level of detail here is astounding!
- benmanns 5y agoYeah, and a good level of “yup that makes sense, should work.” [crash] “wait what?!”
- wokwokwok 5y agoI wonder if there's a future where you have a language model that actually inspects the name of your functions, does an automatic code-gen of them from the context and then compares your implementation and gives you lint errors like: fn add() { ... }: mispleading name or incorrect implementation `a - b` You know, once upon a time I would have thought that was categorically impossible, but now, you know, it actually seems like something that might not be that far off. > It's just not something Rust is designed to protect against - at least, not today. ...maybe, one day. :)
- eru 5y agoIf you are willing to accept quite a few false positives and false negatives, I think current techniques in machine learning can deliver on this promise. (If you put in all the work to train a good mode etc.) Btw, have a look at hlint. It sometimes gives rather involved hints about how you can restructure your code.
- dhosek 5y agoIsn't this what the autopilot thing is supposed to do kind of? Except it just gives you the auto-generated code.
- melony 5y agoJust run the output of autopilot in a symbolic execution environment and diff it against the code paths of the user code. This is one (admittedly convoluted) way of implementing what OP wants.
- kgeist 5y agoThis kind of linter would be useful for teams whose native language isn't English (such as ours). They often choose bizarre names for their functions, it's not always clear what a function does by reading its name alone.
- woodruffw 5y ago> fn add() { ... }: mispleading name or incorrect implementation `a - b` Advice like this is frustratingly non-general: you might be doing addition over GF2[1] in which case `fn add(a, b) { a ^ b }` is a perfectly sound function. I've worked on a codebase, in Rust, where that's the case[2]! [1]: https://en.wikipedia.org/wiki/Finite_field_arithmetic#Effective_polynomial_representation https://en.wikipedia.org/wiki/Finite_field_arithmetic#Effect... [2]: https://github.com/trailofbits/reverie/blob/master/src/algebra/gf2/batch.rs#L49 https://github.com/trailofbits/reverie/blob/master/src/algeb...
- deleted 5y ago[deleted]
- xanathar 5y agoLinters should be frustatingly non-general. They should alert for the common case and special cases should have a "#[allow(xxx)]" or "#pragma warning disable xxx" (or whatever does that in your language) and, possibly, a comment. This, incidentally, also helps fellow humans reading the code who'll see that it's all intentional and won't flip with "wtf why does it XOR in an add method?" even if just for a second.
- marcosdumay 5y agoOnly up to a certain point. If you add a few pragmas before each line of code, your code is not completely illegible. As an example, those linters that have a message for each programing style and a rule to convert between them are stupid. They might be useful as some other kind of code, but as linters they are not just useless, they are actively harmful.
- zild3d 5y ago> you might be doing addition over GF2 Then the function name, class, and/or comments should be making that clear! In this case, the BatchGF2 class does so. Maybe somewhere else a Fourier class has `fn add() { a * b } // a and b are in time domain` but a standalone add() function with no comments or class should probably not be doing a ^ b, a - b, or a * b. some github copilot like linter can then draw from other GF2, Fourier, etc code references vs vanilla add functions
- zesterer 5y agoAs JoshTriplett said above, Rust's Clippy does this today: https://rust-lang.github.io/rust-clippy/master/#suspicious_arithmetic_impl https://rust-lang.github.io/rust-clippy/master/#suspicious_a...
- deleted 5y ago[deleted]
- majewsky 5y agoIt's quite fitting that this sort of thing is dubbed "autopilot", because in all likelihood it will have the same problem as gradually more self-driving cars. There will be a level of automation where the computer still makes errors frequently enough, but the user is sufficiently disconnected from the action that they lose all attentiveness.
- dhosek 5y agoI think the rust making you aware of what you need to do is a good thing. I have a vague memory of writing C code when I was a teenager where I directly manipulated a string from argv[] doing some concatenation or other and it worked fine until it didn't because I was merrily stomping on memory I didn't own (I was thinking in Pascal terms while writing C code). The whole String vs &str thing in Rust can feel a bit fiddly at times, but it does make you aware of what's happening (and I've found that it has improved my JVM coding as well although I do have to occasionally remind myself that I don't need to worry about borrows and lifetimes).
- usrbinbash 5y agoThe thing is, C also makes me aware of what is happening, by letting programs fail when I forget about certain things. This may not be as nice as the compiler yelling at me even before the program is run, but it certainly makes for a much simpler language.
- eklavya 5y agoSure and sometimes it directly tells your users, bypassing you entirely :)
- kaba0 5y agoThe thing is, C won’t even fail in a reproducible way. Good luck hunting down mistakes caused by uninitialized variables for example. Sure, valgrind exists, but that is not native execution.
- usrbinbash 5y agoThe reproducible thing about this error would be: I forgot to initialize my variables.
- MereInterest 5y agoThat results in undefined behavior, which is by definition, not reproducible.
- emptysea 5y agoSomething to note about the JS examples, if you throw them into TypeScript you'll get similar error messages to go vet and rustc. The first example: https://www.typescriptlang.org/play?#code/GYVwdgxgLglg9mABMOcAUMCUiDeBYAKEUQgQGc4AbAUwDpK4BzNAIhThYBpEsBuQgL6FCoSLASIARgEMATmmz4iJclToNmLGbICELTPwJCCI8NHhIAttJhgFuQsRSzEGRAF5EABl49EAHkQAZl8YAGowxUdiZFQMA2jjYllqKBBZMENibQVDY0JrW1ygA https://www.typescriptlang.org/play?#code/GYVwdgxgLglg9mABMO... Unreachable code detected Along with some typos since `i` isn't declared.
- balls187 5y agoA JS Linter will also flag that line as well.
- fasterthanlime 5y ago(Author here) Absolutely! I love TypeScript and refuse to write JavaScript without it (even just in JS-checking mode with jsdoc annotations), but I'm already fighting, like, 4 different battles in that article, I didn't want to bring a fifth one into the mix.
- Dylan16807 5y ago> add("foo".to_string(), "bar") > It's a pretty radical design choice, to force you to be aware that, yeah, since you're creating a new value (out of the two parameters), you will have to allocate. And so it forces you to allocate outside of the Add operation itself. Forcing you to think about allocation is good for many situations. Making you do an extra allocation is a bad way to do that. And the only way I can think of to avoid that would give up having the function be generic.
- hermanradtke 5y agoThere is no extra allocation. No matter how you do it, you will need at least one allocation to hold the result of combined strings.
- Dylan16807 5y agoYes, you need one. Which would mean using String::with_capacity based on both &str. Not making a string from one side, then very likely resizing it to add the other. The generic code+call will often make two allocations.
- Arnavion 5y ago>Making you do an extra allocation is a bad way to do that. And the only way I can think of to avoid that would give up having the function be generic. Sure, if you want to optimize that, then create your "foo" string with the extra capacity instead of doing `"foo".to_string()`, eg `{ let foo = String::with_capacity("foo".len() + "bar".len()); foo.push_str("foo"); foo }`. It has nothing to do with the generic `add` function; it can stay generic as it is just fine. Even a function specialized for adding String and &str could not go back in time and change how much capacity the String was created with. Edit (1h later): It's also worth noting that: - This desire of `Add::add(&str, &str) -> String` only seems to come up with people learning the language and bringing expectations from other languages. I've never seen production code that would've benefitted from it. - As much as the ship has sailed with libstd functions hiding allocations via the global allocator where you may not expect them, the less new cases get added the better. So I'd rather not have `impl Add<&'_ str> for &'_ str { type Output = String; }` added to libstd. - If you really do have two str literals that you want to concatenate, just use `concat!()`. Not only does it work, but also it works at compile-time and without any allocation (it evaluates to a `&'static str`).
- Animats 5y agoDeadlock detection is an interesting problem. As the author points out, Rust will not detect this at compile time, but there are run time analyzers. It's worth looking at deadlock detection at compile time today. Not just to prevent bugs. Sometimes, on some CPUs, the compiler could potentially turn some mutexes into fence operations and become "lock free". In Rust, code tends to do a lot of scope-based locking. There's a lot of item.lock().unwrap().dosomething() Now, if the compiler detects that 1) every lock on item is scoped, and 2) everything done inside the lock is short and non-blocking, then that lock could potentially be optimized into fence instructions. Worth thinking about. There's an analogous case for RefCell items. RefCell is a lot like Mutex, except that you call "borrow" instead of "lock", and if the item is ever "busy", waiting won't help, because your own thread made it busy. If every use is a scoped borrow and there are no inner borrows of the same object within the scope of the outer borrow, that's the good case. Reference counts aren't needed at all. I think that somewhere in this space lies the solution to the back-reference problem, but I haven't gotten there yet.
- masklinn 5y ago> Worth thinking about. Probably not. This would have to be extremely limited and restrictive in order to avoid false positives (which would be outright miscompilations). The more restrictive an optimisation is, the less valuable it is, and the lower the ROI. This would be an unlikely-to-achieve-anything optimisation around atomics, which compilers already don’t bother much with because they’re hard to get right and have a low ROI.
- gpderetta 5y agoI don't see how fences are ever enough for mutual exclusion. Maybe you are thinking of transforming a mutex in a spinlock but that would be catastrophic for realtime programs and generally an optimisation that compilers shouldn't make.
- masklinn 5y agoWhat I assume they’re thinking of is, if the compiler can determine that the lock is scoped and the scoped code only reads from shared memory, then it only needs a sequence point (a read barrier). I don’t think it’s true because if you don’t “hold” the mutex there’s no guarantee that the mutex-protected data is consistent, unless it’s atomic at which point you’re probably better off optimising to an atomic read. Which compilers are unlikely to bother with either way, because optimising atomic operations would be both risky and low-return.
- demindiro 5y agoSeeing the RwLock deadlock surprised me. I tried making the standard library's implementation deadlock instead and I'm unable to reproduce it (even with 1 million iterations). The parking_lot implementation does seem to deadlock however. I suppose the parking_lot implementation elides some checks for the sake of performance. EDIT: It seems it can happen with the standard library's RwLock too[1]. Perhaps it's a Linux peculiarity? [1]: https://doc.rust-lang.org/std/sync/struct.RwLock.html https://doc.rust-lang.org/std/sync/struct.RwLock.html
- masklinn 5y ago> It seems it can happen with the standard library's RwLock too[1]. Perhaps it's a Linux peculiarity? I would not call it a “linux peculiarity” as any system could have any priority policy and could change it unless explicitly guaranteed, which this rarely is.
- codeflo 5y agoThese things are often tricky to reproduce, but the deadlock should happen from the spec, right? Essentially, a pending write lock should block new read locks from being acquired -- this is a feature, if it didn't do that, the write lock would starve -- so if you try to acquire the second read lock while the write lock is pending, you get a deadlock.
- masklinn 5y ago> These things are often tricky to reproduce, but the deadlock should happen from the spec, right? Pretty sure the spec is entirely silent on the subject. > Essentially, a pending write lock should block new read locks from being acquired -- this is a feature, if it didn't do that That is an implementation detail of the lock. I don’t think either POSIX specifies the bias of its rwlock, and Rust explicitly documents that it does not and delegates to the OS. Plenty of RWLock will, in fact, starve writers under heavy contention (both read-bias and write-bias have their pros and cons).
- codeflo 5y ago
- JoshTriplett 5y agoWhile Rust won't catch the use of subtraction in an arbitrary function named "add", it'll catch (in clippy) the use of subtraction in an implementation of the Add or AddAssign traits: https://rust-lang.github.io/rust-clippy/master/#suspicious_arithmetic_impl https://rust-lang.github.io/rust-clippy/master/#suspicious_a...
- majewsky 5y agoThis is one of those lints where there's probably an entire backstory to it.
- cesarb 5y agoProbably not much of a backstory, it's common to make copy-and-paste mistakes when implementing multiple similar things. For instance, when implementing both the Add and Sub traits, it would be normal to start with one of them, and duplicate it to make the other, making the relevant changes in the copy (Add -> Sub, add -> sub, + -> -); it's easy to forget to make one of these changes.
- nsajko 5y agoThis sounds horrid.
- bitcharmer 5y agoWhy?
- phibz 5y agoIsn't subtraction just addition in the negative direction? :-)
- astrange 5y agoYou can subtract two pointers, but you can't add them.
- Dylan16807 5y agoSame with temperatures and timestamps.
- cozzyd 5y agoSometimes you can add temperatures (e.g. noise temperatures), obviously only if using Kelvin.
- rrobukef 5y agoNo, addition is commutative and associative, subtraction isn't.
- bjarneh 5y agoWow, what a horrid and cool way to create a function that accepts generic types in Go :-) func add(a interface{}, b interface{}) interface{} { if a, ok := a.(int); ok { if b, ok := b.(int); ok { return a + b } } if a, ok := a.(string); ok { if b, ok := b.(string); ok { return a + b } } panic("incompatible types") }
- rob74 5y agoYou're a bit (~10 years) late to the game of complaining about having to use `interface{}` as a substitute for generics in Go (although it has worked pretty well in most cases). Go 1.18 will have "real" generics, so stuff like this can be expressed more elegantly...
- bjarneh 5y agoI've been writing Go since 2009, so I'm not late to the party. I was there when it started :-)
- Cthulhu_ 5y agobrouhaha go bad. Why are you writing an add function when 1 + 1 will do? I mean I know there's better examples out there, but in practice I've not missed a lack of generics in Go yet. Mind you, I'm not writing libraries for e.g. json parsing. Actually there is one place in my application where I use interface{}, it's a bit of code that converts values to XML strings, but it needs different logic for different types. I don't think generics will solve that: https://go.dev/play/p/TtwyNZ7oOPx https://go.dev/play/p/TtwyNZ7oOPx
- jatone 5y agoi think constraints will help which are part of generics at least you'll be able to limit the types your functions accept. but otherwise no, generics do not solve sum types by themselves.
- mseepgood 5y ago
- iainmerrick 5y agoThis is a very long post! The bit that actually addresses the title, after all the "here's a thing I tried, here's the weird error message, here's the mistake I made" business, starts way down towards the bottom, with this text: Before we move on, I'd like to congratulate you on reading this far. Believe it or not, you're in the minority! The title is very misleading when the content is a very, very general discussion of compile-time versus runtime errors, mostly focusing on problems that Go doesn't catch at compile time. TL;DR: the mistake Rust doesn't catch is recursive mutex locks. It does seem like it would be good to catch that one, and that the borrow checker in some form ought to be able to do it -- it should be able to track the static scope of a mutex lock, and block further attempts to lock it inside that scope.
- hermanradtke 5y agoThis is a typical fasterthanlime post. It is anti-TL;DR as he takes you on the journey from the very beginning. It is through this journey that we can appreciate how good the Rust compiler is in many cases, but also its limitations (recursive mutex locks).
- fasterthanlime 5y agoAs others have noted, that's just how I write articles. More specifically, I have an end goal in mind, and to get there I need to establish a scene: make sure that we share a lot of context so that we can consider a problem together from the same angle. Most articles don't do that, and so they're "short and to the point", but only to a tiny audience that already has the same shared culture. There's always readers complaining that they knew half of it and so I "wasted their time", but they forget about the other half, and also other readers. Ideally, to please everyone, I would have to write a tailored article /just for them/, filling just the gaps they have. I do that on occasion, but I charge a lot of money for it :)
- iainmerrick 5y agoAbsolutely -- sorry if I came across as overly negative! I was trying to stay reasonably neutral. Your approach isn't one I'm keen on but I can see your subscribers get a ton of value from it.
- nullc 5y ago
- zkldi 5y agoBeen a firefox user forever and have never seen anything like this. What do the panics say? Have you reported this?
- nullc 5y agoI'd be happy to talk all about it, but my comment was flagged and so few other people are likely to see the discussion.
- fortran77 5y ago
- fastball 5y agoPretty sure Firefox doesn't use Servo.
- marcosdumay 5y agoAt least not on any non-experimental release.
- K0nserv 5y agoRust will catch problems that would be UB in C and would continue executing in an invalid state. Crashing is preferable in such scenarios because you avoid potential security issues stemming from the UB and the developers can find out about the problem. I bet Rust programs crash more frequently than C programs overall. Undetected UB is far less desirable than clean panics though.
- codeflo 5y agoNear the end, the blog posts mentions patterns to avoid accidental deadlocks in structs that wrap a Mutex. Here's another one that I like use: struct FooInter { actual_data: HashMap<...>, } impl FooInner { fn do_something(&mut self) { ... } } pub struct Foo(RwLock<FooInner>); impl Foo { pub fn do_something(&self) { self.0.write().do_something(); } } The rule is that FooInner doesn't know about the lock (so calling other methods is safe), while Foo methods can't call other Foo methods (so you can't lock anything twice). You can even move Foo and FooInner to different modules to hide FooInner's contents from Foo, though I rarely find that necessary. I know this is subjective, but for me, this works better than what the blog post suggests -- at least, I haven't gotten it wrong by accident yet.
- couchand 5y agoKudos! I've been successfully applying this pattern in a bunch of places and am happy to report that it is very useful and easy to reason about. It works well in any context where your object access is mediated... Mutex & RwLock, Rc and Arc, etc. I can recommend the ambassador crate to go with this pattern -- it can be very useful for avoiding boilerplate for the newtype struct. Really the hardest part is naming... I've also settled on FooInner, but I'm still not entirely happy with that convention.
- jerf 5y ago"I guess that's why the Go compiler is so fast - it barely checks for anything!" Any serious Go programmer ought to have golangci-lint or equivalent running somewhere. I prefer pre-commit, but CI/CD will do too. An advantage of golangci-lint is that is not in the core compiler, so you can get some really opinionated linters from the community, some of which you're going to love and some of which you're going to hate, or linters written really quickly and hammered out in the real world rather than waiting for the full release cycle. A disadvantage of golangci-lint is that it's not in the core compiler, so you have to actually fetch it yourself, maintain it (configure it), etc. With golangci-lint, Go is still not Rust, of course, but it gets it a pretty decent way towards being a serious, quality language. Most languages need some augmenting to be serious, quality languages, because it isn't generally appropriate to put all the requirements for code into the compiler, for many reasons. Rust is somewhat exceptional in how much they've put in their compiler. Things like C and C++ need immense, high-quality support from static analyzers to be even remotely safe to use on a network. (So, yes, I don't consider C or C++ "serious, quality" languages in the absence of that static support. I don't even consider it close, honestly. I don't consider very many languages to be serious, quality languages out of the box.)
- roblabla 5y agoI suspect you know this, but for anyone not aware: Rust's equivalent to golangci-lint is Clippy, which is distributed as part of Rust (at least, with rustup, it comes with the compiler in the default configuration). You have to run it separately, but it contains a ton of lints[0]. The default configuration is pretty good (Most of the warnings I've got with it were either bugs or nudging me towards better APIs to do what I wanted). And what's really nice about clippy is that it re-uses a lot of the compiler's infrastructure instead of having to duplicate it, so it has the same high-quality error messages, and stays in sync with rust's new language features. [0]: https://rust-lang.github.io/rust-clippy/master/index.html https://rust-lang.github.io/rust-clippy/master/index.html
- zozbot234 5y agoI don't think that was intended as a serious remark, because the compiler-checks part of Rust compilation is not that slow. Rust compiles are slow when generic monomorphization or macros generate a lot of code to be compiled, or when complex macro packages are used as a kind of (slow) compile-time computation. The former can probably be addressed by some version of non-MVP const-generics; we will likely be able to "forward" implementations of generic functions and traits to code that only has to be built once for each set of type-derived parameters.
- doteka 5y agoI’d just like to take a moment to compliment the author on repeatedly producing content that is deeply technical yet very entertaining. As someone who has used both Rust and Go in anger, I enjoyed this one and it’s predecessors a lot. A big step up from a lot of the typical blogspam.