8 ms·
The Git source code audit, viewed as a Rust programmer
- throwaway81523 4y agoThis says again that the right behaviour for int overflow should be to trap, rather than wrap around or go into the weeds with UB.
- nextaccountic 4y agoI think the Rust approach of having different methods for wrapping and checked addition is excellent. If you have two i32, x.wrapping_add(y) [0] is a sum where you have wrapping semantics. x.saturating_add(y) [1] is a saturating sum (when it reaches the max it stays there instead of overflowing), nd x.checked_add(y) [2] is a sum that catches overflow. If you are doing wrapping arithmetic a lot, there's a convenience wrapper (heh), Wrapping [3], so that if x and y have type Wrapping<i32>, then x + y always wraps (like calling wrapping_add on two i32). There's a Saturating<i32> too but it's nightly-only for now. The only downside of Rust's approach is that if you have x + y on two plain integers, the language itself doesn't specify whether it will panic (and interrupt the execution of the program) or not. By default, debug builds will panic and release builds wraps (on the understanding that unintended overflows are always a bug, but catching this bug on release builds has some performance penalty, so it isn't caught by default on release), but you can control this by setting overflow-checks = true on the release profile on Cargo.toml [3] I think future versions of Rust should set overflow-checks = true by default even on release builds, but right now it's only a matter of adding a single line on Cargo.toml to enable it, and almost every program should do it. With this on, x + y will panic on overflow if they are i32, will wrap if they are Wrapping<i32> and will saturate if they are Saturating<i32> [0] https://doc.rust-lang.org/std/primitive.i32.html#method.wrapping_add https://doc.rust-lang.org/std/primitive.i32.html#method.wrap... [1] https://doc.rust-lang.org/std/primitive.i32.html#method.saturating_add https://doc.rust-lang.org/std/primitive.i32.html#method.satu... [2] https://doc.rust-lang.org/std/primitive.i32.html#method.checked_add https://doc.rust-lang.org/std/primitive.i32.html#method.chec... [3] https://doc.rust-lang.org/stable/std/num/struct.Wrapping.html https://doc.rust-lang.org/stable/std/num/struct.Wrapping.htm... [4] https://doc.rust-lang.org/cargo/reference/profiles.html#overflow-checks https://doc.rust-lang.org/cargo/reference/profiles.html#over...
- brundolf 4y agoIt's off by default in production because all those overflow checks add up (hah). Rust's first priority has always been low-level/systems code, where these things matter. It (semi-controversially) makes runtime bounds checks despite that priority, because they're so important to memory safety, but with that guard-rail numeric overflows become less important- they can only cause logic bugs, not safety issues, so it makes sense to me that performance would then be prioritized by default
- stouset 4y agoAlso, thanks to the language, most bounds checks can be elided in practice with idiomatic code. This isn’t as often the case with arithmetic operations. I do wish there was a(n optional) clippy lint to warn against non-explicit integer arithmetic operations though.
- twic 4y agoWhat do they add up to? You can turn overflow checks on in release mode with -C overflow-checks=on, so it feels like it should be really easy to benchmark the current impact. I couldn't find any benchmarks doing this, though.
- brundolf 4y agoI'm sure it varies wildly by the type of program, and I'm sure much smarter people than me on the Rust team did their benchmarking homework when they originally made the choice. Just saying that to me, it seems like a reasonable default
- throwaway81523 4y agoIt seems like a seriously wrong default for a language whose story is supposed to be safety. People wanting unsafe languages already have C and C++ for that :/.
- 4y ago
- jeroenhd 4y agoI agree from a language standpoint but trapping overflows by default would be quite costly. If you get the branch predictor involved with every addition or multiplication then you'll very quickly slow down your program, especially in the hot code paths. I think most general purpose programs don't run into overflow anymore these days, and the performance benefit probably outweighs the safety for most people.
- cerved 4y agoI think the opposite is true, most software projects benefit from safety and not performance
- throwaway81523 4y agoMost programs should be written in HLL's with unbounded integers and garbage collection. There are a few programs that need machine arithmetic and manual memory management, for performance or whatever reasons, and Rust is intended to allow writing those programs safely. So it seems like a mistake to not treat int overflow as a serious error. Idk if branch prediction or whatever would have to be involved. Most architectures will already trap on divide by zero or things like that, and will set a condition code for int overflow. Making the condition code sticky like it is for IEEE floating point arithmetic would allow doing a software trap once a basic block finished. It is a shame that the current trendy architecture (Risc-V) goes the opposite way and has no condition codes at all.
- mayoff 4y agoSwift does this. Bare operators like + and * trap (crash) on overflow. You can use &+ and &* for 2s complement wrapping.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- proto_lambda 4y agoSome of these comparisons are a little lackluster - sure, the issues may be relatively easy to reproduce if you translate the C code 1:1 to (unsafe) Rust, but you wouldn't even have such code in the first place if you wrote idiomatic Rust: - Confusing different integer types is much harder not only because the type checker will complain, but also because you don't really come across anything other than usize/isize unless it's a field in a protocol or something. - Strings in rust are not zero-terminated, so that specific example might not even happen at all. - Integers, pointers and references are not implicitly cast to booleans, so you can't accidentally mix up comparing a pointer to NULL (which is only ever required for FFI calls) and comparing an integer to 0. - You have proper types to indicate that a function encountered an error, so all the issues stemming from checking the integer return code the wrong way (e.g. interpreting an error value as a return value) don't exist. - You simply don't have to invoke potentially dangerous functions like `memcpy` because there are abstractions that take care of this safely and more intuitively. That's not to say that idiomatic Rust code wouldn't come with its own problems, but they tend to be much more concentrated to fundamental logic bugs rather than silly avoidable things like overflows or misinterpreting values.
- nextaccountic 4y ago> - Confusing different integer types is much harder not only because the type checker will complain, but also because you don't really come across anything other than usize/isize unless it's a field in a protocol or something. For me it's the opposite: usize is only used for indexing arrays, isize only used for memory offsets (which is a really niche use case outside of low level code), and most application code should be written with u32, u64, i32 and i64. Why is that? Because usize has a platform-dependent size and generally the logic of the program should be platform-agnostic. So you often need to index something based on an u32 (or something) and then there's a cast. How do avoid it? Well first off, your own types can implement Index<u32> instead of Index<usize>. But better yet, you can have an Idx type that represents an index, and then implement Index<Idx>. That's seen for example in the petgraph API [0], you can index an graph with either a node index [1] or an edge index [2] [0] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.... [1] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html#impl-Index%3CNodeIndex%3CIx%3E%3E https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.... [2] https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph.html#impl-Index%3CEdgeIndex%3CIx%3E%3E https://docs.rs/petgraph/latest/petgraph/graph/struct.Graph....
- justahuman74 4y agoIt'll be interesting to see where the gitoxide[1] project goes, being a rust implementation of git [1] https://github.com/Byron/gitoxide/ https://github.com/Byron/gitoxide/
- ironmagma 4y agoI wouldn’t hold my breath. The scope of Git is quite large and Git-Oxide is just getting started. What I’d like to see is the Git core team adopt Rust gradually for certain subprograms.
- johnny22 4y agothat would fit how git itself evolved. A fair amount of code was in perl and stayed that way for awhile before being written in C.
- cerved 4y agomany of the porcelain commands are or used to be in perl/bash
- deleted 4y ago[deleted]
- kibwen 4y agoI don't know how far he intends to take it, but the author of gitoxide (which is just a library, like libgit2) uses it to power a git CLI called gix. As of October it looks like it can do fetches and clones: https://github.com/Byron/gitoxide/discussions/623 https://github.com/Byron/gitoxide/discussions/623 . Since the git CLI is subcommand-based, it wouldn't be too hard to provide replacements for individual git subcommands one at a time. If you replaced git clone, push, pull, commit, checkout, and rebase, that would suffice for 95% of git invocations.
- davidkunz 4y agoGitoxide is useful in other Rust programs that need to interact with git such as Cargo[1] or Helix[2]. [1]: https://github.com/Byron/gitoxide/issues/106 https://github.com/Byron/gitoxide/issues/106 [2]: https://github.com/helix-editor/helix/pull/3890 https://github.com/helix-editor/helix/pull/3890
- jeroenhd 4y agoFor what it's worth, if you enabled all the useful warnings you'll be warned about all the implicit casts that can cause problems in any modern C compiler. You'll have to specify three or four flags that say "yes I want all the warnings" but you'll know when you do dangerous casts. Rust with its mandatory explicit casts is a lot less strict in this sense, your compiler won't complain because you decided to do casts yourself, which is why you should remember to be wary of simple casts every time you use them. Nothing wrong with casting constants and upcasting, but you have to double check because it's easy to introduce bugs this way. If you want to make full use of Rust's safety features, you may also want to use checked numeric operations rather than the standard operators that may under/overflow. Rust is a great tool for writing safer programs, but it won't defend you from many if not most DoS attacks if you're not careful
- kibwen 4y agoNote that if we're talking opt-in warnings, Rust's linting tool Clippy offers a whole bunch of warnings related to `as` casts. See https://rust-lang.github.io/rust-clippy/master/index.html#as_conversions https://rust-lang.github.io/rust-clippy/master/index.html#as... .
- zahllos 4y ago> Rust is a great tool for writing safer programs, but it won't defend you from many if not most DoS attacks if you're not careful Just want to stop by and point out "behavior not considered unsafe": https://doc.rust-lang.org/reference/behavior-not-considered-unsafe.html https://doc.rust-lang.org/reference/behavior-not-considered-... I have a simple phrase I use to explain this. Rust tries to give you memory safety. It does not imply nor check for correctness.
- stouset 4y agoNot sure why this was downvoted. I’m an unabashed Rust apologist and everything in here is reasonable. `as` casts are generally a code smell, but they’re sometimes the best (or only) way to accomplish a task. This can absolutely make it more challenging to identify correct and incorrect examples at a glance. And Rust is awesome in that you can use explicit numerical operations that wrap, panic, saturate, etc on overflow. But there is no clippy lint (that I know of) to make sure you use them (which I would love).
- voorwerpjes 4y agoI like how this highlights the pitfalls of using `as` to cast between types. The article says to prefer using `try_into()` instead of `as`, which is a good suggestion when available, but `as` lets you cast between floats and ints, while `try_into` is not implemented for these type conversions. I think the reasoning is it is unclear `try_into` should fail when it had to round a decimal number down or only when the float was something above the bounds or `NaN`. Regardless it seems like a missed opportunity in the standard library. https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=8688eebe3c1307035f106c8b915c44f0 https://play.rust-lang.org/?version=stable&mode=debug&editio...
- kevincox 4y agoFor floats you do end up having to use `as` which is unfortunate. However the amount of things that can go wrong is too vast for `TryInto`. The article also says "If you upcast u32 to u64, you can use the keyword as". I would avoid this. In these cases you can just use `.into()`. This is infallible and will raise errors if refactoring causes that conversion to no longer be infallible. (In which case you can fix it or migrate to `.try_into()`) I think the floating point case is similar. Keep `as` super suspicious, but sometimes you do want to do it.
- tialaramex 4y agoAlso, in generic code write try_into() and don't sweat it, for the cases which can't actually fail in the monomorphised results the error type is Infallible, that's an Empty Type (an enum with no variants, as distinct from a Zero Size Type which is a struct with no members) and so no error handling code is emitted. The compiler can't even imagine what the code should look like, after all this type can't exist, does it fit in a register? How do we perform operations on it? If we try to actually realise an empty type by e.g. dereferencing a pointer to it that somebody gave us from a C API the Rust compiler will reject that as nonsense.
- dathinab 4y agocargo clippy is good at yelling at you for using `as` I generally recommend starting with clippy set to pedantic and then disable all the lints which are too pedantic (or don't agree with, some lint style choices which are generally not grate but singing still got best option). Anyway `as` is one of that "is not perfect but we need something now" design choices and most of it's usage have now better choices". So maybe in the long future we will even see `as` depreciated and looked against in rustc without clippy.
- j4nek 4y agothank you for sharing this post. :) can somebody give an example for the line: int len = slen; // Here if slen is too big, it loops backs to 0 what happens if slen is to big? all bits in len are zero? (sorry for this noob question btw, just started with c programming) :)
- colonwqbang 4y agoAs a full time C programmer, implicit type conversion is probably my least favourite feature of C. Especially compared to other infamous aspects of the language. For all its drawbacks, unchecked memory access is a defining feature of C. Try to remove it and you end up with a language that is very clearly distinct from C, hence Rust, Java, etc. I think implicit conversation shuld just be removed from the language. The ergonomic benefits don't outweigh the harms.