8 ms·

Both critical bugs are integer overflows. It's unclear to me why our languages still default to modulo arithmetic semantics. I feel Rust had a chance to fix thi

• shepmaster 4y ago
  > Rust had a chance to fix this, but also dropped the ball. By default, a Rust project will panic on integer overflow in debug builds and will overflow on release builds. Two key points to note, however: 1. You can change the setting so that your project panics in release or overflows in debug mode. 2. We reserved the right to change the default at some point in the future. This will probably be widely communicated before it ever happens, and last I heard we are still waiting for the cost of performing those checks to be "reasonable" before thinking about making such a change.
  • kibwen 4y ago
    Furthermore, because integer overflow is defined behavior, the integer overflow is never considered a root cause in Rust. In order for an integer overflow to express as UB in Rust, you'd have to use it in conjunction with an `unsafe` block that was failing to ensure its invariants, and that would be considered the root cause. If you're not using `unsafe`, then an integer overflow is at worst a logic bug.
    • p4l4g4 4y ago
      A logic bug can be dangerous too though. E.g. Bumping a user ID, to get a "fresh" one or calculate port to open based on offset. When not bounded to a known range, this kind of logic can easily pose a serious security risk. Most of the time, it will probably just work, but under extreme conditions, it will fail. If your language at least catch the overflow and crash instead of wrapping around, you "only" have a denial of service. Can imagine that implementing bounds checking can be costly, when done in software. Wonder if there are any hardware improvements that could reduce risk in this area.
      • wongarsu 4y ago
        If you identify an area as risky, it's trivial in Rust to do a checked_add or saturating_add. The challenge is obviously identifying this, but having easy library functions anectotally leads to people looking for it in code reviews.
      • kibwen 4y ago
        Indeed, nobody ever said that logic bugs were good, but as a category of flaw it means that integer overflow in Rust isn't particularly interesting compared to all the other innumerable ways to introduce logic bugs. And I say that as someone who wouldn't really mind if the behavior was changed to panic-by-default in release mode.
    • astrange 4y ago
      A logic bug can be just as bad as any other kind of bug. Security bugs/memory corruption don't always deserve the extra special treatment they get, nor are they the only kind of remotely exploitable issue.
  • CorrectHorseBat 4y ago
    > I heard we are still waiting for the cost of performing those checks to be "reasonable" before thinking about making such a change. What I don't understand is, checking for integer overflow is extremely cheap in hardware, so why is there any cost for performing those checks? What am I missing?
    • kelnos 4y ago
      Presumably the program has to check bit in a status register or something like that to tell if the previous instruction caused overflow, no? That means an extra branch after each arithmetic instruction. I imagine that's not cheap?
    • dahfizz 4y ago
      Integer arithmetic is a significant part of ~every program. A single branch that checks the overflow flag is not expensive. But branching on that flag every time you do integer math is death by a billion paper cuts.
      • CorrectHorseBat 4y ago
        Your could use interrupts, no? Basically free when not triggered and when triggered you probably don't care about performance anymore.
        • prbs23 4y ago
          Most architectures do not provide an interrupt that is generated by an integer overflow. Since this would be a significant architectural change in the hardware, it can't be simply added in. Additionally, if you are running inside an operating system, handling an interrupt usually incurs a trip through the kernel, which would add extra overhead every time an overflow did happen. Since there's a lot of software which depends on integers overflowing, this overhead on each overflow could significantly impact legacy software.
          • CorrectHorseBat 4y ago
  • travisgriggs 4y ago
    Fascinating. Haven't had a chance to do Rust yet, but I think I would change this so that they were consistent. I do embedded, and that kind of "behaves differently in different places" is the worst kind of bug to figure out.
    • kibwen 4y ago
      > behaves differently in different places It doesn't behave differently in different places, it differs based on build profile. You test in debug mode, which is where overflows will panic, which makes them quite obvious. If you want to pay the price of overflow checks everywhere in release mode, then you can turn them on there as well (it's not that much of a performance penalty, but that might not be true on embedded...). It's effectively just a compiler flag.
  • eklitzke 4y ago
    Just in case people don't know, you can get the same behavior with C or C++ by invoking GCC or Clang with -ftrapv. I don't know about Rust, but for C and C++ -ftrapv will only fault on signed integer overflow, as signed integer overflow is UB in C/C++ but unsigned integer overflow is well defined (it's guaranteed to wrap around to zero on all platforms). So even if you're trapping on integer overflow, there are still plenty of weird things that can happen if you unintentionally overflow an unsigned integer.
• xxpor 4y ago
  >It's unclear to me why our languages still default to modulo arithmetic semantics. Because that's what processors do? (leaving aside backwards compatibility issues)
  • hyperhopper 4y ago
    Our processors also require manually manipulating registers. The whole point of higher level programming languages is to abstract away the fiddly bits of dealing with processors that we don't want to have to deal with. This is one of those cases.
    • robmccoll 4y ago
      In this case it's really that the cost of determining if an overflow did occur or will occur on modern architectures is too high and the likelihood too low for it to be reasonable to perform the checks in most cases in native code. Might be different for interpreted languages depending on a lot of things (whether or not they even use integer arithmetic, whether or not they default to some arbitrary precision integers by default, etc.). If common architectures automatically interrupted on overflow rather than setting a flag at no additional cost, I'd think you'd see safety guarantees instantly.
    • xxpor 4y ago
      In cases where you're willing to take the perf hit, you can just use languages like Python which abstract over integer size entirely.
      • hansvm 4y ago
        Which used to, but at least for parsing ints they've snuck in the perf hit as a "security vulnerability."
  • FatActor 4y ago
    Saturated arithmetic instructions do not do this.
• sshine 4y ago
  Rust does have a fix for this: error: this arithmetic operation will overflow --> src/main.rs:2:18 | 2 | let a: u64 = u64::MAX + 1; | ^^^^^^^^^^^^ attempt to compute `u64::MAX + 1_u64`, which would overflow | = note: `#[deny(arithmetic_overflow)]` on by default Rust also allows for overflowing arithmetic (preserving the default to fail): https://doc.rust-lang.org/std/?search=overflowing https://doc.rust-lang.org/std/?search=overflowing It's generally less ergonomic, e.g. let (zero, _did_overflow) = u64::MAX.overflowing_add(1);
  • howinteresting 4y ago
    You'll get a compile error when rustc can statically prove that it'll overflow (as in your example above). That is generally not possible. The correct answer is what shepmaster said in a sibling comment.
  • deathanatos 4y ago
    Edit: Gah, I'm a bit wrong too. There's the compiler error (this), and the runtime error (what I'm talking about below.) Here's a link to the runtime variant: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=6e1c0a18458154610ba8ac83065ea97c https://play.rust-lang.org/?version=stable&mode=debug&editio... As a sibling notes, currently, this is for debug builds. So, if you change that playground to "Release", you'll see it wrap. (I love this feature, and I wish they had done it in release mode too. The sibling comment has some notes on that, too.) (But, e.g., were `git` written in Rust, presumably the end product would be a release build. Now, you can enable the check there, but that is something you have to do, today.) (But also note, that, in all cases, it's well-defined. Vs. C, where some overflows are UB.)
  • hypeatei 4y ago
    Slightly related, I wonder why the return type for `overflowing_add` isn't `Result<T>` and instead a tuple containing a boolean?
    • re 4y ago
      There are times when you want to know how much overflow occurred -- think of the way you learn to do multi-digit addition. There is a checked_add that returns an Option<T> if you only care about success/failure.
• kgeist 4y ago
  I'm quite paranoid about integer overflows, so in my hobby projects I now have a habit of always using helper functions (which generate an error on overflow) instead of "bare" math operators, and whenever I see a bare math operator without any checks in an open source project (and from what I've seen almost no one checks for overflows) I wonder whether they thought about potential consequences or I'm being too paranoid
  • sshine 4y ago
    My contribution to two open-source projects in recent years has involved a transition to the use of safe arithmetic, too. I think it makes a lot of sense to think about. Ultimately, it matters more in some applications than in others.
    • soiler 4y ago
      Can you explain this as if I were a programmer who doesn't know what that looks like?
      • favorited 4y ago
        There are different ways to design it, but a simple version could be a function that takes 2 operands and 1 result pointer as inputs, and returns boolean (true if success, false if it overflowed): bool SafeAddIntInt(int32_t x, int32_t y, int32_t *r); so the caller could say int32_t result; if (SafeAddIntInt(x, y, &result)) { // do something with result } else { // handle overflow } An even simpler version could just abort on over/underflow.
        • tsimionescu 4y ago
          Note that there is no such thing as integer underflow. INT_MIN-1 is an overflow just as much as INT_MAX+1. Only floats can suffer from underflow, which happens when you want to represent a number whose absolute value is smaller than the floating point precision can allow (e.g. trying to represent 1/2^32 in a 32-bit float).
• jcranmer 4y ago
  Rust makes integer overflow panic in debug builds, so Rust code is effectively required to opt into overflowing operations for correctness reasons. It disables those checks on release builds for performance reasons, but as sibling comments point out, it reserves the right to change that behavior. Unfortunately, there is a circular dependency here. Languages are reluctant to make integer overflows error conditions because there is a moderately high overhead to checking overflow conditions constantly, and processors (and compilers) are unwilling to make overflow checks cheaper because they benchmarks they care about don't do such checks.
  • deckard1 4y ago
    That sounds like the similar, but opposite case of tail recursion optimization. Some languages/compilers don't do it because devs want stack traces. But allow TCO in and now the code that gets written is quite different than the code that would not do tail calls because TCO doesn't exist. Also a surprising amount of undefined behavior gets relied on in code. I don't use Rust, but the idea that they could potentially change the future behavior on overflow seems... risky?
• dcsommer 4y ago
  Integer overflow isn't a security issue unless your program's memory safety depends on the correctness of the integer operation. Safe rust doesn't (in any build mode), but C/C++ does.
  • shiftingleft 4y ago
    To elaborate on this: Rust always performs bounds checks on array accesses, so you can't get an out-of-bound read/write.
    • _8dej 4y ago
      Is there a way to turn this off?
      • kibwen 4y ago
        Not via a compiler flag, no. The way to "opt out" of bounds checks is to replace `foo[bar]` with `unsafe { foo.get_unchecked(bar) }` at a given callsite. And the use of `unsafe` is going to immediately raise the eyebrow of any code reviewer or auditor.
  • hyperhopper 4y ago
    You don't know the business logic of every program. You can't say that a rust program won't have a security issue due to this. `UserAccessLevel > Threshold` Like there could be a million ways an integer becoming small could mess up something. Also there are business logic issues as well
    • HideousKojima 4y ago
      Sure, but a logic error is a fundamentally different class of error compared to a memory error. The potential harm of a logic error is limited in scope to what the program was written to be able to do. A memory error can lead to arbitrary code execution.
      • 4y ago
• topspin 4y ago
  > I feel Rust had a chance to fix this Don't see how. Given the hardware Rust is designed to program you have to compromise some or all of efficiency, memory usage and complexity to solve overflow.
  • viraptor 4y ago
    Rust can guarantee some things about collections which are not possible in C, so a lot more range checks and overflow checks could be omitted. Together with actually having the saturating/overflowing/checked adds, this makes the whole thing a lot safer and easier to deal with where you need to.
    • topspin 4y ago
      Great. Tell bbojan how Rust didn't actually "drop the ball" then. When you do you'll be told about all the different ways Rust doesn't actually solve overflow. And those claims will likely be correct because they're self evident. My point -- my sole point -- is that Rust is like it is because none of the alternatives are viable for Rust; Rust must run efficiently (as a "systems" level language use defines "efficient") on hardware that silently wraps words. Rust can't fix that and still be Rust. There is no ball to drop.
• Kon-Peki 4y ago
  I'm wondering if what you are asking for is what Swift does - overflow kills your program, but you can opt into allowing it by using "Overflow Operators" (&+, &- and &*). This crashes in Swift var potentialOverflow = Int16.max potentialOverflow += 1 This does not crash var potentialOverflow = Int16.max potentialOverflow &+= 1 [1] https://docs.swift.org/swift-book/LanguageGuide/AdvancedOperators.html https://docs.swift.org/swift-book/LanguageGuide/AdvancedOper...
  • LegionMammal978 4y ago
    As others have mentioned, the default overflow behavior in Rust can be configured to panic. To explicitly wrap around on overflow in every configuration, you can use the newtype wrapper Wrapping, or use the wrapping_add(), wrapping_mul(), etc. methods on the basic integer types. There are also variations such as saturating_op(), checked_op(), and overflowing_op() to detect the overflow and handle it appropriately.
    • Kon-Peki 4y ago
      Sure, I was suggesting that what the OP is asking for is not what Rust does, but what Swift does. This is not a configuration thing, if you don't want a runtime exception on overflow, you must use a different arithmetic operator. Swift's behavior comes at a cost - it is not exactly the fastest language out there ;) Another no-overflow oddity is that Swift doesn't have a rand() equivalent. You can't get fast psuedorandom numbers in Swift unless you are on the Mac, in which case you can import GameplayKit and get gaming-appropriate pseudorandom numbers. EDIT - to be clear, I am not suggesting that anyone change their own chosen programming language. But if you'd like, install Swift on your dev machine and make a Swift implementation of the critical section of your Rust code. Debug, optimize, tweak, etc. And you'll get a pretty good idea of what kind of performance you have to give up to do what many people are asking :)
• weinzierl 4y ago
  Because saturating math is not "more right", just "different wrong". The "right" way of checking an error condition after every integer operation is prohibitively expensive. From the language side, what I wish for is a sort of NaN for integer operations. I would not want to check for overflow on every operation, but I would want to know after a couple of them if somewhere an overflow had occurred. On the hardware side this could be done with a sticky overflow bit, which some architectures already support. I think the ball is on the hardware side and in my opinion Rust did the most sensible thing possible with contemporary hardware.
• nine_k 4y ago
  I wonder if using 64-bit integers all over the place would alleviate this a bit. If your integers represent some real quantities (sizes of objects, etc), the sizes have to be unrealistically huge to trigger an overflow. If your integer is a counter, it would take years to increment it in a tight loop to achieve an overflow. The cost of operating on 64-bit integers is about the same as operating on 32-bit integers on most modern CPUs (except maybe 32-bit cores in MCUs).
• hot_gril 4y ago
  The funniest is how Solidity does this. The language focused on transfers of money.