11 ms·

Don’t assume that safety comes for free: a Swift case study

69 points by deafcalculus 10y ago

wlesieutre 10y ago
Site is down, cached here: http://webcache.googleusercontent.com/search?q=cache:MYG52quo1u8J:lemire.me/blog/2016/12/06/dont-assume-that-safety-comes-for-free-a-swift-case-study/ http://webcache.googleusercontent.com/search?q=cache:MYG52qu... EDIT: It's back up
iainmerrick 10y ago
This seems totally misguided and inflammatory. The points made boil down to: 1. Overflow checking adds about a 2x overhead to "+" in a loop 2. Using "reduce" rather than a loop also has ~2x overhead Neither is really surprising, and hardly seems to warrant the final tangent that "I do not think it is universally a good thing that software should crash when an overflow occurs" since "a mission-critical application could crash because an unimportant function in a secondary routine overflows". What about the mission-critical bugs, especially security bugs, that are caused by overflows? Swift's default behavior, of aggressively checking for overflows, is at least arguable, and there's a school of thought in software engineering that says "don't nail your code to the wall", i.e. don't try to keep it running at all costs when something goes wrong. Let it crash and restart it. Obviously that approach works well in some scenarios and less well in others. The creators of Swift reckon it's a good default for most programmers and I reckon they're onto something. The fact that the overhead is only 2x for a tight inner loop should be a cause for celebration, not dire warnings. And as the author himself shows, it's very easy to disable overflow checking on a per-operation basis when you want to.
- planteen 10y ago
  > The fact that the overhead is only 2x for a tight inner loop should be a cause for celebration, not dire warnings. And as the author himself shows, it's very easy to disable overflow checking on a per-operation basis when you want to. Agreed. This smells like premature optimization... on a problem that can be solved in O(1) time.
- the_af 10y ago
  Agreed that it's misguided, inflammatory and presented in the most flame-baity terms. I would have expected this from some random blogger dealing with an unfamiliar programming language, but from a supposed professor of computer science? Does he really expect sympathy when he considers something as trivial as array.reduce(0,+) "obfuscated"? I've never written a single line of Swift in my life and I -- being familiar with FP idioms -- immediately understand what it means. The premise that many abstractions and safety checks come at a cost is reasonable. Unfortunately the rest of his post isn't.
  - bluecalm 10y ago
    How is it inflammatory? If you don't like his opinions or writing style then fine but it's in no way inflammatory. He takes a stab at functional programming fashion, that's all there is to it. He also shows significant performance penalty in the default language construct. If anything it's an informative, if a bit opinionated, post.
    - the_af 10y ago
      It's inflammatory because his main point -- that abstractions come at a cost -- is buried in digs at FP, mock surprise, and claiming really simple things are difficult to understand, when they are not (and I know they aren't difficult to him either, so it's also mock ignorance). I don't see any deep insight. The performance loss may very well be caused by an immature Swift compiler, and not ultimately tied to the abstraction, but he mentions nothing of this. I expect this lack of understanding from a novice, not from a CS professor. What he wrote isn't even a "case study", contradicting even the title of his blog.
      - caconym_ 10y ago
        > It's inflammatory because his main point -- that abstractions come at a cost It's also not always correct. See the Rust benchmarks posted here. Someone seeing this article might think it is correct, though, and thusly misinformed, they might avoid "functional-style" programming in the future when in fact many languages manage to provide FP idioms with no performance penalty. It's irresponsible, really. If he wants to state his opinions on FP, that's all well and good, but he should make it clear that they are his opinions and that the performance penalty he saw in the fold case is Swift-specific.
- TickleSteve 10y ago
  Thinking out loud here, but... If you're going to take the overhead of range checking for a value, it may as well be a more appropriate range, not just for overflow conditions... Thinking it would be nice to be able to specify this as part of the type...
  - junke 10y ago
    Use Ada.
  - nickpsecurity 10y ago
    Exactly. This is what Ada and annotation-based verification do. Usually, there's bad input at one of the interface points (esp function calls or I/O) where some conditional checks can catch it. Not always but most cases.
- mikeash 10y ago
  I'm puzzled by how many people want their programs to just keep on trucking and try their best after something unexpected occurs. Crashing is not the worst thing your program can do. In fact, I'd argue that it's the second-best thing your program can do, right behind working correctly. Losing data is much worse than crashing, and corrupting data is worse still. Both are strong possibilities once your program's state is something unexpected.
  - arcticbull 10y ago
    I agree and disagree. What the author says is true, if a thoroughly irrelevant part of your program (let's say a drawing routine for UI somewhere) hits an overflow and crashes your app in the middle of taking a payment, or in the middle of surgery -- while that much more critical element continued to work perfectly -- that's not ideal. Not all parts of your app make sense to fail-fast. By enforcing this behavior even on peripheral parts of your app, you're taking away the decision making ability about what's important and what isn't from the system designer. I understand that position, though I still think on the whole explicitness is right. Being able to explicitly disable the check when you know what you want is the right balance, whether it's &+ in Swift or (better, imo) .wrapping_add() in Rust.
    - mikeash 10y ago
      Displaying corrupt data in the UI can be just as bad as saving it to disk, if the user actually reads and uses the data you display (and if not, why are you displaying it?). And since the UI almost certainly shares data with the rest of the program, bugs in the UI can corrupt the underlying data. Separation of concerns is great, but if you really want different parts of your system to have different reliability constraints, then you need to truly separate them, for example by running them in different processes. Running them in the same process but having some of the code be more lax about state because it's less important is just begging for horrible bugs.
      - lomnakkus 10y ago
        > Separation of concerns is great, but if you really want different parts of your system to have different reliability constraints, then you need to truly separate them, for example by running them in different processes. Running them in the same process but having some of the code be more lax about state because it's less important is just begging for horrible bugs. Indeed, and you might even want to run them on entirely different and independent systems (connected by some type of network/bus) depending on exactly what bits are critical vs. non-critical.
- AndrewStephens 10y ago
  I totally agree. The most pernicious defect I ever fixed was a result of a subtle overflow during part of a larger calculation. The erroneous result could cause problems months after the overflow occurred and took me days to track down. The fact that Swift defaults to checking for overflow should be treated as good news.
  - planteen 10y ago
    I had a nasty bug like that once as well. It was in some FPGA logic written in VHDL with an overflow on a FIR filter. Recreating the problem was very difficult since it only happened near max signal power with a noisy input. The correct solution there was to saturate, not wrap, which is almost always the case in DSP code.
- steveklabnik 10y ago
  > Neither is really surprising, It depends! Consider these three functions, in Rust: https://gist.github.com/steveklabnik/da82b51f9dbedd5321924927af091131 https://gist.github.com/steveklabnik/da82b51f9dbedd532192492... One uses a loop, one uses fold with a closure, one uses point-free style. On my machine, summing a vector of a million fives gives running 3 tests test sum_fold ... bench: 147,129 ns/iter (+/- 9,101) test sum_loop ... bench: 148,016 ns/iter (+/- 10,294) test sum_pointfree ... bench: 146,230 ns/iter (+/- 9,114) (there is some variance here between runs, sometimes the +/- for each changes a bit.) Cargo's bench profile doesn't use debug assertions, and hence, no overflow checks by default. Let's turn them on: running 3 tests test sum_fold ... bench: 147,638 ns/iter (+/- 12,654) test sum_loop ... bench: 143,485 ns/iter (+/- 7,255) test sum_pointfree ... bench: 143,050 ns/iter (+/- 7,549) Looks like they're being eliminated by LLVM anyway. Using godbolt, both the loop and the fold versions (I didn't bother to look at pointfree) are both getting vectorized, and while they're not _exactly_ identical, they're quite similar. (disclaimer: YMMV, I might have made a mistake, always inspect assembly for your situation, etc)
  - iainmerrick 10y ago
    Good point! You're right, it is a little surprising that reduce() adds any overhead at all in this trivial example. It's interesting that Swift can't seem to fully eliminate that abstraction overhead when runtime checks are enabled -- maybe there's something other than overflow checks that's getting in the way? The author hints that the abstraction cost does vanish when checks are disabled, but doesn't bother showing any numbers.
    - jckarter 10y ago
      Looking at the generated code, we have a couple issues that prevent full optimization. One of them I already knew about (https://bugs.swift.org/browse/SR-2926 https://bugs.swift.org/browse/SR-2926) -- as a debugging aid, we guard every trap we emit with the equivalent of an `asm volatile("")` barrier, because LLVM will otherwise happily fold all the traps together into one trap. We really want every trap to have a distinct address so that the debugger can map that trap back to the source code that emitted it, but the asm barrier prevents other optimizations as well. As for `reduce`, it looks like the compiler does inline away the closure and turn the inner loop into what you would expect, but for some reason there's more pre-matter validation of the array than there is with a for loop. That's a bug; by all means, reduce is intended to optimize the same.
- int_19h 10y ago
  > Using "reduce" rather than a loop also has ~2x overhead ... Neither is really surprising I disagree. It all depends on where you're coming from, and what your expectations are. In C++, using reduce (std::accumulate) with a lambda is normally as fast as using a loop, because the compiler will inline the lambda into accumulate, and then accumulate into the calling function, producing the same exact loop.
- deleted 10y ago
  [deleted]
- caconym_ 10y ago
  It's a little bit surprising that using a fold rather than a simple loop has such a high overhead. I was already pretty sure of the result, but I went ahead and wrote a quick benchmark in Rust, and there is no overhead to using a fold rather than a loop (each iteration is addition of a million values of the stated numeric type): test tests::bench_fold_f64 ... bench: 984,829 ns/iter (+/- 30,827) test tests::bench_fold_i32 ... bench: 84,497 ns/iter (+/- 16,109) test tests::bench_loop_f64 ... bench: 993,406 ns/iter (+/- 45,791) test tests::bench_loop_i32 ... bench: 81,306 ns/iter (+/- 17,815) That this isn't true in Swift is way more interesting to me than the fact that overflow checking has overhead (duh). It's not difficult to come up with some theories as to why this might be the case, but still, if using these "functional" constructs in Swift has such a high overhead, programmers should probably be questioning whether it's worth using them at all. edit: I see that I've been beaten to the punch posting a Rust benchmark :)
dispose13432 10y ago
How do languages which aim to replace C (such as Rust) deal with this issue? Now I agree, your average webapp won't see any benefit by removing checks and will see security features by keeping them in, so I'm all for it. But in OSs (or browsers), speed does matter. And there's no way to optimize it (every + or array operation involves an if). Is this just one of the "costs of doing business"?
- iainmerrick 10y ago
  You optimize by using "&+" instead of "+", or by compiling with "-Ounchecked". You figure out if you need to optimize by testing and benchmarking.
- maxpolun 10y ago
  Rust has overflow checking in debug mode, but release mode does not have any overflow checking. The thinking here is that you'll hopefully catch most of your overflows while testing the app, but get no performance penalty in production use. If you want to overflow there are special types that you can use.
- mikeash 10y ago
  In Swift, you have total control over this. If you want checks, use +. If you don't want checks, use &+. In Rust, the operators check in debug builds, and wrap in release builds. Calls are provided so you can always have them check, or always have them wrap, if you need it. Note that C has some really ugly behavior here. Signed overflow is undefined behavior, which means that the compiler doesn't need to check for it, but also that it can assume it never happens and optimized based on that. Many programmers don't realize this, or don't notice that they've accidentally written code that depends on overflow behavior, which can lead to many entertaining bugs: https://lwn.net/Articles/511259/ https://lwn.net/Articles/511259/
  - deleted 10y ago
    [deleted]
  - dispose13432 10y ago
lmm 10y ago
> That’s because I do not think it is universally a good thing that software should crash when an overflow occurs. Think about the software that runs in your brain. Sometimes you experience bugs. For example, an optical illusion is a fault in your vision software. Would you want to fall dead whenever you encounter an optical illusion? That does not sound entirely reasonable, does it? Moreover, would you want this “fall dead” switch to make all of your brain run at half its best speed? If you lived in a world where hackers crafted optical illusions that made you send all your money to them when you viewed them, you would probably want to go blind or some such when you encountered such an illusion.
- mikeash 10y ago
  A bit of a tangent here, but anyone intrigued by the notion of hacking the brain using optical illusions should read "comp.basilisk FAQ": http://ansible.uk/writing/c-b-faq.html http://ansible.uk/writing/c-b-faq.html
  - e28eta 10y ago
    Neat! Reminded me of Snow Crash, by Neal Stephenson. I'm sure nearly everyone is familiar with it, but for those who aren't: it's pretty good!
acqq 10y ago
I see a number of posts that proudly claim (in different forms) "fail early and fast" like it's a good thing to simply do a run-time crash always, and even that the article author is "misguided and inflammatory." I don't agree with both claims. Regarding the first, as an example where the crash is definitely not the solution, see the planteen's post: https://news.ycombinator.com/item?id=13117170 https://news.ycombinator.com/item?id=13117170 Or consider that once you use the computers to calculate the real life stuff (like the movement of your car, or even the spaceship 50 million kilometers away) the worst thing you can do is introduce the "fatal discontinuities" in the processing. Regarding the second, allow me to just roll my eyes. The politics is not allowed this week on HN, but the political approaches start to be used automatically. Please just write which his claim is wrong. Labeling is destructive.
- mikeash 10y ago
  A crash is never the right solution. It is, however, likely to be a better solution than whatever random default behavior you might get. Regarding the linked comment, if the default had been to crash on overflow (or whatever the FPGA equivalent might be, if there is such a thing) then the problem would have been apparent the first time it happened, and they wouldn't be describing it as "a nasty bug" in the first place. A crash would not have been the best solution there, but it would have been miles better than the wrap that they actually got. As far as real life stuff, my understanding is that it's pretty common to fail fast and design the system to restart quickly so it can get back to work. If your needs are more sophisticated then you'd need to actually analyze the problem and come up with ways to handle unexpected data gracefully, not just hope that default behavior from the compiler ends up doing something sensible for your problem.
  - z3t4 10y ago
    If 1-2 ms is too much to recover from a crash, you better have three or four backup systems written in different languages and running on different hardware. Yet there is no guarantee you will have 100% uptime. And in such a system, having data corruption would probably be far worse then a crash. One application where I think crashes are bad are computer games, for example if you are fighting a boss, it's better to get a few glitches then having the game crash.
Animats 10y ago
Now that more languages are checking for integer overflow, it's time for integer overflow exceptions to re-appear in hardware. DEC VAX machines had this, but C didn't use them. With the hardware doing the checking in parallel, there's no performance penalty. If you want wrap-around arithmetic (which is rare) you should have to write something like unsigned short i,j; i = (i + j) % 65536; which the compiler should optimize into a no-check add. This gets you the same answer on all platforms.
- gok 10y ago
  Do you find a shorthand for wraparound math (like Swift's &+) so bad?
c0ffe 10y ago
After chasing strange bugs when using dynamic languages like PHP and JavaScript that keep running by default when minor errors happen (PHP warnings, or undefined variables in JavaScript), I think its good that Swift priorizes safety rather than speed.
deleted 10y ago
[deleted]
sulam 10y ago
I found this statement funny: "Moreover, would you want this “fall dead” switch to make all of your brain run at half its best speed?" ...because this is actually how our brains work! We are much slower to process and respond to all sorts of stimuli (visual, auditory, conceptual [reading]) when it is contradictory. Think of the feeling you get looking at an Escher sketch.
xenadu02 10y ago
I don't get the same numbers he gets. The reduce version is the same speed as the simple for loop. He must have made a mistake somewhere.
bjourne 10y ago
Most costly software bug I've ever witnessed was caused by an integer overflow. Thankfully, it wasn't caused by me, but if I had been auditing the code, I probably wouldn't have found the bug. The system was billing customers credit cards depending on how long they had used the service. Time was measured in milliseconds (uh-oh!) for no apparent reason. Usage could have been measured in seconds or even days but someone thought it was good to be extra precise. And System.currentTimeMillis() returns milliseconds. The default charging period was 14 days which worked well. So the maximum number of milliseconds that could be charged (for someone who used the system the whole month) was 1,209,600,000. Then the company decided to change the period to every two months (60 days) instead to save money as there was a fixed cost added to every credit card transfer. Guess what 60 * 24 * 3600 * 1000 is? It's a number a bit bigger than 2^31 - 1 which is the most positive primitive integer value in Java. And the "totalDuration" variable had type "int". :) So totalDuration wrapped around which caused the system to retry the transaction over and over and debit customers hundreds of times more than what they really owed. The resulting fallout from that debacle was one of the reasons the company went bankrupt. Integer overflow checking could have saved them.
nkurz 10y ago
I thought it might be interesting to see how this effect changes with the size of the array being summed. How do the relative speeds change when operating out of L1, L3, and memory? Does the lower speed of memory access overwhelm the overhead of the overflow checking? $ swift build --configuration release $ cset proc -s nohz -e .build/release/reduce # count (basic, reduce, unsafe basic, unsafe reduce) 1000 (0.546, 0.661, 0.197, 0.576) 10000 (0.403, 0.598, 0.169, 0.544) 100000 (0.391, 0.595, 0.194, 0.542) 1000000 (0.477, 0.663, 0.294, 0.582) 10000000 (0.507, 0.655, 0.337, 0.608) 100000000 (0.509, 0.655, 0.339, 0.608) 1000000000(0.511, 0.656, 0.345, 0.611) $ swift build --configuration release -Xswiftc -Ounchecked $ cset proc -s nohz -e .build/release/reduce # count (basic, reduce, unsafe basic, unsafe reduce) 1000 (0.309, 0.253, 0.180, 0.226) 10000 (0.195, 0.170, 0.168, 0.170) 100000 (0.217, 0.203, 0.196, 0.201) 1000000 (0.292, 0.326, 0.299, 0.252) 10000000 (0.334, 0.337, 0.333, 0.337) 100000000 (0.339, 0.339, 0.340, 0.339) 1000000000(0.344, 0.344, 0.344, 0.344) Code is from https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2016/12/05 https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/... with modification to loop over the different array lengths. Numbers are for Skylake at 3.4 GHz with swift-3.0.1-RELEASE-ubuntu16.04. Count is the number of 8B ints in the array being summed. Results shown were truncated by hand --- I wasn't sure how to specify precision from within Swift. The execution with "cset proc -s nohz" was to reduce jitter between runs, but doesn't significantly affect total run time. The anomalously fast result for the L3 sized 'unsafe' 'unchecked' is consistent.
mcguire 10y ago
"That’s because I do not think it is universally a good thing that software should crash when an overflow occurs. Think about the software that runs in your brain. Sometimes you experience bugs. For example, an optical illusion is a fault in your vision software. Would you want to fall dead whenever you encounter an optical illusion? That does not sound entirely reasonable, does it? Moreover, would you want this “fall dead” switch to make all of your brain run at half its best speed? In software terms, this means that a mission-critical application could crash because an unimportant function in a secondary routine overflows." That is a ridiculous analogy. What if we replace "optical illusion" with "hallucination"? More importantly, what if there were some sort of middle ground between continuing on as if nothing happened on an error and crashing completely?
emodendroket 10y ago
A factor of three for addition is probably not really significant in most programs.

69 points by deafcalculus 10y ago

Site is down, cached here: http://webcache.googleusercontent.com/search?q=cache:MYG52quo1u8J:lemire.me/blog/2016/12/06/dont-assume-that-safety-comes-for-free-a-swift-case-study/ http://webcache.googleusercontent.com/search?q=cache:MYG52qu... EDIT: It's back up

iainmerrick 10y ago

This seems totally misguided and inflammatory. The points made boil down to: 1. Overflow checking adds about a 2x overhead to "+" in a loop 2. Using "reduce" rather than a loop also has ~2x overhead Neither is really surprising, and hardly seems to warrant the final tangent that "I do not think it is universally a good thing that software should crash when an overflow occurs" since "a mission-critical application could crash because an unimportant function in a secondary routine overflows". What about the mission-critical bugs, especially security bugs, that are caused by overflows? Swift's default behavior, of aggressively checking for overflows, is at least arguable, and there's a school of thought in software engineering that says "don't nail your code to the wall", i.e. don't try to keep it running at all costs when something goes wrong. Let it crash and restart it. Obviously that approach works well in some scenarios and less well in others. The creators of Swift reckon it's a good default for most programmers and I reckon they're onto something. The fact that the overhead is only 2x for a tight inner loop should be a cause for celebration, not dire warnings. And as the author himself shows, it's very easy to disable overflow checking on a per-operation basis when you want to.

planteen 10y ago
> The fact that the overhead is only 2x for a tight inner loop should be a cause for celebration, not dire warnings. And as the author himself shows, it's very easy to disable overflow checking on a per-operation basis when you want to. Agreed. This smells like premature optimization... on a problem that can be solved in O(1) time.
the_af 10y ago
Agreed that it's misguided, inflammatory and presented in the most flame-baity terms. I would have expected this from some random blogger dealing with an unfamiliar programming language, but from a supposed professor of computer science? Does he really expect sympathy when he considers something as trivial as array.reduce(0,+) "obfuscated"? I've never written a single line of Swift in my life and I -- being familiar with FP idioms -- immediately understand what it means. The premise that many abstractions and safety checks come at a cost is reasonable. Unfortunately the rest of his post isn't.
- bluecalm 10y ago
  How is it inflammatory? If you don't like his opinions or writing style then fine but it's in no way inflammatory. He takes a stab at functional programming fashion, that's all there is to it. He also shows significant performance penalty in the default language construct. If anything it's an informative, if a bit opinionated, post.
  - the_af 10y ago
    It's inflammatory because his main point -- that abstractions come at a cost -- is buried in digs at FP, mock surprise, and claiming really simple things are difficult to understand, when they are not (and I know they aren't difficult to him either, so it's also mock ignorance). I don't see any deep insight. The performance loss may very well be caused by an immature Swift compiler, and not ultimately tied to the abstraction, but he mentions nothing of this. I expect this lack of understanding from a novice, not from a CS professor. What he wrote isn't even a "case study", contradicting even the title of his blog.
    - caconym_ 10y ago
      > It's inflammatory because his main point -- that abstractions come at a cost It's also not always correct. See the Rust benchmarks posted here. Someone seeing this article might think it is correct, though, and thusly misinformed, they might avoid "functional-style" programming in the future when in fact many languages manage to provide FP idioms with no performance penalty. It's irresponsible, really. If he wants to state his opinions on FP, that's all well and good, but he should make it clear that they are his opinions and that the performance penalty he saw in the fold case is Swift-specific.
TickleSteve 10y ago
Thinking out loud here, but... If you're going to take the overhead of range checking for a value, it may as well be a more appropriate range, not just for overflow conditions... Thinking it would be nice to be able to specify this as part of the type...
- junke 10y ago
  Use Ada.
- nickpsecurity 10y ago
  Exactly. This is what Ada and annotation-based verification do. Usually, there's bad input at one of the interface points (esp function calls or I/O) where some conditional checks can catch it. Not always but most cases.
mikeash 10y ago
I'm puzzled by how many people want their programs to just keep on trucking and try their best after something unexpected occurs. Crashing is not the worst thing your program can do. In fact, I'd argue that it's the second-best thing your program can do, right behind working correctly. Losing data is much worse than crashing, and corrupting data is worse still. Both are strong possibilities once your program's state is something unexpected.
- arcticbull 10y ago
  I agree and disagree. What the author says is true, if a thoroughly irrelevant part of your program (let's say a drawing routine for UI somewhere) hits an overflow and crashes your app in the middle of taking a payment, or in the middle of surgery -- while that much more critical element continued to work perfectly -- that's not ideal. Not all parts of your app make sense to fail-fast. By enforcing this behavior even on peripheral parts of your app, you're taking away the decision making ability about what's important and what isn't from the system designer. I understand that position, though I still think on the whole explicitness is right. Being able to explicitly disable the check when you know what you want is the right balance, whether it's &+ in Swift or (better, imo) .wrapping_add() in Rust.
  - mikeash 10y ago
    Displaying corrupt data in the UI can be just as bad as saving it to disk, if the user actually reads and uses the data you display (and if not, why are you displaying it?). And since the UI almost certainly shares data with the rest of the program, bugs in the UI can corrupt the underlying data. Separation of concerns is great, but if you really want different parts of your system to have different reliability constraints, then you need to truly separate them, for example by running them in different processes. Running them in the same process but having some of the code be more lax about state because it's less important is just begging for horrible bugs.
    - lomnakkus 10y ago
      > Separation of concerns is great, but if you really want different parts of your system to have different reliability constraints, then you need to truly separate them, for example by running them in different processes. Running them in the same process but having some of the code be more lax about state because it's less important is just begging for horrible bugs. Indeed, and you might even want to run them on entirely different and independent systems (connected by some type of network/bus) depending on exactly what bits are critical vs. non-critical.
AndrewStephens 10y ago
I totally agree. The most pernicious defect I ever fixed was a result of a subtle overflow during part of a larger calculation. The erroneous result could cause problems months after the overflow occurred and took me days to track down. The fact that Swift defaults to checking for overflow should be treated as good news.
- planteen 10y ago
  I had a nasty bug like that once as well. It was in some FPGA logic written in VHDL with an overflow on a FIR filter. Recreating the problem was very difficult since it only happened near max signal power with a noisy input. The correct solution there was to saturate, not wrap, which is almost always the case in DSP code.
steveklabnik 10y ago
> Neither is really surprising, It depends! Consider these three functions, in Rust: https://gist.github.com/steveklabnik/da82b51f9dbedd5321924927af091131 https://gist.github.com/steveklabnik/da82b51f9dbedd532192492... One uses a loop, one uses fold with a closure, one uses point-free style. On my machine, summing a vector of a million fives gives running 3 tests test sum_fold ... bench: 147,129 ns/iter (+/- 9,101) test sum_loop ... bench: 148,016 ns/iter (+/- 10,294) test sum_pointfree ... bench: 146,230 ns/iter (+/- 9,114) (there is some variance here between runs, sometimes the +/- for each changes a bit.) Cargo's bench profile doesn't use debug assertions, and hence, no overflow checks by default. Let's turn them on: running 3 tests test sum_fold ... bench: 147,638 ns/iter (+/- 12,654) test sum_loop ... bench: 143,485 ns/iter (+/- 7,255) test sum_pointfree ... bench: 143,050 ns/iter (+/- 7,549) Looks like they're being eliminated by LLVM anyway. Using godbolt, both the loop and the fold versions (I didn't bother to look at pointfree) are both getting vectorized, and while they're not _exactly_ identical, they're quite similar. (disclaimer: YMMV, I might have made a mistake, always inspect assembly for your situation, etc)
- iainmerrick 10y ago
  Good point! You're right, it is a little surprising that reduce() adds any overhead at all in this trivial example. It's interesting that Swift can't seem to fully eliminate that abstraction overhead when runtime checks are enabled -- maybe there's something other than overflow checks that's getting in the way? The author hints that the abstraction cost does vanish when checks are disabled, but doesn't bother showing any numbers.
  - jckarter 10y ago
    Looking at the generated code, we have a couple issues that prevent full optimization. One of them I already knew about (https://bugs.swift.org/browse/SR-2926 https://bugs.swift.org/browse/SR-2926) -- as a debugging aid, we guard every trap we emit with the equivalent of an `asm volatile("")` barrier, because LLVM will otherwise happily fold all the traps together into one trap. We really want every trap to have a distinct address so that the debugger can map that trap back to the source code that emitted it, but the asm barrier prevents other optimizations as well. As for `reduce`, it looks like the compiler does inline away the closure and turn the inner loop into what you would expect, but for some reason there's more pre-matter validation of the array than there is with a for loop. That's a bug; by all means, reduce is intended to optimize the same.
int_19h 10y ago
> Using "reduce" rather than a loop also has ~2x overhead ... Neither is really surprising I disagree. It all depends on where you're coming from, and what your expectations are. In C++, using reduce (std::accumulate) with a lambda is normally as fast as using a loop, because the compiler will inline the lambda into accumulate, and then accumulate into the calling function, producing the same exact loop.
deleted 10y ago
[deleted]
caconym_ 10y ago
It's a little bit surprising that using a fold rather than a simple loop has such a high overhead. I was already pretty sure of the result, but I went ahead and wrote a quick benchmark in Rust, and there is no overhead to using a fold rather than a loop (each iteration is addition of a million values of the stated numeric type): test tests::bench_fold_f64 ... bench: 984,829 ns/iter (+/- 30,827) test tests::bench_fold_i32 ... bench: 84,497 ns/iter (+/- 16,109) test tests::bench_loop_f64 ... bench: 993,406 ns/iter (+/- 45,791) test tests::bench_loop_i32 ... bench: 81,306 ns/iter (+/- 17,815) That this isn't true in Swift is way more interesting to me than the fact that overflow checking has overhead (duh). It's not difficult to come up with some theories as to why this might be the case, but still, if using these "functional" constructs in Swift has such a high overhead, programmers should probably be questioning whether it's worth using them at all. edit: I see that I've been beaten to the punch posting a Rust benchmark :)

dispose13432 10y ago

How do languages which aim to replace C (such as Rust) deal with this issue? Now I agree, your average webapp won't see any benefit by removing checks and will see security features by keeping them in, so I'm all for it. But in OSs (or browsers), speed does matter. And there's no way to optimize it (every + or array operation involves an if). Is this just one of the "costs of doing business"?

iainmerrick 10y ago
You optimize by using "&+" instead of "+", or by compiling with "-Ounchecked". You figure out if you need to optimize by testing and benchmarking.
maxpolun 10y ago
Rust has overflow checking in debug mode, but release mode does not have any overflow checking. The thinking here is that you'll hopefully catch most of your overflows while testing the app, but get no performance penalty in production use. If you want to overflow there are special types that you can use.
mikeash 10y ago
In Swift, you have total control over this. If you want checks, use +. If you don't want checks, use &+. In Rust, the operators check in debug builds, and wrap in release builds. Calls are provided so you can always have them check, or always have them wrap, if you need it. Note that C has some really ugly behavior here. Signed overflow is undefined behavior, which means that the compiler doesn't need to check for it, but also that it can assume it never happens and optimized based on that. Many programmers don't realize this, or don't notice that they've accidentally written code that depends on overflow behavior, which can lead to many entertaining bugs: https://lwn.net/Articles/511259/ https://lwn.net/Articles/511259/
- deleted 10y ago
  [deleted]
- dispose13432 10y ago

lmm 10y ago

> That’s because I do not think it is universally a good thing that software should crash when an overflow occurs. Think about the software that runs in your brain. Sometimes you experience bugs. For example, an optical illusion is a fault in your vision software. Would you want to fall dead whenever you encounter an optical illusion? That does not sound entirely reasonable, does it? Moreover, would you want this “fall dead” switch to make all of your brain run at half its best speed? If you lived in a world where hackers crafted optical illusions that made you send all your money to them when you viewed them, you would probably want to go blind or some such when you encountered such an illusion.

mikeash 10y ago
A bit of a tangent here, but anyone intrigued by the notion of hacking the brain using optical illusions should read "comp.basilisk FAQ": http://ansible.uk/writing/c-b-faq.html http://ansible.uk/writing/c-b-faq.html
- e28eta 10y ago
  Neat! Reminded me of Snow Crash, by Neal Stephenson. I'm sure nearly everyone is familiar with it, but for those who aren't: it's pretty good!

acqq 10y ago

I see a number of posts that proudly claim (in different forms) "fail early and fast" like it's a good thing to simply do a run-time crash always, and even that the article author is "misguided and inflammatory." I don't agree with both claims. Regarding the first, as an example where the crash is definitely not the solution, see the planteen's post: https://news.ycombinator.com/item?id=13117170 https://news.ycombinator.com/item?id=13117170 Or consider that once you use the computers to calculate the real life stuff (like the movement of your car, or even the spaceship 50 million kilometers away) the worst thing you can do is introduce the "fatal discontinuities" in the processing. Regarding the second, allow me to just roll my eyes. The politics is not allowed this week on HN, but the political approaches start to be used automatically. Please just write which his claim is wrong. Labeling is destructive.

mikeash 10y ago
A crash is never the right solution. It is, however, likely to be a better solution than whatever random default behavior you might get. Regarding the linked comment, if the default had been to crash on overflow (or whatever the FPGA equivalent might be, if there is such a thing) then the problem would have been apparent the first time it happened, and they wouldn't be describing it as "a nasty bug" in the first place. A crash would not have been the best solution there, but it would have been miles better than the wrap that they actually got. As far as real life stuff, my understanding is that it's pretty common to fail fast and design the system to restart quickly so it can get back to work. If your needs are more sophisticated then you'd need to actually analyze the problem and come up with ways to handle unexpected data gracefully, not just hope that default behavior from the compiler ends up doing something sensible for your problem.
- z3t4 10y ago
  If 1-2 ms is too much to recover from a crash, you better have three or four backup systems written in different languages and running on different hardware. Yet there is no guarantee you will have 100% uptime. And in such a system, having data corruption would probably be far worse then a crash. One application where I think crashes are bad are computer games, for example if you are fighting a boss, it's better to get a few glitches then having the game crash.

Animats 10y ago

Now that more languages are checking for integer overflow, it's time for integer overflow exceptions to re-appear in hardware. DEC VAX machines had this, but C didn't use them. With the hardware doing the checking in parallel, there's no performance penalty. If you want wrap-around arithmetic (which is rare) you should have to write something like unsigned short i,j; i = (i + j) % 65536; which the compiler should optimize into a no-check add. This gets you the same answer on all platforms.

gok 10y ago
Do you find a shorthand for wraparound math (like Swift's &+) so bad?

c0ffe 10y ago

After chasing strange bugs when using dynamic languages like PHP and JavaScript that keep running by default when minor errors happen (PHP warnings, or undefined variables in JavaScript), I think its good that Swift priorizes safety rather than speed.

deleted 10y ago

[deleted]

sulam 10y ago

I found this statement funny: "Moreover, would you want this “fall dead” switch to make all of your brain run at half its best speed?" ...because this is actually how our brains work! We are much slower to process and respond to all sorts of stimuli (visual, auditory, conceptual [reading]) when it is contradictory. Think of the feeling you get looking at an Escher sketch.

xenadu02 10y ago

I don't get the same numbers he gets. The reduce version is the same speed as the simple for loop. He must have made a mistake somewhere.

bjourne 10y ago

Most costly software bug I've ever witnessed was caused by an integer overflow. Thankfully, it wasn't caused by me, but if I had been auditing the code, I probably wouldn't have found the bug. The system was billing customers credit cards depending on how long they had used the service. Time was measured in milliseconds (uh-oh!) for no apparent reason. Usage could have been measured in seconds or even days but someone thought it was good to be extra precise. And System.currentTimeMillis() returns milliseconds. The default charging period was 14 days which worked well. So the maximum number of milliseconds that could be charged (for someone who used the system the whole month) was 1,209,600,000. Then the company decided to change the period to every two months (60 days) instead to save money as there was a fixed cost added to every credit card transfer. Guess what 60 * 24 * 3600 * 1000 is? It's a number a bit bigger than 2^31 - 1 which is the most positive primitive integer value in Java. And the "totalDuration" variable had type "int". :) So totalDuration wrapped around which caused the system to retry the transaction over and over and debit customers hundreds of times more than what they really owed. The resulting fallout from that debacle was one of the reasons the company went bankrupt. Integer overflow checking could have saved them.

nkurz 10y ago

I thought it might be interesting to see how this effect changes with the size of the array being summed. How do the relative speeds change when operating out of L1, L3, and memory? Does the lower speed of memory access overwhelm the overhead of the overflow checking? $ swift build --configuration release $ cset proc -s nohz -e .build/release/reduce # count (basic, reduce, unsafe basic, unsafe reduce) 1000 (0.546, 0.661, 0.197, 0.576) 10000 (0.403, 0.598, 0.169, 0.544) 100000 (0.391, 0.595, 0.194, 0.542) 1000000 (0.477, 0.663, 0.294, 0.582) 10000000 (0.507, 0.655, 0.337, 0.608) 100000000 (0.509, 0.655, 0.339, 0.608) 1000000000(0.511, 0.656, 0.345, 0.611) $ swift build --configuration release -Xswiftc -Ounchecked $ cset proc -s nohz -e .build/release/reduce # count (basic, reduce, unsafe basic, unsafe reduce) 1000 (0.309, 0.253, 0.180, 0.226) 10000 (0.195, 0.170, 0.168, 0.170) 100000 (0.217, 0.203, 0.196, 0.201) 1000000 (0.292, 0.326, 0.299, 0.252) 10000000 (0.334, 0.337, 0.333, 0.337) 100000000 (0.339, 0.339, 0.340, 0.339) 1000000000(0.344, 0.344, 0.344, 0.344) Code is from https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/tree/master/2016/12/05 https://github.com/lemire/Code-used-on-Daniel-Lemire-s-blog/... with modification to loop over the different array lengths. Numbers are for Skylake at 3.4 GHz with swift-3.0.1-RELEASE-ubuntu16.04. Count is the number of 8B ints in the array being summed. Results shown were truncated by hand --- I wasn't sure how to specify precision from within Swift. The execution with "cset proc -s nohz" was to reduce jitter between runs, but doesn't significantly affect total run time. The anomalously fast result for the L3 sized 'unsafe' 'unchecked' is consistent.

mcguire 10y ago

"That’s because I do not think it is universally a good thing that software should crash when an overflow occurs. Think about the software that runs in your brain. Sometimes you experience bugs. For example, an optical illusion is a fault in your vision software. Would you want to fall dead whenever you encounter an optical illusion? That does not sound entirely reasonable, does it? Moreover, would you want this “fall dead” switch to make all of your brain run at half its best speed? In software terms, this means that a mission-critical application could crash because an unimportant function in a secondary routine overflows." That is a ridiculous analogy. What if we replace "optical illusion" with "hallucination"? More importantly, what if there were some sort of middle ground between continuing on as if nothing happened on an error and crashing completely?

emodendroket 10y ago

A factor of three for addition is probably not really significant in most programs.