3 ms·

I'd be very interested to know what that benchmark is, because you're either doing something very wrong, or it isn't relevant to the systems programming topic b

I'd be very interested to know what that benchmark is, because you're either doing something very wrong, or it isn't relevant to the systems programming topic being discussed. A 1000x perf difference should be easy to show.

mpweiher 11y ago
Just summing floats in an array, compiled without optimization, and yes, that surprised me. My Postscript interpreter written in Objective-C was 2x faster. :-) With optimization that went down to around 10x and wouldn't budge, even with -Ounchecked, UnsafeMutablePointer, varying an indexed for-loop and reduce. As I wrote, it's not just the absolute perf., it's the variability and unpredictability.
- rarepostinlurkr 11y ago
  Gist?
- chrislattner 11y ago
  >Just summing floats in an array, compiled without optimization, Ok, so you're both doing it wrong and doing something irrelevant to that level of systems programming. Measuring the performance of non-optimized builds will always be pretty meaningless (even C compilers can have radical perf swings from release to release at -O0), and reporting that as a rebuttal of whether swift is relevant for "bootloader or firmware" is just outright trying to mislead people. > With optimization that went down to around 10x and wouldn't budge, even with -Ounchecked, UnsafeMutablePointer, varying an indexed for-loop and reduce. Beyond what you said, I suspect that all of these impressions were formed with Swift 1. Believe it or not, things have improved a lot since then. On a "add array of floats" benchmark, the vectorizer kicking in or not is a 4x or more performance difference. This should be fixed in Swift 2, but more to the point is a benchmark irrelevant to "boot loaders and firmware". -Chris
  - mpweiher 11y ago
    > doing something irrelevant to that level of systems programming Except: no. I reported the float-array result because it was literally the 2nd test I ran and hadn't gotten around to integer arrays yet. I was expecting this to be completely doozy, one of the things Swift should be acing by now. Of course I am aware that you wouldn't use floats in systems programming, but as you should know, floats today are simple machine-level scalars that present little if any overhead to the CPU compared to integers. That is, this test uses a few basic machine operations: indexed memory access, loops, scalar arithmetic. Whether those scalars are floats or integers should be largely irrelevant to the outcome of the test and therefore the test is a good enough (if not perfect) proxy for the types of operations you would see in systems programming. Are you seriously claiming that the results would be different if the scalars in question had been integers instead of floats? Seriously?? If you had made such a claim, you'd be wrong: I've now run the same test with integer arrays and the result is exactly the same. Next you are going to tell me that integers are irrelevant to systems programming? > [non-optimized builds] and reporting that as a rebuttal of whether swift is relevant for "bootloader or firmware" is just outright trying to mislead people. No, it is you who is misleading people by completely mischaracterizing what I wrote. I wrote that performance varies from 30% worse to 1000x worse "depending on optimization level". This is exactly what is happening. (Of course, for this precise test it is 10x worse to 1000x worse, but who's counting?) Claiming that non-optimized builds are irrelevant is also disingenuous at best, because non-optimized is the default setting for non-release builds in Xcode. Having this type of performance (variance) means that debug builds are effectively unusable: an operation that would take 100ms in a release build would take close to two minutes in a debug build. Yes, other languages also have different performance levels between debug and non-debug builds, but size matters. 2-10x is something you can deal with, 1000x is not. Furthermore, the problem is not so much the 1000x worse performance (although that is funny), it is the difference between optimized and non-optimized builds. The optimizer is not part of the language spec, meaning you can't rely on it. Next compiler release the compiler can change a little bit so it won't optimize one part of the code that you needed to run fast. Or you change the code a little bit in a way that the optimizer doesn't like and the effect is the same. And of course the compiler doesn't tell you what optimizations ran or did not run, so you are dealing with the blackest of black boxes. One day your code is fast, the other day it is not. This is not predictable performance, and today predictability is often more important than raw speed for performance work. See my Jitterdämmerung post[1] or The Death of Optimizing Compilers[2]. > Beyond what you said, I suspect that all of these impressions were formed with Swift 1 You suspect wrong, all this was using the newest Xcode tools (7.1.1 previously, just repeated with 7.2): Apple Swift version 2.1.1 (swiftlang-700.1.101.15 clang-700.1.81) Target: x86_64-apple-darwin15.2.0 And by suspecting wrong, you kinda make my point for me, so thank you, Chris! Swift's performance model is so amazingly unpredictable that even the language's creator, compiler- and optimizer-wizard extraordinaire can't figure it out. How are mere mortals supposed to do so? When doing systems programming and embedded programming, predictability is paramount. I remember the story of the railway engineer who scoured the planet for remaining stocks of 8085 processors, because he thought he understood the bugs in that particular processor. Now sure, you certainly can use Swift for these types of tasks despite all the issues. But I suspect you'd be better off using FORTH. Marcel [1] http://blog.metaobject.com/2015/10/jitterdammerung.html http://blog.metaobject.com/2015/10/jitterdammerung.html [2] http://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf http://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf

mpweiher 11y ago

Just summing floats in an array, compiled without optimization, and yes, that surprised me. My Postscript interpreter written in Objective-C was 2x faster. :-) With optimization that went down to around 10x and wouldn't budge, even with -Ounchecked, UnsafeMutablePointer, varying an indexed for-loop and reduce. As I wrote, it's not just the absolute perf., it's the variability and unpredictability.

rarepostinlurkr 11y ago
Gist?
chrislattner 11y ago
>Just summing floats in an array, compiled without optimization, Ok, so you're both doing it wrong and doing something irrelevant to that level of systems programming. Measuring the performance of non-optimized builds will always be pretty meaningless (even C compilers can have radical perf swings from release to release at -O0), and reporting that as a rebuttal of whether swift is relevant for "bootloader or firmware" is just outright trying to mislead people. > With optimization that went down to around 10x and wouldn't budge, even with -Ounchecked, UnsafeMutablePointer, varying an indexed for-loop and reduce. Beyond what you said, I suspect that all of these impressions were formed with Swift 1. Believe it or not, things have improved a lot since then. On a "add array of floats" benchmark, the vectorizer kicking in or not is a 4x or more performance difference. This should be fixed in Swift 2, but more to the point is a benchmark irrelevant to "boot loaders and firmware". -Chris
- mpweiher 11y ago
  > doing something irrelevant to that level of systems programming Except: no. I reported the float-array result because it was literally the 2nd test I ran and hadn't gotten around to integer arrays yet. I was expecting this to be completely doozy, one of the things Swift should be acing by now. Of course I am aware that you wouldn't use floats in systems programming, but as you should know, floats today are simple machine-level scalars that present little if any overhead to the CPU compared to integers. That is, this test uses a few basic machine operations: indexed memory access, loops, scalar arithmetic. Whether those scalars are floats or integers should be largely irrelevant to the outcome of the test and therefore the test is a good enough (if not perfect) proxy for the types of operations you would see in systems programming. Are you seriously claiming that the results would be different if the scalars in question had been integers instead of floats? Seriously?? If you had made such a claim, you'd be wrong: I've now run the same test with integer arrays and the result is exactly the same. Next you are going to tell me that integers are irrelevant to systems programming? > [non-optimized builds] and reporting that as a rebuttal of whether swift is relevant for "bootloader or firmware" is just outright trying to mislead people. No, it is you who is misleading people by completely mischaracterizing what I wrote. I wrote that performance varies from 30% worse to 1000x worse "depending on optimization level". This is exactly what is happening. (Of course, for this precise test it is 10x worse to 1000x worse, but who's counting?) Claiming that non-optimized builds are irrelevant is also disingenuous at best, because non-optimized is the default setting for non-release builds in Xcode. Having this type of performance (variance) means that debug builds are effectively unusable: an operation that would take 100ms in a release build would take close to two minutes in a debug build. Yes, other languages also have different performance levels between debug and non-debug builds, but size matters. 2-10x is something you can deal with, 1000x is not. Furthermore, the problem is not so much the 1000x worse performance (although that is funny), it is the difference between optimized and non-optimized builds. The optimizer is not part of the language spec, meaning you can't rely on it. Next compiler release the compiler can change a little bit so it won't optimize one part of the code that you needed to run fast. Or you change the code a little bit in a way that the optimizer doesn't like and the effect is the same. And of course the compiler doesn't tell you what optimizations ran or did not run, so you are dealing with the blackest of black boxes. One day your code is fast, the other day it is not. This is not predictable performance, and today predictability is often more important than raw speed for performance work. See my Jitterdämmerung post[1] or The Death of Optimizing Compilers[2]. > Beyond what you said, I suspect that all of these impressions were formed with Swift 1 You suspect wrong, all this was using the newest Xcode tools (7.1.1 previously, just repeated with 7.2): Apple Swift version 2.1.1 (swiftlang-700.1.101.15 clang-700.1.81) Target: x86_64-apple-darwin15.2.0 And by suspecting wrong, you kinda make my point for me, so thank you, Chris! Swift's performance model is so amazingly unpredictable that even the language's creator, compiler- and optimizer-wizard extraordinaire can't figure it out. How are mere mortals supposed to do so? When doing systems programming and embedded programming, predictability is paramount. I remember the story of the railway engineer who scoured the planet for remaining stocks of 8085 processors, because he thought he understood the bugs in that particular processor. Now sure, you certainly can use Swift for these types of tasks despite all the issues. But I suspect you'd be better off using FORTH. Marcel [1] http://blog.metaobject.com/2015/10/jitterdammerung.html http://blog.metaobject.com/2015/10/jitterdammerung.html [2] http://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf http://cr.yp.to/talks/2015.04.16/slides-djb-20150416-a4.pdf