4 ms·

Apples to Apples, Part II

90 points by poolpool 12y ago

mpweiher 12y ago
Looking at the original article, it seems like he is comparing sorting an array of NSNumber objects in Objective-C with sorting an array of primitive integers in Swift. The only really big surprise is that the latter actually used to be slower. Boggle.
- tolmasky 12y ago
  Yeah, not to mention that NSMutableArray is not necessarily an array in Objective-C, see: http://ridiculousfish.com/blog/posts/array.html http://ridiculousfish.com/blog/posts/array.html . It really seems like this benchmark is comparing different framework approaches more than any "core language speed" (similar to comparing stl::vector sort of ints in Objective-C++ to NSMutableArray sort of NSNumbers) Just looking at the "std lib sort", what this might actually be testing off the top of my head is: 1. C-style-array access/write speed of "native arrays" vs. potentially hash-mappy access/write speed of array-like NSMutableArrays. 2. Direct integer comparison (a < b) vs -[NSNumber compare:] method dispatch speed. 3. I don't know how sorted() is implemented internally in Swift, but given that no comparison method is being passed in, you may also be comparing a fast internal int[] sort that directly compares ints to the speed of calling the NSComparisonResult lambda O(n lg n) times.
- chucknelson 12y ago
  Can anyone explain the reasoning behind this difference? The article only mentions "In Objective-C, NSArray and NSNumber are used intentionally as the counterparts to Swift's Array and Int."
  - tolmasky 12y ago
    I can't think of a good reason other than "I don't want to compare C-style sort with Swift because I want to talk about Objective-C". This is an incredibly poor benchmark in my opinion, since if you were dealing with a ton of ints in an Objective-C program you'd probably be dropping down to C or (in the case of ObjC++), C++. Back when Swift was slower and losing, I suppose it was eye opening that despite all this ObjC was still winning. That's not to say that there isn't any insight here, I just think the much more powerful argument would be "look, in Swift we can get C-like speed without having to use icky primitive arrays". Which is actually a pretty powerful selling point (if its true, which we don't know, since C-style sorts aren't compared here).
    - antimagic 12y ago
      That's a little harsh. I for one think that it's a perfectly reasonable thing to do. The benchmark was how does Swift go compared to Objective-C, not C. That is, if you're using those nice data structures that make life easy in Cocoa, what sort of a speed hit can you expect. As an example, say I want to use an NSArray because I need to serialize the contents, or because some other module expects an NSArray, it's a pain in the neck and a source of bugs if I have to translate safely from a C array to an NSArray every timie I want to use it (and if that array is mutable?) This benchmark shows how things perform when you're using the nice, safe array (because let's not forget that an NSArray is much safer than a C array). In Swift you get performance that absolutely trounces the equivalent functionality in Objective-C. That's a nice thing to know.
      - tolmasky 12y ago
        I think we agree, but let me explain. > The benchmark was how does Swift go compared to Objective-C, not C Objective-C is a strict superset of C, in other words, there's no such thing as "Objective-C, not C". This is not me being pedantic, its an incredibly important feature and selling point of the language. You are making ObjC perform with one arm tied behind its back, exposing all of the pitfalls with none of the benefits. > In Swift you get performance that absolutely trounces the equivalent functionality in Objective-C. That's a nice thing to know. Precisely. I think there is a great blog post about how you get certain things in Swift for free that would be time consuming/hard/whatever in Objective-C. But without making the actual comparison, we can't know or understand that tradeoff. If this also showed pure-C or ObjC++ perf, I would walk away thinking "damn, I can get 0.9x the speed of C AND have it be safe? Cool." Instead, here I'm just left thinking "ok, so Swift is faster than something I'd never do in Obj-C. Great." Then immediately I'd start asking "wait, isn't this maybe just because we're using an expensive -[NSNumber compare:] thousands of times? Maybe this can be trivially remedied by changing that call to compare to direct accesses to the internal ints?" Edit: As an additional comment, I'd point out that you'd make an IntArray subclass to the NSArray class cluster to get both the perf and serialization of NSArray. Again, convoluted and unnecessary work, but ultimately what you'd actually do in ObjC.
    - 12y ago
- free652 12y ago
  Swift is slower in the debug mode and faster with full optimizations. I don't know either language, but I think NSArray cannot store primitive types, so he used NSNumber object wrapper.
  - christoph 12y ago
    Thats correct, you need wrap the primitive types when putting them in an array. Unless you use a C style array, but thats more hassle than it's worth generally.
- JonathonW 12y ago
  It's worth noting that Swift Ints (and Arrays) aren't primitives; they're named types defined as structs in Swift code in the standard library, and have methods and can be extended like any other types in the library (see [1]). That's why the unoptimized case is slow, and why the author gets such a dramatic speedup from optimization-- the Swift optimizer (unlike the Objective C optimizer, which, thanks to ObjC's messaging system, has to keep method calls intact) can use techniques like method inlining to bring performance to a point close to that of a native int/plain C array. Regardless, I'd still like to see a comparison with C++ (where method dispatch looks a lot like Swift and the optimizer should be able to do similar things) and plain C. Might have to try that this evening if I get bored. 1: https://developer.apple.com/library/prerelease/mac/documentation/Swift/Conceptual/Swift_Programming_Language/Types.html#//apple_ref/doc/uid/TP40014097-CH31-XID_1078 https://developer.apple.com/library/prerelease/mac/documenta...
  - coob 12y ago
    Likewise, it's also worth noting that since 10.7/iOS 5, NSNumbers can be tagged pointers rather than separately allocated objects.
ljosa 12y ago
Expected an Apple II; was disappointed.

90 points by poolpool 12y ago

Looking at the original article, it seems like he is comparing sorting an array of NSNumber objects in Objective-C with sorting an array of primitive integers in Swift. The only really big surprise is that the latter actually used to be slower. Boggle.

tolmasky 12y ago
Yeah, not to mention that NSMutableArray is not necessarily an array in Objective-C, see: http://ridiculousfish.com/blog/posts/array.html http://ridiculousfish.com/blog/posts/array.html . It really seems like this benchmark is comparing different framework approaches more than any "core language speed" (similar to comparing stl::vector sort of ints in Objective-C++ to NSMutableArray sort of NSNumbers) Just looking at the "std lib sort", what this might actually be testing off the top of my head is: 1. C-style-array access/write speed of "native arrays" vs. potentially hash-mappy access/write speed of array-like NSMutableArrays. 2. Direct integer comparison (a < b) vs -[NSNumber compare:] method dispatch speed. 3. I don't know how sorted() is implemented internally in Swift, but given that no comparison method is being passed in, you may also be comparing a fast internal int[] sort that directly compares ints to the speed of calling the NSComparisonResult lambda O(n lg n) times.
chucknelson 12y ago
Can anyone explain the reasoning behind this difference? The article only mentions "In Objective-C, NSArray and NSNumber are used intentionally as the counterparts to Swift's Array and Int."
- tolmasky 12y ago
  I can't think of a good reason other than "I don't want to compare C-style sort with Swift because I want to talk about Objective-C". This is an incredibly poor benchmark in my opinion, since if you were dealing with a ton of ints in an Objective-C program you'd probably be dropping down to C or (in the case of ObjC++), C++. Back when Swift was slower and losing, I suppose it was eye opening that despite all this ObjC was still winning. That's not to say that there isn't any insight here, I just think the much more powerful argument would be "look, in Swift we can get C-like speed without having to use icky primitive arrays". Which is actually a pretty powerful selling point (if its true, which we don't know, since C-style sorts aren't compared here).
  - antimagic 12y ago
    That's a little harsh. I for one think that it's a perfectly reasonable thing to do. The benchmark was how does Swift go compared to Objective-C, not C. That is, if you're using those nice data structures that make life easy in Cocoa, what sort of a speed hit can you expect. As an example, say I want to use an NSArray because I need to serialize the contents, or because some other module expects an NSArray, it's a pain in the neck and a source of bugs if I have to translate safely from a C array to an NSArray every timie I want to use it (and if that array is mutable?) This benchmark shows how things perform when you're using the nice, safe array (because let's not forget that an NSArray is much safer than a C array). In Swift you get performance that absolutely trounces the equivalent functionality in Objective-C. That's a nice thing to know.
    - tolmasky 12y ago
      I think we agree, but let me explain. > The benchmark was how does Swift go compared to Objective-C, not C Objective-C is a strict superset of C, in other words, there's no such thing as "Objective-C, not C". This is not me being pedantic, its an incredibly important feature and selling point of the language. You are making ObjC perform with one arm tied behind its back, exposing all of the pitfalls with none of the benefits. > In Swift you get performance that absolutely trounces the equivalent functionality in Objective-C. That's a nice thing to know. Precisely. I think there is a great blog post about how you get certain things in Swift for free that would be time consuming/hard/whatever in Objective-C. But without making the actual comparison, we can't know or understand that tradeoff. If this also showed pure-C or ObjC++ perf, I would walk away thinking "damn, I can get 0.9x the speed of C AND have it be safe? Cool." Instead, here I'm just left thinking "ok, so Swift is faster than something I'd never do in Obj-C. Great." Then immediately I'd start asking "wait, isn't this maybe just because we're using an expensive -[NSNumber compare:] thousands of times? Maybe this can be trivially remedied by changing that call to compare to direct accesses to the internal ints?" Edit: As an additional comment, I'd point out that you'd make an IntArray subclass to the NSArray class cluster to get both the perf and serialization of NSArray. Again, convoluted and unnecessary work, but ultimately what you'd actually do in ObjC.
  - 12y ago
free652 12y ago
Swift is slower in the debug mode and faster with full optimizations. I don't know either language, but I think NSArray cannot store primitive types, so he used NSNumber object wrapper.
- christoph 12y ago
  Thats correct, you need wrap the primitive types when putting them in an array. Unless you use a C style array, but thats more hassle than it's worth generally.
JonathonW 12y ago
It's worth noting that Swift Ints (and Arrays) aren't primitives; they're named types defined as structs in Swift code in the standard library, and have methods and can be extended like any other types in the library (see [1]). That's why the unoptimized case is slow, and why the author gets such a dramatic speedup from optimization-- the Swift optimizer (unlike the Objective C optimizer, which, thanks to ObjC's messaging system, has to keep method calls intact) can use techniques like method inlining to bring performance to a point close to that of a native int/plain C array. Regardless, I'd still like to see a comparison with C++ (where method dispatch looks a lot like Swift and the optimizer should be able to do similar things) and plain C. Might have to try that this evening if I get bored. 1: https://developer.apple.com/library/prerelease/mac/documentation/Swift/Conceptual/Swift_Programming_Language/Types.html#//apple_ref/doc/uid/TP40014097-CH31-XID_1078 https://developer.apple.com/library/prerelease/mac/documenta...
- coob 12y ago
  Likewise, it's also worth noting that since 10.7/iOS 5, NSNumbers can be tagged pointers rather than separately allocated objects.

ljosa 12y ago

Expected an Apple II; was disappointed.