7 ms·
Swift is not
- cowsandmilk 12y agoAuthor did not say if he compiled with compiler optimizations, which appear to be very important for swift being "swift".[1] [1] https://twitter.com/Catfish_Man/status/473752917347139584 https://twitter.com/Catfish_Man/status/473752917347139584
- mpweiher 12y agoI made similar tests with optimization and saw similar numbers. Without optimization it was significantly worse.
- electic 12y agoSame here. There seems to be a stiff price to pay for all that convenience...
- solarexplorer 12y agoI don't get it. "Similar numbers" but "significantly worse"? You measure performance and compile without optimization?! If you have better numbers, please share them!
- dragonwriter 12y ago> I don't get it. "Similar numbers" but "significantly worse"? GP said with optimization he got similar numbers, and without optimization he got significantly worse numbers. It was crystal clear. > You measure performance and compile without optimization?! GP clearly stated he compiled both with and without optimization, and characterized (qualitatively) the results of each compared to the results in the article.
- mpweiher 12y ago> "Similar numbers" but "significantly worse"? Yes. Similar numbers to the ones posted when compiling with optimization. Significantly worse when compiling without optimization. > You measure performance and compile without optimization?! No, I tend to measure with optimization. In fact, I personally don't do Debug builds at the all. However, when the numbers were so bad, I did a Debug build to cross check, and lo and behold, those numbers were even worse. That clear things up?
- solarexplorer 12y agoYep. Thank you.
- itistoday2 12y ago@milend via twitter: "He's definitely running the Swift code in Debug mode as I saw numbers in the same ballpark." [1] https://gist.github.com/anonymous/4df3610970891c6afc5a https://gist.github.com/anonymous/7d7fcf0a5ce24c37999c ObjC: 0.063209, Swift: 0.255287 (Release) ObjC: 0.060600 Swift: 5.200207 (Debug)" [2] [1] https://twitter.com/milend/status/474349620311883776 [2] https://twitter.com/milend/status/474354899527155712
- deleted 12y ago[deleted]
- milen 12y agoI did some further investigation [1] by running 20 million iterations in Release and profiled the code. Turns out, at least 82.9% of it was just swift_retain / swift_release (ref counting). Whether those retains / released can be optimised away, I don't know, we'll have to wait and see. [1] https://twitter.com/milend/status/474360984413683713 https://twitter.com/milend/status/474360984413683713
- muaddirac 12y agoI never trust apple keynote benchmarks, but I was really holding out for the swift ones to be true. It probably makes no difference in the majority of common iOS/OSX development, though.
- TheMagicHorsey 12y agoThe issue might be that the benchmarks shown during the presentation were for iOS, and the author here might have built for OS X. The other possibility may be that the we may need to tweak the compiler settings. I certainly don't think that Apple was outright lying in their benchmarks. Those sort of shenanigans are a PR nightmare, and nobody thinks its worth it anymore.
- wutbrodo 12y ago> nobody thinks its worth it anymore. I'm curious which period you're referring to here. I haven't been following Apple presentations in detail for a while, but I remember the iPad 2 announcement was chock-full of flat-out lies. Do you mean that this is something that's changed since then (March 2011)? Or do you mean that it's not worth it for developer-focused presentations vs consumers (who would presumably be more credulous)? EDIT w/ source: http://fortune.com/2011/03/03/steve-jobs-reality-distortion-takes-its-toll-on-truth/ http://fortune.com/2011/03/03/steve-jobs-reality-distortion-...
- EdwardCoffin 12y ago> the iPad 2 announcement was chock-full of flat-out lies Do you have anything concrete to back this up with?
- wutbrodo 12y agoAh of course, that was dumb of me. I was remembering this from a few years ago but I should've taken the time to dig up a source referring to it. For some reason I was thinking "everybody must remember that". Here's an article that's pretty succinct about the inaccuracy and lies in the iPad 2 announcement: http://fortune.com/2011/03/03/steve-jobs-reality-distortion-takes-its-toll-on-truth/ http://fortune.com/2011/03/03/steve-jobs-reality-distortion-...
- axman6 12y agoShock news: Marketing puts company's products in positive light! All companies tweak the truth to make themselves look good (which company was touting their huge sales numbers when they had barely sold any products to actual customers, just retailers?). Apple aren't alone, they just get the most press.
- koenigdavidmj 12y agoAnd this is why C++ can still be fast, even when the common STL implementations are a steaming pile performance-wise. The naturally developed style has been one-reference-only, so a lot of this never has to get done. With Objective-C, anything that isn't C is on the heap and reference counted. Even Java can allocate objects on the stack in tight loops (though this is done by the JIT, not by a change in how variables are declared).
- TillE 12y agoWhat parts of modern, popular STL implementations are notably slow? The only issue I've ever had is the blatantly obvious one of reallocating std::vector as it grows. You really want to reserve() enough space in advance if at all possible, because copying large chunks of memory is not fast.
- yoklov 12y agoYou don't always get to use a modern, popular STL. Sometimes you need to use the hardware vendor's crappy, unoptimized STL (because they assume you won't use it). This is part of the reason the STL is unpopular in game development, though the story here has been getting better. To answer your question, in my experience when the standard doesn't mention the desired performance characteristics of a function or method, even the popular STLs (libstdc++ and libc++ certainly) optimize for maintainability over performance. Not sure I can pull up an example of this right now but they are numerous. A good place to start is the standard containers other than deque or vector. A bigger issue IMO is that the existence of some methods in various standard classes/templates really murder performance, no matter how good the implementation is. One particularly bad example is the bucket() method, local_iterator types, etc. in c++11's unordered_map (probably the rest of the unordered family too). These force the table to be implemented in such a way that every insertion requires a heap allocation, and iteration requires much more pointer chasing than would otherwise be necessary (e.g. with a probed implementation), which is... unfortunate for the cache.
- jrajav 12y agoGoogle cache of the post, since the blog 'sploded: http://webcache.googleusercontent.com/search?q=cache:www.splasmata.com/?p=2798 http://webcache.googleusercontent.com/search?q=cache:www.spl...
- deleted 12y ago[deleted]
- adamnemecek 12y agoThese tests are meaningless without comparing the produced assembly (maybe the obj-c compiler is very well optimized since they've been doing that for quite a while now) and even then these microscopic tests don't really tell you much about performance in real scenarios. Swift has the advantage, as far as performance goes, that methods are not dispatched dynamically. In obj-c, every single time you call a method, you call objc_msgSend which executes at least 12 (IIRC) additional instructions (or more, depending on the situation). Plus as far as I know, not all objects are heap allocated which will also make things faster.
- mpweiher 12y agoMethod dispatch tends to not be a major factor in ObjC performance.
- adamnemecek 12y agoI'm inclined to disagree and this article seems to disagree as well http://it.toolbox.com/blogs/macsploitation/bypassing-objectivecs-message-passing-mechanism-for-speed-24946 http://it.toolbox.com/blogs/macsploitation/bypassing-objecti... (it's not a great test but it's a test) Given I've found some other articles that claim that it's not that slow but none of them really present any data. I guess that it depends on your definition of "major factor".
- blazespin 12y agoYes, if you were doing a quick sort and each op required a msgsend it'd be a factor. But you'd have to be crazy to write code like that. This is why there are a lot of C apis that Apple has written that you call and bridge with ObjC. With Swift you can't do that. You have to wrap the calls and generate bridge headers. What a pain.
- adamnemecek 12y agoAre you surprised that a language that's officially 2 days old has worse support than a language that they've been using for the last maybe 25 years?
- supercoder 12y agoWe might be missing something here and I think ultimately Swift will be faster than Obj-C but it does highlight it's probably not wise to just start blindly converting all your code to Swift just yet until we get a better feel for what it's good (and not) good at. Also with the fact that Apple says they will be breaking code as time goes on I'll also be on the Swift sideline for now. Edit: Why are people finding this comment so offensive ?
- gumby 12y ago> I think ultimately Swift will be faster than Obj-C Is there something about the semantics swift that you think would inherently permit faster code than well-written ObjC? Edit: I don't know why this comment was down voted -- it's a perfectly reasonable question that others have responded to. Oddly, the parent comment _I_ was responding too is apparently also being down voted. WTF?
- supercoder 12y agoNot semantics specifically but the engineering effort Apple will be investing in it.
- gte910h 12y agoYes, the types. Strong types are far easier to optimize.
- gumby 12y agoWell, swift does a lot of type inference it's true but it doesn't appear to be more to be more strongly typed than ObjC (I could be wrong because I have only skimmed the ebook). A lot of ObjC code simply returns an id when it could return a tighter type. They seem semantically equivalent to me. Typically a language's efficiency comes from its ability to restrict ambiguity. E.g. a scoped sequence of expressions without gotos allows the compiler to eliminate variables, reorder statements, unroll loops etc because it can see 100% of the uses of a variable by examining that scope. On the other extreme a language like BASIC is harder to optimize in such a fashion because the program counter could be set into the middle of the loop, and because all variables are global, they have to survive the scope. I am genuinely interested if their are features in swift that aren't also in ObjC. A language can be equivalent to another and still be easier to program in because the cognitive load on the programmer is lower so good code can be written more quickly. Swift seems to be an interesting attempt at doing that. But that's orthogonal to the question of compiled efficiency.
- mindstab 12y agoOk so I won't be writing engine code necessarily in Swift. Or servers. But I'm not sure that's what it's for in the first place. How about user facing apps? Haven't we been through this a billion times on language of choice for job. Java, python, php etc. Each has a use. For a lot of what Swift will probably and should be used for I don't see this being an issue that hasn't already been beaten to death.
- Davertron 12y agoFor normal apps I would mostly agree, but for a lot of games performance matters.
- alejgoon 12y agoThe author seems to argue that these benchmarks disprove that Swift is fast. While I have no interest in Swift whatsoever, I'm unswayed by a few tight loop, simple benchmarks. It would be interesting to compare larger programs that are part of cpu-bound computations.
- dinkumthinkum 12y agoIf you watch the keynote, some bold, sort of specifc claims are made. There is some big disparity going on between what is being said in the WWDC and what this person what found. They are claiming over 100x speedups and this person is seeing it be 10x slower.
- ctide 12y agoIt'd be interesting to see how Objective-C would compare to Swift with ARC enabled. For the majority of us, who use ARC in our projects, these benchmarks don't mean a whole lot.
- wting 12y agoMaybe Swift is in reference to developer speed. From a PL / compiler standpoint it's pretty obvious that Swift is not going to match Objective C from a performance standpoint. That's why Apple used a micro benchmark comparing a static, compiled language vs a dynamic, interpreted language (Python) for marketing purposes. > We can’t know exactly what’s going on behind the scenes, but my hunch is some of what we take for granted in Objective-C – the straight C scalar data types – are actually classes in Swift. And the more you rely on classes, the more Automatic Reference Counting is in there somewhere, retaining and releasing like there’s no tomorrow, often for no good reason. If they switched from primitive types to classes for built-in types, that means vtables and an additional memory lookup per element access weakening caching and memory locality. Creating a Swift int would involve instantiating a class, incrementing the ARC count, and storing data compared to only storing data for primitive types. That explains why appending Swift's Ints to an array is so much slower than NSNumber. On a side note, this isn't as much of a problem with Java—despite everything being implemented with vtables—because of JIT and better CPU branch prediction. Disclaimer: I have no Objective C / Swift experience.
- spicyj 12y agoThey advertised favorable performance comparisons against both Objective-C and Python.
- mpweiher 12y agoObjC has an extreme performance range, from slower than Ruby to faster than Java with primitives (= C). It's easy to pick and choose.
- spicyj 12y agoI don't disagree; I was just clarifying on the parent's claim that Apple was benchmarking only against Python: > That's why Apple used a micro benchmark comparing a static, compiled language vs a dynamic, interpreted language (Python) for marketing purposes.
- nardi 12y ago
- gte910h 12y agoThese appear to be in line with unoptimized swift. mpweiher, can we have some gists of the proj or something for these numbers?
- mpweiher 12y agoNot my numbers, but I've gotten similar results, with optimized builds. Yes, unoptimized is even worse. See also: https://devforums.apple.com/message/974858#974858 https://devforums.apple.com/message/974858#974858 https://devforums.apple.com/thread/227905 https://devforums.apple.com/thread/227905 https://devforums.apple.com/message/971211#971211 https://devforums.apple.com/message/971211#971211
- bigdubs 12y agoI know they said on stage that they've spent a lot of time getting Swift fast, but aren't we still in beta? Isn't it a bit early to throw Swift under the bus for performance reasons?
- bsaul 12y agoProblem is everyone wants to know whether they just skip to swift right now or just wait a few years until the tech matures. Claiming a speed increase of x2 and ending with -x2 is a 400% difference. We're not talking minor differences here. As a sidenote, i've recently heard Core Data wasn't used internaly by Apple. Makes me think that i'll skip to swift once apple gives me the list of which of their app is coded with this language.
- bigdubs 12y agoBased on the differences between ++i and i+=1 alone I would think that there are just bugs they need to squash and that the open beta, and all these eyes on it, are going to help find the cases that are stumbling. Also, for many of the tests where it was a 400% or order of magnitude knock in performance there are a lot of comments in here refuting those results, so I'm not sure what to think yet.
- SoftwareMaven 12y agoSo we are comparing a language with 10+ years of optimizations against a language that just barely released in beta? We have no idea what settings the author chose (I'm not implying the author is attempting to deceive) and, assuming they are the defaults, it seems safe to assume Apple would choose very conservative defaults for the beta, especially the first release (it doesn't matter how fast it goes if it doesn't do what it's supposed to). In two months, after people start to understand what's going on, where the trouble spots are, and what the right way to fiddle bits, if we are still seeing performance like this, I'll be interested.
- na85 12y agoIsn't the point of a high-level dynamic language that you shouldn't need to do any bit-fiddling to write performant code? Swift seems self-defeating.
- alejgoon 12y agoI think that's hardly the case. Or are javascript and python and ruby self-defeating as well?
- deleted 12y ago[deleted]
- danbruc 12y agoDepends on the way you look at it - on the one hand JavaScript is a terrible language, on the other hand it is (more or less) the only thing you have available in a browser.
- alayne 12y agoNo, the point is to ease application development.
- Fomite 12y agoIf you could use a high-level dynamic language to write performant code without "bit-fiddling", why would anyone write in anything else?
- eddieroger 12y agoShouldn't we wait until Swift is in production for us and we're not using a compiler still built for beta before we come up accusations like these? Swift may not end up being the greatest thing since sliced bread, but how about we get our hands on the production-ready build of it before we decide.
- _random_ 12y agoApple posted a benchmark during the presentation. It was validated. They lied. Maybe they shouldn't boast about performance before releasing?
- eddieroger 12y agoThe tools they used to benchmark may not be the same ones we all got in the beta. Maybe our validations are the wrong data point.
- StefanKarpinski 12y agoBenchmarking things like "assignment" in compiled languages is pretty meaningless. The compiler may or may not eliminate the code altogether, depending on whether it determines that it's dead code. Assignment may not even actually do anything. Even with more complicated code that definitely does something, you have to be a bit careful to make sure that the whole computation isn't deemed to be dead. It's not unlikely that Swift's current LLVM optimization passes aren't doing as much as Objective-C does, even with optimizations turned on. This won't have as much effect on benchmarks that do some actual work, even simple ones.
- pessimizer 12y agoBrutal. Hopefully Swift makes up for it in developer productivity.
- ori_b 12y agoFor the empty loops, I'd be willing to be that the objc compiler is optimizing them away entirely. I suspect that the Swift compiler may not be that smart yet. There are plenty of times I've wanted to benchmark some C operation, looked at the generated assembly to figure out what is going on, and found that it was empty. Without source, disassembly, and something that you can test for yourself, this benchmark is rather uninformative.
- cromwellian 12y agoIt would be interesting to see something like Octane ported to Swift just so we can compare it to other languages that are supposedly "slow".
- deleted 12y ago[deleted]
- octopus 12y agoInteresting, did you run your tests on iOS8, OS X 10.10 or on OS X 10.9.3 ? Xcode or command line ?
- sriku 12y agoI think the optimizations for both need to be looked into carefully. For the following micro benchmark, Swift came in 1.5x slower than the C version. Both were built in Release mode with "-O3", and both with "Debug" checked OFF in their respective schemes. This factor is way smaller than the orders of magnitude being reported by the OP. Can't seem to find OP's benchmark source too. Swift - var j : Int = 0 var k : Int = 0; var i : Int = 0; for (i = 0; i < 1000; i += 1) { k = 0; for (j = 0; j < 1000000; j += 1) { k += j % 7; } } C - int j = 0, k = 0; int i = 0; for (i = 0; i < 1000; ++i) { k = 0; for (j = 0; j < 1000000; ++j) { k += j % 7; } } update: If I use "fastest unchecked" mode when building swift instead of the "fastest" mode, the swift version is 8% faster than the C version. machine: 1.7GHz core i5 macbook air (mid 2011), 4GB RAM. edit: For clarity, "1.5x slower" means "swift took 1.5x the time C took" and "8% faster" means "C took 1.08x the time swift took". edit: To OP - Please put a link to your benchmark source and projects somewhere, or show your project settings as a snapshot or something. For one, I don't believe any of your results based on my own trials. edit: Changed ++i and ++j to i+=1 and j+=1. This is the version that is 8% faster than C. OP seems right in that i+=1 seems faster than ++i. edit: (It's becoming impractical to keep this in sync with my project.) When using Int/int, I get swift 8% faster and when using Int64/int64_t, I get C to be 17% faster.
- m_mueller 12y agoFinally some actual data, thank you! Do you know the functionality behind 'fastest unchecked'? Does it only leave away memory bounds checks or arithmetic overflow checks or all of it? Depending on how it works, Swift could be quite interesting for HPC programming in the future. My dream workflow would be something like create working and correct program with all the comfort of modern languages / IDEs --> switch to unchecked-but-warn (or whatever that's called like), run some test suites and get rid of all warnings. --> switch on highest optimizations and switch off checking, verify results. --> port to multicore / multinode / accelerators using some directive based approach. note that you won't have much safety on todays' accelerators anyways, so removing that safety net when still in an easily debuggable environment is going to make life easier when debugging the accelerated code - it's all about being able to exclude error classes.
- PaulRobinson 12y agoBenchmarks like this miss the point of what it is to write production software code. The only people who care about this style of benchmark are people with very specialised use cases and academics (and their students), the latter of which are trying to prove something that is easy to prove, but doesn't really tell us anything of practical use. For most purposes, a user of your software is not going to feel the difference between 0.0006 seconds and 0.06 seconds. One is 100x faster, so is "better", but no user will care. The stuff that is really expensive (3D graphics, etc.) that users care about are being handed off into custom APIs that are highly optimised and hardware acceleration anyway. What's more important for most programmers when choosing a language is not "how fast is it for this code to run?", but "how quickly can I ship code?" Code in the app store next quarter is going to beat code you never get shipped, every single time. Not long ago I had a project I thought would be perfect for Haskell. I don't know Haskell but I am a very experienced Ruby-ist. I thought about using the project as a means to learn Haskell but wanted the code shipped within a month. I did it in Ruby. It's probably slower, it's perhaps not as elegant as a Haskell solution. But it's shipped. And that therefore beats the Haskell solution which does not exist. Swift appears to have modern features that mean many programmers will feel more comfortable developing applications using it, many of whom would have struggled with Obj-C. In that sense, it's going to beat Objective C for performance in the only benchmark that matters commercially: the one that gets ideas into user's hands the quickest. If you think the rate of development so far has been fast, well, just watch. It's about to get mental. I don't expect an academic to understand that, nor would I expect most junior/undergrad devs, and I can see in some special cases you would want to think about optimisation (in which case turn that little piece of code into C and link it in), but in the real World, that's how coding works.
- mantrax5 12y agoWise people know to avoid micro-benchmarks, but this is not a even micro-benchmark, it's a nano-benchmark.... Testing isolated things like empty loops or ...a single assignment. Those really don't make sense for modern compilers. The dead giveaway is having to modify your code in weird ways, so you don't trigger dead code elimination. If you trigger dead code elimination it means you code isn't producing anything, and you're benchmarking various edge cases that won't occur in the real world. You need to at least implement a simple algorithm or some small unit of functionality that makes sense in a real program. Swift's compiler is designed to aggressively inline and remove reference counting overhead in cohesive units of code, however if you test everything statement by statement, then whether you enable optimization or not, it can't optimize that much. That said I don't mind the article. It might help Apple discover places where Swift might be made even faster.