9 ms·
iPhone update for Meltdown-Spectre: before/after performance benchmarks
- bitmapbrother 9y agoSomething isn't right with his benchmarks. My iPhone 6 benchmarks: Geekbench test taken on Dec 18, 2017 with iOS 11.2 Single Core: 1566 Multi-Core: 2697 Geekbench test taken on Jan 10, 2018 with iOS 11.2.2 Single Core: 1551 Multi-Core: 2675 Percentage slowdown: Single Core: 0.96% Multi-Core: 0.82%
- jasonlotito 9y agoDo you have Geekbench test with iOS 11.1.2 instead of 11.2?
- mkagenius 9y agoShouldn’t be needed if you go by absolute numbers - OP had 9xx instead 15xx .. so, clearly something is not right
- deleted 9y ago[deleted]
- stefanwlb 9y agoAn the apple police bring down another anti-applr website. Also, the slow down is related more to the battery = slow down than the new meltdown scare. Amazing, they reduced 50% of the CPU power so you don't have "battery" issues haha
- viraptor 9y agoOk, I'm sceptical about the results. The reason is that there doesn't seem to be a massive difference between the tests. Since this fix is about speculative exec, why would it affect crypto code which is very register based and branchless as much as sqlite which is full of branches and memory/storage based? Why would it affect AES which is hardware accelerated as much as integer processing which is not? I'm not saying this is impossible - maybe there's something that I'm missing. But it just doesn't add up at the moment. I'd love a more detailed / repeatable test.
- StringyBob 9y agoIs it possible that this benchmark has coincided with a battery that's just crossing the threshold of battery-throttling induced slowdown? The low scores in the article are in a similar ballpark to my (slow) iphone 6 on ios 10.3.3 with a 3yr old battery - i.e. about half what they should be with a fresh battery.
- scott_s 9y agoI can't speak about these benchmarks (the site is down). But Meltdown is not about speculative execution, it's about out-of-order execution. The fixes for Meltdown involve getting kernel data out of user virtual memory, and flushing the TLB if the processor does not have the pcid instruction. So context switches - including system calls - will tend to be more expensive, and if the processor does not implement pcid, way more expensive. See https://meltdownattack.com/ https://meltdownattack.com/. I haven't yet read the Spectre paper, but I can say that the Meltdown paper is very readable and walks readers through everything. I anticipate the Spectre paper is as well.
- jokr004 9y agoThis is about Spectre, not Meltdown, which is about speculative execution. From the article: > Apple released iOS 11.2.2 update to address Spectre security issues. I'm not sure why Meltdown was added to the HN title...
- deleted 9y ago[deleted]
- Terribledactyl 9y agoThe A8 doesn't have a dedicated crypto engine like Intel chips do (uncore I think they call it). The A8 does have a couple of AES specific instructions so each core can speed up the work, but they were executed speculatively and presumably are no longer done so.
- viraptor 9y agoFrom Geekbench knowledgebase: "Geekbench will use AES instructions when available, and fall back to software implementations otherwise." I expect this test to use them.
- enstyled 9y agoI made similar benchmarks recently on my iPhone 6S, running iOS iOS 11.1.2 vs 11.2.1, before and after replacing the battery: https://twitter.com/enstyled/status/948892447765131264 https://twitter.com/enstyled/status/948892447765131264 TLDR: It's 11.2.1 that is throttling the older iPhones, because of the battery wear.
- FBISurveillance 9y agoI've went through Denial, Anger, Bargaining, and now in an Acceptance stage after taking up to 31% performance hit on some of services managed by my team. Worst case has been Elasticsearch so far with our load pattern, taking that 31% hit. Oh well, too bad, enjoying the ride.
- bfrog 9y agoTime to switch to an Epyc host?
- dogma1138 9y agoAMD is vulnerable to Spectre and are not fixing variant 2 because it’s “near zero risk”. By near zero risk they mean that no one has yet reversed engineered their branch prediction which is required to effectively exploit it. Give it sometime and the BTI patch would have to be enforced on AMD CPUs also.
- FBISurveillance 9y agoWe're actually using those new 24-core Epyc hosts for our data science pipeline (HDP/Spark cluster) and they work much better than our previous generation Xeon-based builds. Unfortunately, with those things it takes plenty of time to re-evaluate hardware + in our case we're mostly in the cloud where Skylakes on GCP is the most we can get.
- euroclydon 9y agoWhat about using dedicated hardware and not running the patches? At attacker has to run malicious code on your server to exploit the vulns anyhow, right?
- jrochkind1 9y agoSure, but I think that would (potentially, with the proper exploit) turn any non-root-level code-execution-able breach of your machine of a machine into a root level one, no? Maybe that's okay?
- 9y ago
- cdolan 9y agoCan we get a before/after with a new battery vs a battery with 1k, then 10k, cycles on it?
- dijit 9y agoAaand it's down. Anybody have a mirror?
- cmg 9y agoarchive.org does: http://web.archive.org/web/20180110193140/https://melv1n.com/iphone-performance-benchmarks-after-spectre-update/ http://web.archive.org/web/20180110193140/https://melv1n.com...
- qrohlf 9y agoI'm curious what the delta is between perf hit on an iPhone 6 and an iPhone 8. I'm guessing that, given the deadline apple had to get this out, most of the attention & optimization went to current devices versus near-EOL devices like iPhone 6.
- rconti 9y agoWhy does Cloudflare tell me I'm able to browse a snapshot while the site is offline, but there is no snapshot?
- z5h 9y agoI get the same, and have seen it for other sites recently.
- jasonlotito 9y agoI've never known Cloudflare to do things well. It's the last CDN I'd want to use. Edit: For example, the 502 page is showing, but supposedly, you are supposed to see the site based on this: "However, because the site uses Cloudflare's Always Online™ technology you can continue to surf a snapshot of the site."
- rconti 9y agoYeah, this is the first time I've ever seen the "you can continue to surf a snapshot of the site" message along with the 502 though. Or maybe I've just never looked that closely.
- Pfhortune 9y agoThis is because, humorously, the snapshot is actually of the 502 page. Apparently Cloudflare kicked in too late.
- george3383 9y agoNo, it's not a snapshot of the 502 page, because I see my own IP there.
- chinhodado 9y agoIs that how is's supposed to work? Because any time this happened in the past to a site where it is overcrowded and the Cloudflare banner is showing, the 502 page is showing as well, so I thought it's part of the Cloudflare error page?
- tn_ 9y agoThis might sound dumb but.. with these Meltdown + Spectre bugs would an attacker be able to penetrate to gain full access only if a user downloads a native app or would it be possible through the browser/js?
- tibu 9y agoAs far I remember JS exploit is possible. That's why Mozilla patched Firefox already.
- neilsimp1 9y agoThis post, which I saw on HN yesterday, may answer some of that for you: https://webkit.org/blog/8048/what-spectre-and-meltdown-mean-for-webkit/ https://webkit.org/blog/8048/what-spectre-and-meltdown-mean-... Short answer: yes.
- MBCook 9y agoNot if their browser hasn’t been patched.
- anvandare 9y agoSpectre can give you read-access to the browser process via Javascript, yes, as they described in the original article: https://spectreattack.com/spectre.pdf https://spectreattack.com/spectre.pdf (section 4.3 on page 6). No write access, or access to the OS internals.
- StringyBob 9y agoWow. After ios 10.2.1 halved performance on my iphone 6 due to the battery issues, I next avoided 11.0 as I accepted responsiveness would be slower on the older iphone6 model. Now if we want these security fixes that's another -40%... Yikes! However, it looks like ios 11.2.2 and a new battery is still slightly faster than ios 10.3.3 and an old battery! (geekbench single/multi core score from article of 924/1616 vs my 844/1379)
- intopieces 9y agoI don't understand this logic at all. The fixes are being made for a reason... avoiding the update just means you are getting more performance for a less secure system. Are you betting on a law of averages to break in your favor? Hoping you're not a target of hackers? what am I missing with this strategy?
- deleted 9y ago[deleted]
- StringyBob 9y agoJust that I could be looking at a > 75% performance reduction on the same phone compared to when it was new due to the combo of battery degradation, security fixes and ios9 to ios11 general slow down. I think that if they've disabled some cpu core features that might reduce the maximum power loading anyway and make the double whammy impact of security and battery related slowdown unnecessary.
- untog 9y agoThe OP doesn't seem to suggest that they won't update, just that there is a considerable price to be paid in doing so. > Are you betting on a law of averages to break in your favor? Hoping you're not a target of hackers? Aside from anything else, I wouldn't be at all surprised if some people do take this gamble. They shouldn't, but if they're informed about it then it's in their own hands.
- qeternity 9y agoAll of these issues require local execution. It’s perfectly fine for many single user, performance critical use cases to make this type of security trade off.
- 2bitencryption 9y agothis leads me to a couple questions, hopefully you smart folk can answer: 1. I thought Spectre was "Intel-only", and Meltdown was the general case, which is less severe but effectively nearly everywhere? If so, how is an iPhone susceptible to Spectre? 2. Beyond that, I thought meltdown/spectre was an x86 problem. So why all this trouble on phones, with ARM? 3. I've read the first, simplest variant of meltdown, and it is so beautifully simple. Is this "speculative execution + cache timing" thing an entirely novel exploit, or have we seen incarnations of this before?
- deleted 9y ago[deleted]
- Scaevolus 9y ago1. is reversed: meltdown is intel-only. Spectre is a more generic speculation attack. 2. Apple's custom ARM chips have extensive speculative execution, and clearly their fixes for it are extremely expensive. 3. It's novel, which is why security researchers are so excited! Branching based side-channel attacks were previously only known to be able to track execution (so branch-based AES encryption was vulnerable to having its keys stolen)-- but reading arbitrary data is much stronger.
- qeternity 9y agoRe: #2 - it’s a speculative execution issue, not x86 specific.
- deleted 9y ago[deleted]
- JoachimSchipper 9y ago1: Spectre is the general case, Meltdown is the Intel-only evil cousin. 2: Spectre applies to all modern processors with speculative execution, which includes smartphone-class ARM processors. Raspberry Pi's and microprocessors ("toasters") are not affected. 3: There's some prior work - Spectre didn't fall from thin air - but using speculative execution as the basis of an exploit makes Spectre the first bugs in, I believe, a new class. (Meltdown, on the other hand, is just a silly mistake that shouldn't be repeated.)
- euph0ria 9y agoI wonder how much extra revenue this will bring to AWS/GCP etc selling more instances to cover the performance loss on servers?
- MBCook 9y agoIt’s a hit on AWS users, that means it’s also a hit for Amazon themselves. They’ll sell more servers, but they just lost 10% (or whatever) of their headroom all around the world.
- jdc0589 9y agosadly, this is the first thing I wondered when all the news came out.
- perfmode 9y agoIt's energy providers that will win the most.
- toomuchtodo 9y agoAll major cloud providers are buying renewables for their operations. Yay? At least the additional renewable generation capacity built out will exist for decades (while processors should become more efficient as these security issues are addressed and new hardware rolls out).
- hueving 9y ago>All major cloud providers are buying renewables for their operations. Yay? Not yay. This is an overnight massive bump so they are going to consume from the same existing pool of renewables, forcing utilities to use more non renewables to make up for the demand difference. Also, even when they do build out the renewable generation to make up the difference, it's still environmentally a problem due to the manufacturing of the renewable generation (solar panels, windmills, transformers, etc). We have no surplus of renewable energy to absorb this kinda stuff so more energy usage is always bad at this point. It's just slightly less bad when they commit to paying to fund renewable.
- bluedino 9y ago>> a significant decrease in performance on the iPhone 6 up to 50% Something's up. I updated my 6S yesterday and have noticed zero in performance changes or battery loss. Still plenty of bugs though, I was reading an email and the 'flag/file/trash/reply/new' bar totally went away. At least the touchscreen hasn't gone unresponsive, causing me to have to hit sleep/wake to toggle it back on. Maybe they finally fixed that.
- Infernal 9y agoOn an iPhone 6 - I actually gained performance from 11.2 to 11.2.2 (I neglected to run Geekbench on 11.2.1). Previously my best single-core score on iPhone 6 was 1536 (and only when battery was above 93% would I get in the 1500s), now today I got 1540 single core on only 66% battery - previously anything below about 70% battery would give me a score down in the 1200s.
- fulafel 9y ago6 and 6S have different CPUs.
- ponchoalv 9y agosame behaivor here... no perfomance / battery hit.
- oatmealsnap 9y ago6 or 6s?
- jobu 9y ago
- sschueller 9y agoIs the Samsung S8 CPU also affected? I saw that some ARMs are not affected. EDIT: It appears not the be on the list of affected ARMs [1]. The S8 Exynos 9 Octa 8895 is based on the Cortex-A53 [1] https://developer.arm.com/support/security-update https://developer.arm.com/support/security-update EDIT2: The Snapdragon 820 and 835 do appear to be affected which are the CPU in the US version of the S8
- JoachimSchipper 9y agoAssume that [EDIT: high-end - dogma1138 is right] smartphone-class processors and above are affected. Raspberry Pi and microcontroller-class ARMs are fine, though (but much slower!)
- dogma1138 9y agoMost of them aren't affected mid range ARM and below aren't vulnerable, basically if you have a flagship product you should worry. If we take the snapdragon example the 6XX's aren't affected while the 8XX are.
- mrpippy 9y agoIt looks like the Exynos 9 Octa 8895's 4 main (big) cores are a custom Samsung architecture which is almost certainly out-of-order, and is probably affected. The "little" cores are Cortex-A53, which is in-order and not affected.
- brador 9y agoIs this a "bug" we can recover from (in future hardware or software) or are we going to collectively take a lifetime -18 hit to our Moore's Law progress and lose the gains from this technique forever?
- perlgeek 9y agoAt least for Meltdown there are CPU microcode updates out there that should mitigate it, which means the OS level mitigations can be rolled back. And I haven't measured it yet, but I suspect the microcode mitigates are faster. Regarding Spectre, I think it's too early to tell.
- stcredzero 9y agoI'd like a new computer architecture, where communication between different parts of software systems was more explicitly treated as communication. Instead of an exception to pass control to the kernel, how about sending data? Instead of multiple CPU communication through memory (and therefore the cache) how about sending data?
- fancyfacebook 9y agoI keep getting bad gateway from some awful CDN, is there an alternative link?
- marcc 9y agoThat's Cloudflare and it's not sending the bad gateway, it's telling you that the upstream (the OP server) is returning a bad gateway.
- jasonlotito 9y agoExcept Cloudflare is saying this: "However, because the site uses Cloudflare's Always Online™ technology you can continue to surf a snapshot of the site" It shouldn't be telling you that the host is failing.
- ucha 9y agoI just ran GeekBench 4 on my iPhone X on iOS 11.2.2 vs 11.2.1. Single core: 4239 vs 4241 Multi-core: 10081 to 10203 So no difference.
- jpambrun 9y agoDoes GeekBench 4 make a lot of syscalls? My understanding is that these are the most affected by the mitigations.
- TheSithMaster 9y agoThe test done here was comparing 11.2.2 vs 11.1.2
- dogma1138 9y agoIsn’t 11.2.1 the battery patch?
- TheSithMaster 9y agoYou are correct, it appears to be that "patch". Source: https://discussions.apple.com/thread/8211392 https://discussions.apple.com/thread/8211392
- ucha 9y agoSo basically, the difference in performance is likely almost entirely due to the battery patch and has nothing to do the Meltdown-Spectre fix.
- rmcpherson 9y agoI wonder what the difference is between the 6 used in the benchmark in the article and the iPhone X that would cause these different results. They both support speculative execution. Either way, encouraging results to a fellow iPhone X user.
- 9y ago
- paws 9y agoMy iPhone X has been on 11.2.5 since the day it came out, and I haven't perceived any slowdowns. Have not done any benchmarking, and I'm not saying performance is the same as before, but anecdotally I haven't seen a difference. Just one guy's opinion.
- hueving 9y agoDo you game or do anything CPU intensive?
- jackson1way 9y agothe latest iOS version as of today is 11.2.2 I don't think 11.2.5 exists
- taspeotis 9y agoIt has existed in beta for a while. https://www.macrumors.com/2018/01/09/apple-seeds-ios-11-2-5-beta-4-to-developers/ https://www.macrumors.com/2018/01/09/apple-seeds-ios-11-2-5-...
- deleted 9y ago[deleted]
- bitmapbrother 9y agoGoogle cache: http://webcache.googleusercontent.com/search?ei=jXNWWojdC4vcjwSj2ozgCg&q=cache%3Ahttps%3A%2F%2Fmelv1n.com%2Fiphone-performance-benchmarks-after-spectre-meltdown-update%2F&oq=cache%3Ahttps%3A%2F%2Fmelv1n.com%2Fiphone-performance-benchmarks-after-spectre-meltdown-update http://webcache.googleusercontent.com/search?ei=jXNWWojdC4vc...
- Negative1 9y agoHaven't seen much coverage on how this affects game consoles like the Xbox and Playstation (which use AMD CPUs). Does anyone know if they've talked about patching it and how this would affect game performance (which is a pretty big deal for gamers who expect a consistent experience)?
- deleted 9y ago[deleted]
- Richs99 9y agoMike Ybarra has said there's no need for them to patch the Xbox One against these attacks: https://www.trueachievements.com/n31065/xbox-one-consoles-unaffected-by-spectre-and-meltdown-chip-issues https://www.trueachievements.com/n31065/xbox-one-consoles-un...
- schneidmaster 9y agoSample size of 1, but I Geekbench'd my iPhone X before and after upgrade. 11.2.1: Single-Core 4137, Multi-Core 9315 11.2.2: Single-Core 4039, Multi-Core 9876 Anecdotally as well, I haven't seen a noticeable difference in performance. So your mileage may vary substantially based on what device you have.
- gok 9y agoThis is...highly questionable. 11.2.2 only updates WebKit, which shouldn't affect GeekBench.
- eridius 9y agoThis article doesn't make any sense. The author is doing generic phone benchmarks. But the Spectre fix is a webkit fix only, not an OS-wide fix. If they're seeing performance regressions across the whole OS because of fixing Spectre, something's seriously wrong with their benchmark methodology. Edit: The author upgraded from iOS 11.1.2 to iOS 11.2.2. This isn't just a test of the Spectre fix. The most likely explanation here is upgrading to iOS 11.2 caused their iPhone 6 to start throttling due to battery wear (11.2 added throttling to iPhone 7, and it's plausible that it changed the conditions for throttling on iPhone 6). It's also possible that this is instead caused by the Meltdown patch, but these numbers are still way out of line with what was expected for Meltdown on iOS, whereas they're very much in line with what we've been seeing with battery throttling.
- cma 9y agoWhat sucks is there is no way to fix Spectre without going the battery throttled version.
- eridius 9y agoYeah there is. Get your battery replaced. Apple will replace your battery for $29 (as long as there's sufficient wear, but if you're seeing throttling, you're well past the cutoff).
- Anechoic 9y ago>as long as there's sufficient wear, That policy has changed, they will replace your battery upon request for $29 no matter what their diagnostic tests say.
- eridius 9y agoWhat's your source on that? Apple's message¹ said > Apple is reducing the price of an out-of-warranty iPhone battery replacement by $50 — from $79 to $29 — for anyone with an iPhone 6 or later whose battery needs to be replaced, available worldwide through December 2018. The "whose battery needs to be replaced" sure implies they're still going to actually test it. ¹https://www.apple.com/iphone-battery-and-performance/ https://www.apple.com/iphone-battery-and-performance/
- sWallo 9y agotested with GeekBench 4 on iPhone 8 Plus if anyone interested: - iOS 11.2.1: Single-Core 4257, Multi-Core: 10187 - iOS 11.2.2: Single-Core 4259, Multi-Core: 10287 Looks like no performance impact at all.
- apohn 9y agoI just updated my 6+ and ran Geekbench before and after. No major difference in scores and the single and multi-core benchmarks are consistent with the comparison numbers for other 6+ phones. Battery is in good health.
- mrmondo 9y agoI’ve more than a little skeptical of these results, I too am very disappointed in major CPU vendors (especially their PR and management teams), but these results seem more than a little smelly. I am what I’d consider a heavy iOS user on multiple devices and I don’t believe I’ve honestly noticed any difference at all. If these results were correct, I believe I’d notice at least a 10-15% decrease in performance - but no. What I haven’t looked at is battery life so I cannot comment on that.
- dotdi 9y agoCan't wait to see before/after benchmarks for Android. Something tells me it's going to be even worse than iOS.
- happycube 9y agoDepends on the phone. Lower end phones use cores like the A53 which don't have enough speculation HW to be affected at all.
- AuroraBob 9y agoI will repeat these benchmarks on my iPhone 6 when I get home tonight...but in the meantime, the benchmark numbers in this article look almost exactly like the ones I had as a result of battery throttling. I bet the actual patch-related performance hit is pretty minor.
- moepstar 9y agoI benchmarked my 8 Plus yesterday, updated from 11.2.1 to 11.2.2. No differences noticeable under same circumstances, maybe even slightly faster (Compute benchmark, Geekbench 4).