5 ms·

So Consumer Reports had Safari Developer Mode enabled which forced full page loads (plus some other bug). And Apple claims "this is not a setting used by custom

So Consumer Reports had Safari Developer Mode enabled which forced full page loads (plus some other bug). And Apple claims "this is not a setting used by customers and does not reflect real-world usage." While I agree most people wouldn't have that option enabled, I would argue that the average user is somewhere between the extremes. For example, most users are not disabling cache and browsing to 20 MB webpages and clicking refresh hundreds of times. But at the same time, neither are users leaving cache on and reloading the same page hundreds of times; they're browsing from site to site, refreshing Facebook feeds, etc. If Apple can make this statement, I'd be curious as to what tests they are performing. I'm almost certain the battery specs released by Apple are not indicative of how average users would use the MacBook (i.e. brigtness 10%, no movement, WiFi off, all background apps disabled, etc.). EDIT: as a reply mentions, Apple does provide (some) conditions used for their tests. However I'd still imagine that the conditions they do choose are the "most optimal" (for example - who do you know that only browses 25 sites in a single charge?)

IBM 10y ago
This is Apple's battery life test: >The wireless web test measures battery life by wirelessly browsing 25 popular websites with display brightness set to 12 clicks from bottom or 75 percent. The iTunes movie playback test measures battery life by playing back HD 1080p content with display brightness set to 12 clicks from bottom or 75 percent. The standby test measures battery life by allowing a system, connected to a wireless network and signed in to an iCloud account, to enter standby mode with Safari and Mail applications launched and all system settings left at default. Battery life varies by use and configuration. http://www.apple.com/macbook-pro/specs/ http://www.apple.com/macbook-pro/specs/
- r00fus 10y ago
  Thing is, with dynamic brightness (Macs have long been better at that than PCs), it's often the case that 75% brightness or less is used - at night for me it's more like 25% bright.
r00fus 10y ago
So the test if flawed. You can't simply disable cache and call it a good test. Do they do this on other browsers? Browse the same site over and over while disabling cache? Why did this not cause a problem when they ran Chrome (did they also disable cache there)? The CR tests raise more concerns about how CR runs their tests (and specifically how they run this test) as opposed to the MacbookPro.
- Oletros 10y ago
  Disabling the cache is the way to test battery life. And yes, they disable the cache in every browser and in every PC. And if you read the Apple answer, the problem was a bug in Safari so no, others browsers are not affected
- slantyyz 10y ago
  >> The CR tests raise more concerns about how CR runs their tests (and specifically how they run this test) as opposed to the MacbookPro. Why? At some point, all of these battery tests are arbitrary and not 100% reflective of the real world. If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models.
  - intopieces 10y ago
    >If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models. Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology? It's my understanding that this aspect is just now being revealed.
    - slantyyz 10y ago
      >> Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology? Probably not. >> It's my understanding that this aspect is just now being revealed. That's probably because nobody cared to ask before. The new Macbook Pros were already under a microscope even before Consumer Reports did their tests. Any less than stellar review from CR was going to get lots of eyeballs. In any case, the big reveal is not that caching was disabled, it's that there was a bug on Apple's side that is the likely root cause, and I'm guessing that bug didn't exist in the previous versions of Safari that were used to test the previous Macbook Pros that got the "Recommended" badge. CR's response today: http://www.consumerreports.org/apple/apple-releases-fix-to-macbook-pros-in-response-to-consumer-reports-battery-test-results/ http://www.consumerreports.org/apple/apple-releases-fix-to-m... Some key quotes: "We turn off caching as part of Consumer Reports' standard laptop test protocol" -- I can only take that at its word. It doesn't say "updated" standard protocol. "At Consumer Reports, we test every laptop from every manufacturer in a comparable way. Because people use laptops differently and because their usage can vary from day to day, our battery tests are not designed to be a direct simulation of a consumer’s experience. Rather, we look to control as many variables as possible, then perform a test that gives potential users a reasonable expectation of battery life when a computer’s processors, screen, memory, and antennas are under a light to moderate workload. This test has served as a good proxy for battery life on the hundreds of laptops in our ratings." They don't review hundreds of laptops per year, that's spread out over multiple years. So I don't think it's a huge leap of faith to assume that they turned off the cache on prior Macbook Pro models. FWIW, John Gruber does not "think it’s fair to say that disabling the caches is unfair or a flawed method" -- which says a lot.
eridius 10y ago
> for example - who do you know that only browses 25 sites in a single charge? You're right, most people probably browse fewer than 25 sites.
bluedino 10y ago
>> Consumer Reports had Safari Developer Mode enabled I always have it enabled and get 8+ hours from my previous model Pro.

IBM 10y ago

This is Apple's battery life test: >The wireless web test measures battery life by wirelessly browsing 25 popular websites with display brightness set to 12 clicks from bottom or 75 percent. The iTunes movie playback test measures battery life by playing back HD 1080p content with display brightness set to 12 clicks from bottom or 75 percent. The standby test measures battery life by allowing a system, connected to a wireless network and signed in to an iCloud account, to enter standby mode with Safari and Mail applications launched and all system settings left at default. Battery life varies by use and configuration. http://www.apple.com/macbook-pro/specs/ http://www.apple.com/macbook-pro/specs/

r00fus 10y ago
Thing is, with dynamic brightness (Macs have long been better at that than PCs), it's often the case that 75% brightness or less is used - at night for me it's more like 25% bright.

r00fus 10y ago

So the test if flawed. You can't simply disable cache and call it a good test. Do they do this on other browsers? Browse the same site over and over while disabling cache? Why did this not cause a problem when they ran Chrome (did they also disable cache there)? The CR tests raise more concerns about how CR runs their tests (and specifically how they run this test) as opposed to the MacbookPro.

Oletros 10y ago
Disabling the cache is the way to test battery life. And yes, they disable the cache in every browser and in every PC. And if you read the Apple answer, the problem was a bug in Safari so no, others browsers are not affected
slantyyz 10y ago
>> The CR tests raise more concerns about how CR runs their tests (and specifically how they run this test) as opposed to the MacbookPro. Why? At some point, all of these battery tests are arbitrary and not 100% reflective of the real world. If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models.
- intopieces 10y ago
  >If I'm not mistaken, previous Macbook Pro models underwent the exact same test and still got the "Recommended" badge. Nobody was complaining about the test methodology for those models. Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology? It's my understanding that this aspect is just now being revealed.
  - slantyyz 10y ago
    >> Did Consumer Reports indicate in any previous reviews that this is an aspect of their methodology? Probably not. >> It's my understanding that this aspect is just now being revealed. That's probably because nobody cared to ask before. The new Macbook Pros were already under a microscope even before Consumer Reports did their tests. Any less than stellar review from CR was going to get lots of eyeballs. In any case, the big reveal is not that caching was disabled, it's that there was a bug on Apple's side that is the likely root cause, and I'm guessing that bug didn't exist in the previous versions of Safari that were used to test the previous Macbook Pros that got the "Recommended" badge. CR's response today: http://www.consumerreports.org/apple/apple-releases-fix-to-macbook-pros-in-response-to-consumer-reports-battery-test-results/ http://www.consumerreports.org/apple/apple-releases-fix-to-m... Some key quotes: "We turn off caching as part of Consumer Reports' standard laptop test protocol" -- I can only take that at its word. It doesn't say "updated" standard protocol. "At Consumer Reports, we test every laptop from every manufacturer in a comparable way. Because people use laptops differently and because their usage can vary from day to day, our battery tests are not designed to be a direct simulation of a consumer’s experience. Rather, we look to control as many variables as possible, then perform a test that gives potential users a reasonable expectation of battery life when a computer’s processors, screen, memory, and antennas are under a light to moderate workload. This test has served as a good proxy for battery life on the hundreds of laptops in our ratings." They don't review hundreds of laptops per year, that's spread out over multiple years. So I don't think it's a huge leap of faith to assume that they turned off the cache on prior Macbook Pro models. FWIW, John Gruber does not "think it’s fair to say that disabling the caches is unfair or a flawed method" -- which says a lot.

eridius 10y ago

> for example - who do you know that only browses 25 sites in a single charge? You're right, most people probably browse fewer than 25 sites.

bluedino 10y ago

>> Consumer Reports had Safari Developer Mode enabled I always have it enabled and get 8+ hours from my previous model Pro.