5 ms·

What testing?

seanhunter 4y ago
The claim in TFA is they run a/b style tests where a sample group would have their battery deliberately drained to test performance of the app in low-power situations. Based on the general fixation on tests at Meta and Alphabet et al, it seems pretty plausible to me that they might try such a thing although: 1) the article is just the employee's claims in the context of employment arbitration, rather than evidence. For example they claim to have been shown a document about running such tests, but it's not clear whether that document has been entered into evidence (and certainly not made public). That document would presumably be subject to discovery in a lawsuit, although in arbitration it's unlikely it would be in the public record unless someone with standing made a freedom of information request (which may or may not reveal it). 2) It seems to me such a test would be unnecessary regardless of the ethics of it. You could just look at instances when a user's battery was low organically without draining it on purpose. The installed base is large enough that this would almost certainly work just as well for most practical purposes.
- a1studmuffin 4y ago
  Plus 2) would require additional design, engineering and QA work to implement the code to drain users' batteries. And it would cause plenty of users to uninstall/complain if Messenger was draining huge amounts of battery. It really doesn't make sense considering Messenger has 1.3 billion users, and remaining battery percent would likely be a bell curve across all users regardless of sample size.
- womitt 4y ago
  Exactly, why don't just wait, this would make no sense from a risk rewards perspective
  - otherme123 4y ago
    You want to test your app in negative scenarios, and one of them is "when and how to fail gracefully when battery is extremely low". My phone rarely goes below 10%, and maybe it has seen 5% one or two times. If Facebook waits for that, they will never run the tests. But if they detect my battery is at 10%, then drain it to 3%, launch the controlled failure testing, then drain the phone to complete shutdown, on the next boot they have their very valuable data.
    - foldr 4y ago
      Or...buy some phones and run the tests yourself.
      - otherme123 4y ago
        You could buy 100 iphones... Or you could "borrow" a million for free, to get rock solid stats. Also there are hundreds, if not thousands, of different phone models. The allegedly used way of negative testing is much better than buying phones, and is virtually free.
        foldr 4y ago
        I think Facebook Can afford to buy thousands of different phone models (and presumably already does?) Handling low battery states isn’t exactly rocket science anyway. Do less computation. The rest is up to the OS.
        pawelmurias 4y ago
        "Borrowing" a million phones and draining their battery down would make increase the perception of your app draining batteries. Seems pretty dubious if you would be willing to do that just to get a bit better low battery prices.
        isleyaardvark 4y ago
        It's also unethical.
        otherme123 4y ago
        Coming from a business whom founder said: "I have over 4,000 emails, pictures, addresses, SMS... People just submitted it. I don't know why. They trust me. Dumb f**s", ethical considerations has zero weight here.
    - mantas 4y ago
      Running that on end-user phones sounds wrong on so many levels.
- kyleyeats 4y ago
  None of the mentioned tests had anything to do with directly testing power drain or low-power performance. He just thinks they aren't giving enough weight to battery drain as an undesirable side effect of features that don't directly benefit the user. That's it. That's his whole gripe. They're testing how fast an image loads, that uses up the user's battery, and that's bad for the user.
  - seanhunter 4y ago
    TFA literally says the battery drain was the purpose, not a side-effect: If you believe former Facebook employee George Hayward, a data scientist, Facebook can secretly drain the battery on its users' cellphones on purpose.
    - kyleyeats 4y ago
      Okay, but the "you" in this sentence doesn't transfer to other readers. It's you when you read it, but when I read it, it's me. And I don't believe him.
- cma 4y ago
  > That document would presumably be subject to discovery in a lawsuit, although in arbitration it's unlikely it would be in the public record unless someone with standing made a freedom of information request (which may or may not reveal it). Facebook doesn't retain documents for any significant period of time to the point where they lobotomize their own memory to avoid stuff like this in lawsuits (and to reduce costs of discovery).
Shish2k 4y ago
For a random subset of users, when they view one image, the app downloads two images (eg webp and jpeg) and makes a note of which one decodes fastest; the goal being that in the future they can just download the fastest-decoding one, saving CPU load (and thus battery usage) for everybody in the future. The argument is that downloading two images when you only need one is “deliberately wasting battery”.

What testing?

The claim in TFA is they run a/b style tests where a sample group would have their battery deliberately drained to test performance of the app in low-power situations. Based on the general fixation on tests at Meta and Alphabet et al, it seems pretty plausible to me that they might try such a thing although: 1) the article is just the employee's claims in the context of employment arbitration, rather than evidence. For example they claim to have been shown a document about running such tests, but it's not clear whether that document has been entered into evidence (and certainly not made public). That document would presumably be subject to discovery in a lawsuit, although in arbitration it's unlikely it would be in the public record unless someone with standing made a freedom of information request (which may or may not reveal it). 2) It seems to me such a test would be unnecessary regardless of the ethics of it. You could just look at instances when a user's battery was low organically without draining it on purpose. The installed base is large enough that this would almost certainly work just as well for most practical purposes.

a1studmuffin 4y ago
Plus 2) would require additional design, engineering and QA work to implement the code to drain users' batteries. And it would cause plenty of users to uninstall/complain if Messenger was draining huge amounts of battery. It really doesn't make sense considering Messenger has 1.3 billion users, and remaining battery percent would likely be a bell curve across all users regardless of sample size.
womitt 4y ago
Exactly, why don't just wait, this would make no sense from a risk rewards perspective
- otherme123 4y ago
  You want to test your app in negative scenarios, and one of them is "when and how to fail gracefully when battery is extremely low". My phone rarely goes below 10%, and maybe it has seen 5% one or two times. If Facebook waits for that, they will never run the tests. But if they detect my battery is at 10%, then drain it to 3%, launch the controlled failure testing, then drain the phone to complete shutdown, on the next boot they have their very valuable data.
  - foldr 4y ago
    Or...buy some phones and run the tests yourself.
    - otherme123 4y ago
      You could buy 100 iphones... Or you could "borrow" a million for free, to get rock solid stats. Also there are hundreds, if not thousands, of different phone models. The allegedly used way of negative testing is much better than buying phones, and is virtually free.
      - foldr 4y ago
        I think Facebook Can afford to buy thousands of different phone models (and presumably already does?) Handling low battery states isn’t exactly rocket science anyway. Do less computation. The rest is up to the OS.
      - pawelmurias 4y ago
        "Borrowing" a million phones and draining their battery down would make increase the perception of your app draining batteries. Seems pretty dubious if you would be willing to do that just to get a bit better low battery prices.
      - isleyaardvark 4y ago
        It's also unethical.
        otherme123 4y ago
        Coming from a business whom founder said: "I have over 4,000 emails, pictures, addresses, SMS... People just submitted it. I don't know why. They trust me. Dumb f**s", ethical considerations has zero weight here.
  - mantas 4y ago
    Running that on end-user phones sounds wrong on so many levels.
kyleyeats 4y ago
None of the mentioned tests had anything to do with directly testing power drain or low-power performance. He just thinks they aren't giving enough weight to battery drain as an undesirable side effect of features that don't directly benefit the user. That's it. That's his whole gripe. They're testing how fast an image loads, that uses up the user's battery, and that's bad for the user.
- seanhunter 4y ago
  TFA literally says the battery drain was the purpose, not a side-effect: If you believe former Facebook employee George Hayward, a data scientist, Facebook can secretly drain the battery on its users' cellphones on purpose.
  - kyleyeats 4y ago
    Okay, but the "you" in this sentence doesn't transfer to other readers. It's you when you read it, but when I read it, it's me. And I don't believe him.
cma 4y ago
> That document would presumably be subject to discovery in a lawsuit, although in arbitration it's unlikely it would be in the public record unless someone with standing made a freedom of information request (which may or may not reveal it). Facebook doesn't retain documents for any significant period of time to the point where they lobotomize their own memory to avoid stuff like this in lawsuits (and to reduce costs of discovery).

Shish2k 4y ago

For a random subset of users, when they view one image, the app downloads two images (eg webp and jpeg) and makes a note of which one decodes fastest; the goal being that in the future they can just download the fastest-decoding one, saving CPU load (and thus battery usage) for everybody in the future. The argument is that downloading two images when you only need one is “deliberately wasting battery”.