13 ms·

I'm always curious how a bug like this ships. I mean QA & Testing should catch it, sure. But even before then. Some engineer wrote code for FaceTime that has it

I'm always curious how a bug like this ships. I mean QA & Testing should catch it, sure. But even before then. Some engineer wrote code for FaceTime that has it open the microphone before the call is accepted. And transmit the audio over the network before the call is accepted. Who did that? And why? I'm not suggesting malice but I do wonder at the lack of defensive programming.

wmf 8y ago
This is a form of speculation which increases performance in the common case where the call is accepted.
- angott 8y ago
  I always wondered why FaceTime Audio has almost zero call establishment time on iOS, as opposed to WhatsApp or FB Messenger where you need to wait half a second before the audio feed kicks in after answering. Guess this is the reason!
  - kennywinker 8y ago
    That has not been my experience with facetime audio. For me it’s 3-10s wait after tapping the accept-call button before audio is connected.
- hopler 8y ago
  Similar to how Nextels cellular "push to talk" fake walkie talkies worked.
  - jsjohnst 8y ago
    > Similar to how Nextels cellular "push to talk" fake walkie talkies worked. Not really. The reason the PTT was virtually instant connect has nothing to do with optimization tricks, but rather is a purpose built design of the network tech it was using. Nextel PTT didn't go over a normal cellular network, it went over something called iDEN[0]. iDEN provides a trunked radio service[1] which has a similarish feature to a conventional two-way radio. Sprint acquired Nextel and as iDEN wasn't as relevant anymore with the advances in cellular networks (despite those who actually used the PTT functionality), in 2013 Sprint shutdown the network to use it for additional LTE bandwidth in the 800mhz band[2]. [0] https://en.wikipedia.org/wiki/IDEN https://en.wikipedia.org/wiki/IDEN [1] https://en.wikipedia.org/wiki/Trunked_radio_system https://en.wikipedia.org/wiki/Trunked_radio_system [2] https://en.wikipedia.org/wiki/Nextel_Communications https://en.wikipedia.org/wiki/Nextel_Communications
  - dragonwriter 8y ago
    No, PTT would send audio from the caller without the receiver accepting anything, but it wouldn't send anything from the receiver without action on the receiver end. It was obnoxious at times, but very different than the kind of privacy invasion that the receiver's sending data without active involvement is.
- gene91 8y ago
  This sound horrifying. If I use a hacked/modified FaceTime client, does this mean I'll be able to hear and see the other end before they pick up, even after the fix?
  - askafriend 8y ago
    Facetime calls are end to end encrypted so I doubt a modified client would work.
    - saagarjha 8y ago
      If FaceTime is implemented the way the parent comment mentions (which it very well may not) I don't see why you could make a client that performs a proper key exchange to set up an end-to-end call and simply pretend grab the video instead of hiding it behind the "dailing" screen?
      - askafriend 8y ago
        "The initial FaceTime connection is made through Apple server infrastructure that relays data packets between the users’ registered devices. Using APNs notifications and Session Traversal Utilities for NAT (STUN) messages over the relayed connection, the devices verify their identity certificates and establish a shared secret for each session. The shared secret is used to derive session keys for media channels streamed via the Secure Real-time Transport Protocol (SRTP). SRTP packets are encrypted using AES-256 in Counter Mode and HMAC-SHA1. Subsequent to the initial connection and security setup, FaceTime uses STUN and Internet Connectivity Establishment (ICE) to establish a peer-to-peer connection between devices, if possible." Source: https://www.apple.com/business/docs/iOS_Security_Guide.pdf https://www.apple.com/business/docs/iOS_Security_Guide.pdf
        saagarjha 8y ago
        I'm not doubting whether FaceTime is end-to-end encrypted, I'm not sure whether FaceTime sends data before the call is accepted for speculation reasons.
laythea 8y ago
Maybe it did used to work, but now because of a seemingly unrelated bug fix, it has broken it. Wack-a-mole style. Takes the engineering right out of software engineering if you ask me.
mygo 8y ago
doesn’t help that their bug bounty program is invite-only ( at least the last time I checked )
- saagarjha 8y ago
  Their bug bounty program seems to mostly be aimed at jailbreakers, rather than bugs like these.
  - mygo 8y ago
    They can aim it at who they want. When it's invite-only they'll only get what they're looking for. I'm certain that just having open bug hunters looking would have at least brought this up. I remember noticing this bug myself but also remember it not being worth my while to report it.
oil25 8y ago
There is a general lack of unit testing culture at Apple. This became all too apparent after Goto Fail (https://martinfowler.com/articles/testing-culture.html https://martinfowler.com/articles/testing-culture.html) and remains an unacceptable risk for using iOS products in my view. Incidents such as https://www.macrumors.com/2017/11/28/macos-high-sierra-bug-admin-access/ https://www.macrumors.com/2017/11/28/macos-high-sierra-bug-a... make me skeptical about using macOS, too.
deleted 8y ago
[deleted]
aphextron 8y ago
Apple's application programming has fallen off a cliff in the last 4 years. I blame Swift. There's nothing wrong with the language itself, but its outward similarities to.... other dynamic languages I feel has led to a rot in the core of what was once a rock solid culture of embedded C/C++/Objective-C programmers building robust, well architected applications. The average Apple app today is just as buggy and disjointed as anything else on the App Store, and that never used to be the case.
- maxerickson 8y ago
  You need to integrate the famous iTunes into this thesis. (people have complained about it approximately since it was released)
  - peteretep 8y ago
    iTunes has always felt robust and well-architected, but it's UI has been a dumpster fire since whenever.
    - audiometry 8y ago
      No way. I regularly have problems with it hanging or just ceasing to respond to “play” command. Ana this in on OS X, not windows.
    - Gigablah 8y ago
      "This app is preventing shutdown". Yep, it's always iTunes.
  - aphextron 8y ago
    >You need to integrate the famous iTunes into this thesis. Desktop iTunes is a dumpster fire, but their iOS software is what I was specifically referring to.
    - jimmaswell 8y ago
      I actually liked some of the versions of iTunes for Windows back when I had to use it with an iPod Touch around 2008 to 2012 or so. For just playing music from my music library, the interface wasn't bad. You just had to turn off the option to automatically convert everything into Apple's format.
Someone1234 8y ago
It reminds me of this MacOS bug from last year, where simply hitting the login box over and over with no password would eventually bypass the security entirely: https://www.theregister.co.uk/2017/11/28/root_access_bypass_macos_high_sierra/ https://www.theregister.co.uk/2017/11/28/root_access_bypass_... And this other MacOS bug, also from last year, where the password hint would contain the plain text encryption password: https://www.theregister.co.uk/2017/10/05/apple_patches_password_hint_bug_that_revealed_password/ https://www.theregister.co.uk/2017/10/05/apple_patches_passw... All within a month of each other.
- judge2020 8y ago
  Not Apple related, but you could log into any linux box using a specific few versions of grub2 by hitting backspace 28 times: https://motherboard.vice.com/en_us/article/ezpdqz/hack-into-a-linux-computer-by-hitting-the-backspace-28-timestimes/ https://motherboard.vice.com/en_us/article/ezpdqz/hack-into-...
  - codedokode 8y ago
    Also, you can bypass screen lock in Ubuntu by removing the HDD [1]. The bug still is not fixed. [1] https://bugs.launchpad.net/ubuntu/+source/unity/+bug/1777415 https://bugs.launchpad.net/ubuntu/+source/unity/+bug/1777415
    - laumars 8y ago
      Honestly, I wouldn’t but that in the same class if bugs as those that preceded it because if the attacker he removed the HDD he will have access to your contents anyway (unless the HDD is encrypted) and it’s not a quick and convenient process either (unlike tapping backspace multiple times). That said, I also don’t agree that this bug should never get fixed either.
      - nurettin 8y ago
        This is a very naive estimation of what might happen when someone has access to all your running software. Since it is linux, it loads executables into memory and is able to run them even when they are deleted along with runtime dependencies. Anything that caches anything to anywhere other than disk will be accessible. Your memcache, your redis, some databases, keychains, your non-userdata browser sessions.
padiyar83 8y ago
Yeah. Never attribute to malice that which is adequately explained by stupidity ~Hanlon's razor
segmondy 8y ago
Most likely 2 different teams worked on it. One worked on accepting calls, the other worked on transmitting audio, another probably worked on video. Then it got integrated and complete e2e testing wasn't done.
- bduerst 8y ago
  This are my thoughts as well. Probably a recent release that exposed this also didn't go through full e2e testing because hey, it passed previously.
- consumer451 8y ago
  How would an e2e test catch this though? What would the test condition be? Wouldn’t I have to be checking for audio from the other party during all seemingly random events like adding a person to the call group? Apologies in advance for my ignorance, I haven’t written code for a long time.
  - ddebernardy 8y ago
    Not sending any audio or video until a call gets accepted seems like a clearcut test case. They probably have a few tests like that, too, but forgot to add weird edge cases.
  - flukus 8y ago
    > How would an e2e test catch this though? I doubt there would be a specific test (or maybe there would, real testers are better than me at thinking of this stuff), but there should be logs for events like "microphone turned on" and "user joined group chat" and the testers should be monitoring those logs.
ct520 8y ago
This reminds me of having the Nokia 3650 phone. I could hear phone calls before the call was accepted.. awesome uh feature .
trumped 8y ago
Apple only cares about your privacy when talking out loud. (and Apple fanboys still don't want to believe it)
jtr_47 8y ago
Maybe it's not a bug, but something akin to a backdoor that has been uncovered.
m3kw9 8y ago
Would be a server bug for sure
ipstas 8y ago
To get a bug like that is easy. The question, who the hell still using FB facebug. And why?
throwaway415415 8y ago
Abstraction. I can tell you with 100% confidence that any of these logic bugs are created by unecessary abstraction. Anybody working in security will tell you the same. Piles of abstractions make it impossible to find out these bugs. You need a month of work to understand these codebases, often only the main developer has the architecture in its head.
- saagarjha 8y ago
  > I can tell you with 100% confidence that any of these logic bugs are created by unecessary abstraction. That's a lot of confidence…
lathiat 8y ago
It generally happens because you don’t follow a defined state machine. An example of how this might happen is when starting the call the microphone isn’t opened. Then you add someone else to the Group FaceTime, the event handler for they didn’t stop to consider if the call is active (just assumed it is) and now the code for that handler opens a new port to the microphone so that it can encrypt the audio stream differently for that recipient. Super easy and not remotely malicious. It’s a failed state check. The actual bug here might be different but that’s an easy example. But it may also effectively be the bug since all the examples mention adding yourself to the call.
- Hydraulix989 8y ago
  I did a bit of WebRTC development for video chat, and the state machine for that is one of the most complicated I've ever dealt with. Even household name-brand commercial providers don't handle all of the edge cases. It took me about a week to get it right. Session negotiation gone wrong can easily cause audio to be heard before the call is established (and this will certainly happen with the naive implementation -- even Google's own reference implementation had issues). If you're curious, check out this flowchart slide from a Google I/O WebRTC talk: https://image.slidesharecdn.com/2014q2-geekandkranky-scalability-150620155937-lva1-app6891/95/webrtc-infrastructure-scalability-notes-geekn-kranky-june-2014-google-sf-9-638.jpg?cb=1450690287 https://image.slidesharecdn.com/2014q2-geekandkranky-scalabi...
mcpherrinm 8y ago
This seems like a state machine bug to me. If you consider what happens when you add a third person to a call: You start sending them everyone's audio! That's the desired behavior and exactly what happened here. Except the client should have probably checked if the call had been accepted first. That's why I say it's a state machine bug: The "Send audio" function should have never been activated in state "waiting to accept".
- auiya 8y ago
  > If you consider what happens when you add a third person to a call: You start sending them everyone's audio! Not before they accept the call connection request you don't.
  - mcpherrinm 8y ago
    Well clearly. That's the bug.
yalok 8y ago
My guess is that it’s an unfortunate combination of several problems: - audio and video capture has to start going before call is actually established at signaling level, in order to minimize call establishment delay. Audio maybe going through Bluetooth, for example, and waking up Handsfree mode of BT may take 1-2 sec - most of the group calling functionality was developed by a separate team, and group calling signaling may be loosely integrated at UI level, where, once UI triggers a switch to a group call - internally, the whole new library may kick in and get the current 1-1 call state transferred to it. - when this “transfer” happens, the state of the first 1-1 call gets affected (at either local or remote side (due to signaling), which leads to either remote side think that the call was answered (a lack of protection in the call signaling state machine to ensure it was users UI action) or local side thinks it’s ok that remote users answers the call (in this case FT must have streamed audio even during 1-1 call establishment phase) - lack of a check for your own phone number added to a call. This, due to having the same IDs/tokens twice in a group call, may lead to unexpected call signaling state machine switch - lack of manual testing with focus on edge cases (like the described flow to repro the bug may not be the main flow for how users start group calls on FT) I never worked at Apple, but I built VoIP stuff for the past 20 years.
- nolok 8y ago
  > audio and video capture has to start going before call is actually established at signaling level, in order to minimize call establishment delay. Audio maybe going through Bluetooth, for example, and waking up Handsfree mode of BT may take 1-2 sec As a user, while I can accept capture starting before I answer, I cannot accept sending. I understand how it helps the speed of establishing calls. But it means the only thing needed to spy on me from that is a software change ON THE OTHER SIDE. No way to know from my side if I'm good or not.
  - akvadrako 8y ago
    I don't think that's what he meant. It can be sending empty audio, but the whole path needs to be up and running.
  - enedil 8y ago
rahoulb 8y ago
Although others have mentioned state machines I wonder if it’s a deliberate feature with an unforeseen side effect. The Apple Watch walkie talkie feature allows you to talk with anyone who has approved you and has walkie talkie switched on. But it appears to use FaceTime under the hood as when it was launched it failed if FaceTime was switched off on your phone. So the “auto answer” mechanism is already in there, just getting triggered at the wrong time?
TomK32 8y ago
I'm pretty sure that person also wrote a few lines so the phone being called won't show there's a call... /tinfoil
StreamBright 8y ago
Lack of formal methods in the design phase. It would be trivial to express a rule that you cannot hear audio from the called person before they accept it with it.
nvarsj 8y ago
You’re curious how a complex piece of software can have a bug? Come on now. You’re either naive or being hyperbolic.
blablabla123 8y ago
> I'm not suggesting malice but I do wonder at the lack of defensive programming. I've never worked at Apple but neither have I in 10 years seen people truly appreciate any attempts I did in defensive programming. On the contrary, usually I hear - sometimes loud - complaints about that. Usually from fellow engineers when they see it but on occasions also from non technical people. It's a joke that everybody is so sad about bad application security but at the same time virtually nobody cares about it at all when involved in an actual development project. Classical case of NIMBY. FWIW Apple's security is far over average compared to other companies. But I guess they cannot isolate themselves 100%.
C14L 8y ago
If the implementation is complicated enough, the code that opens the mic and the code that accepts the call are so far away from each other that its impossible to see. That's why keeping things simple is the greatest art in programming.
Haga 8y ago
When in doubt collect all the data.
JustSomeNobody 8y ago
I agree this should have been caught in QA/dev testing. However, I would bet a dollar this wasn't "Some Engineer" deciding to do it this way. This design was the culmination of some back and forth discussions and it is this way for a sound (no pun) reason.
skywhopper 8y ago
The code involved is a lot more complicated, involving a lot more pieces written by different teams working in concert with each other. There's not a single routine that takes the steps you describe that got a few lines out of order or something.
scraft 8y ago
Possibly a product owner trying out the latest build, receiving a call, accepting it, and then waiting for the call initiator to receive the message that the call has been accepted, and then start sending data and asking: "Why doesn't it take X seconds before I can start talking". To which the engineers possibly explained the reasons and the product owner saying: "But I want it instant, let's bypass all this extra stuff and get a proof of concept instant answer working" To which the engineer said: "But we'd technically be sending data before the call has even been accepted" To which the product owner said: "That's okay, the user can't actually see that data, let us just get this in for now, we can worry about the security/privacy side later". To which the engineer said "but, but, but" saw the product owners eyes glaze over and just made the commit: Commit 1279: Remove very important security/privacy feature of ensuring no data is transmitted until the call has been accepted. This is again my best judgement, do not come to me when this blows up, please speak to the product owner. Then went to the pub in despair.
- whamlastxmas 8y ago
  I think it's much more likely their code is a mess and it's not super obvious when stuff starts and ends and this is just an unintentional mistake.
  - pbhjpbhj 8y ago
    You hear this excuse all the time, don't FAANG employ the world's very best developers? Maybe their code is a mess for orthogonal reasons - management, profit-motive? Aside: I thought I'd heard devs have automated analysers that step through and find all possible code paths, allowing complex code to be audited for security issues and such? Presumably that's how these sorts of bugs should be found in testing.
    - MagicPropmaker 8y ago
      Judging from their quality issues, I'm not so sure they're attracting the "world's very best developers" any more.
jeromebaek 8y ago
An old product like FaceTime comes with tons of legacy code. Mistakes like this while fiddling with legacy code is not unreasonable.
JangoSteve 8y ago
I'm wondering if it's something really simple, like comparing the unique number of callers versus the total number who have accepted the call. Imagine a method for group calling that returns whether or not the call should be considered active or not by seeing if the number of callers is equal to the number who have accepted the call. But now imagine the method that tallies up the number of callers does a unique count by phone number, while the method that tallies how many people have accepted the call does not. Since adding your own number is adding a caller who has already accepted a call, you end up with unique_callers (2, you and them) == total_accepted (2, you and you). This could be tested by adding a third person to a group call twice (if iOS will let you do that) instead of adding yourself to the call. Furthermore, if the interface on the recipient's phone only looks at whether or not they've accepted the call, that would explain why the call doesn't auto-accept on their end and go into the call before they've accepted or rejected the call.
update 8y ago
> I mean QA & Testing should catch it, sure. I believe they have been working on FaceTime Group for 5+ years now. In 2015, Modern Family had an episode where the entire plot focused around the family using FaceTime on their iPhones --- but they had the Group feature[1]! I remember being blown away when I saw that, figured it'd be coming in the next iOS update. It didn't. 4 years later, they finally released it..then recalled it immediately [1] https://www.mlive.com/entertainment/index.ssf/2015/02/tech_review_modern_family_epis.html https://www.mlive.com/entertainment/index.ssf/2015/02/tech_r... edit: managed to find a short clip from said Modern Family episode: https://www.youtube.com/watch?v=vy3jUOBxQuI https://www.youtube.com/watch?v=vy3jUOBxQuI Though in the full episode you see a lot more of the Group feature
m3kw9 8y ago
Is rarely that they didn’t test it, it was likely caused by a new scenario that they have not thought of that brings out this bug.
foobaw 8y ago
Agreed - looks like something very simple to add as a test-case. If not manual, an automated QA process could've easily caught this if it was even remotely robust enough. The steps to reproduce is trivial so I'm shocked.
iforgotpassword 8y ago
Our company's phone system, or our provider, had a problem with outgoing calls once: you couldn't call users of a certain mobile provider, but only if they were currently using GSM. Anything newer would make it work. The symptom was that even when the cell user picked up, you'd still here the sound like their phone is still ringing. That alone was already mindboggling to me, but the crazy part was that with certain phone models (at our company), the cell user would be able to hear you while you still thought their phone was ringing.

wmf 8y ago

This is a form of speculation which increases performance in the common case where the call is accepted.

angott 8y ago
I always wondered why FaceTime Audio has almost zero call establishment time on iOS, as opposed to WhatsApp or FB Messenger where you need to wait half a second before the audio feed kicks in after answering. Guess this is the reason!
- kennywinker 8y ago
  That has not been my experience with facetime audio. For me it’s 3-10s wait after tapping the accept-call button before audio is connected.
hopler 8y ago
Similar to how Nextels cellular "push to talk" fake walkie talkies worked.
- jsjohnst 8y ago
  > Similar to how Nextels cellular "push to talk" fake walkie talkies worked. Not really. The reason the PTT was virtually instant connect has nothing to do with optimization tricks, but rather is a purpose built design of the network tech it was using. Nextel PTT didn't go over a normal cellular network, it went over something called iDEN[0]. iDEN provides a trunked radio service[1] which has a similarish feature to a conventional two-way radio. Sprint acquired Nextel and as iDEN wasn't as relevant anymore with the advances in cellular networks (despite those who actually used the PTT functionality), in 2013 Sprint shutdown the network to use it for additional LTE bandwidth in the 800mhz band[2]. [0] https://en.wikipedia.org/wiki/IDEN https://en.wikipedia.org/wiki/IDEN [1] https://en.wikipedia.org/wiki/Trunked_radio_system https://en.wikipedia.org/wiki/Trunked_radio_system [2] https://en.wikipedia.org/wiki/Nextel_Communications https://en.wikipedia.org/wiki/Nextel_Communications
- dragonwriter 8y ago
  No, PTT would send audio from the caller without the receiver accepting anything, but it wouldn't send anything from the receiver without action on the receiver end. It was obnoxious at times, but very different than the kind of privacy invasion that the receiver's sending data without active involvement is.
gene91 8y ago
This sound horrifying. If I use a hacked/modified FaceTime client, does this mean I'll be able to hear and see the other end before they pick up, even after the fix?
- askafriend 8y ago
  Facetime calls are end to end encrypted so I doubt a modified client would work.
  - saagarjha 8y ago
    If FaceTime is implemented the way the parent comment mentions (which it very well may not) I don't see why you could make a client that performs a proper key exchange to set up an end-to-end call and simply pretend grab the video instead of hiding it behind the "dailing" screen?
    - askafriend 8y ago
      "The initial FaceTime connection is made through Apple server infrastructure that relays data packets between the users’ registered devices. Using APNs notifications and Session Traversal Utilities for NAT (STUN) messages over the relayed connection, the devices verify their identity certificates and establish a shared secret for each session. The shared secret is used to derive session keys for media channels streamed via the Secure Real-time Transport Protocol (SRTP). SRTP packets are encrypted using AES-256 in Counter Mode and HMAC-SHA1. Subsequent to the initial connection and security setup, FaceTime uses STUN and Internet Connectivity Establishment (ICE) to establish a peer-to-peer connection between devices, if possible." Source: https://www.apple.com/business/docs/iOS_Security_Guide.pdf https://www.apple.com/business/docs/iOS_Security_Guide.pdf
      - saagarjha 8y ago
        I'm not doubting whether FaceTime is end-to-end encrypted, I'm not sure whether FaceTime sends data before the call is accepted for speculation reasons.

laythea 8y ago

Maybe it did used to work, but now because of a seemingly unrelated bug fix, it has broken it. Wack-a-mole style. Takes the engineering right out of software engineering if you ask me.

mygo 8y ago

doesn’t help that their bug bounty program is invite-only ( at least the last time I checked )

saagarjha 8y ago
Their bug bounty program seems to mostly be aimed at jailbreakers, rather than bugs like these.
- mygo 8y ago
  They can aim it at who they want. When it's invite-only they'll only get what they're looking for. I'm certain that just having open bug hunters looking would have at least brought this up. I remember noticing this bug myself but also remember it not being worth my while to report it.

oil25 8y ago

There is a general lack of unit testing culture at Apple. This became all too apparent after Goto Fail (https://martinfowler.com/articles/testing-culture.html https://martinfowler.com/articles/testing-culture.html) and remains an unacceptable risk for using iOS products in my view. Incidents such as https://www.macrumors.com/2017/11/28/macos-high-sierra-bug-admin-access/ https://www.macrumors.com/2017/11/28/macos-high-sierra-bug-a... make me skeptical about using macOS, too.

deleted 8y ago

[deleted]

aphextron 8y ago

Apple's application programming has fallen off a cliff in the last 4 years. I blame Swift. There's nothing wrong with the language itself, but its outward similarities to.... other dynamic languages I feel has led to a rot in the core of what was once a rock solid culture of embedded C/C++/Objective-C programmers building robust, well architected applications. The average Apple app today is just as buggy and disjointed as anything else on the App Store, and that never used to be the case.

maxerickson 8y ago
You need to integrate the famous iTunes into this thesis. (people have complained about it approximately since it was released)
- peteretep 8y ago
  iTunes has always felt robust and well-architected, but it's UI has been a dumpster fire since whenever.
  - audiometry 8y ago
    No way. I regularly have problems with it hanging or just ceasing to respond to “play” command. Ana this in on OS X, not windows.
  - Gigablah 8y ago
    "This app is preventing shutdown". Yep, it's always iTunes.
- aphextron 8y ago
  >You need to integrate the famous iTunes into this thesis. Desktop iTunes is a dumpster fire, but their iOS software is what I was specifically referring to.
  - jimmaswell 8y ago
    I actually liked some of the versions of iTunes for Windows back when I had to use it with an iPod Touch around 2008 to 2012 or so. For just playing music from my music library, the interface wasn't bad. You just had to turn off the option to automatically convert everything into Apple's format.

Someone1234 8y ago

It reminds me of this MacOS bug from last year, where simply hitting the login box over and over with no password would eventually bypass the security entirely: https://www.theregister.co.uk/2017/11/28/root_access_bypass_macos_high_sierra/ https://www.theregister.co.uk/2017/11/28/root_access_bypass_... And this other MacOS bug, also from last year, where the password hint would contain the plain text encryption password: https://www.theregister.co.uk/2017/10/05/apple_patches_password_hint_bug_that_revealed_password/ https://www.theregister.co.uk/2017/10/05/apple_patches_passw... All within a month of each other.

judge2020 8y ago
Not Apple related, but you could log into any linux box using a specific few versions of grub2 by hitting backspace 28 times: https://motherboard.vice.com/en_us/article/ezpdqz/hack-into-a-linux-computer-by-hitting-the-backspace-28-timestimes/ https://motherboard.vice.com/en_us/article/ezpdqz/hack-into-...
- codedokode 8y ago
  Also, you can bypass screen lock in Ubuntu by removing the HDD [1]. The bug still is not fixed. [1] https://bugs.launchpad.net/ubuntu/+source/unity/+bug/1777415 https://bugs.launchpad.net/ubuntu/+source/unity/+bug/1777415
  - laumars 8y ago
    Honestly, I wouldn’t but that in the same class if bugs as those that preceded it because if the attacker he removed the HDD he will have access to your contents anyway (unless the HDD is encrypted) and it’s not a quick and convenient process either (unlike tapping backspace multiple times). That said, I also don’t agree that this bug should never get fixed either.
    - nurettin 8y ago
      This is a very naive estimation of what might happen when someone has access to all your running software. Since it is linux, it loads executables into memory and is able to run them even when they are deleted along with runtime dependencies. Anything that caches anything to anywhere other than disk will be accessible. Your memcache, your redis, some databases, keychains, your non-userdata browser sessions.

padiyar83 8y ago

Yeah. Never attribute to malice that which is adequately explained by stupidity ~Hanlon's razor

segmondy 8y ago

Most likely 2 different teams worked on it. One worked on accepting calls, the other worked on transmitting audio, another probably worked on video. Then it got integrated and complete e2e testing wasn't done.

bduerst 8y ago
This are my thoughts as well. Probably a recent release that exposed this also didn't go through full e2e testing because hey, it passed previously.
consumer451 8y ago
How would an e2e test catch this though? What would the test condition be? Wouldn’t I have to be checking for audio from the other party during all seemingly random events like adding a person to the call group? Apologies in advance for my ignorance, I haven’t written code for a long time.
- ddebernardy 8y ago
  Not sending any audio or video until a call gets accepted seems like a clearcut test case. They probably have a few tests like that, too, but forgot to add weird edge cases.
- flukus 8y ago
  > How would an e2e test catch this though? I doubt there would be a specific test (or maybe there would, real testers are better than me at thinking of this stuff), but there should be logs for events like "microphone turned on" and "user joined group chat" and the testers should be monitoring those logs.

ct520 8y ago

This reminds me of having the Nokia 3650 phone. I could hear phone calls before the call was accepted.. awesome uh feature .

trumped 8y ago

Apple only cares about your privacy when talking out loud. (and Apple fanboys still don't want to believe it)

jtr_47 8y ago

Maybe it's not a bug, but something akin to a backdoor that has been uncovered.

m3kw9 8y ago

Would be a server bug for sure

ipstas 8y ago

To get a bug like that is easy. The question, who the hell still using FB facebug. And why?

throwaway415415 8y ago

Abstraction. I can tell you with 100% confidence that any of these logic bugs are created by unecessary abstraction. Anybody working in security will tell you the same. Piles of abstractions make it impossible to find out these bugs. You need a month of work to understand these codebases, often only the main developer has the architecture in its head.

saagarjha 8y ago
> I can tell you with 100% confidence that any of these logic bugs are created by unecessary abstraction. That's a lot of confidence…

lathiat 8y ago

It generally happens because you don’t follow a defined state machine. An example of how this might happen is when starting the call the microphone isn’t opened. Then you add someone else to the Group FaceTime, the event handler for they didn’t stop to consider if the call is active (just assumed it is) and now the code for that handler opens a new port to the microphone so that it can encrypt the audio stream differently for that recipient. Super easy and not remotely malicious. It’s a failed state check. The actual bug here might be different but that’s an easy example. But it may also effectively be the bug since all the examples mention adding yourself to the call.

Hydraulix989 8y ago
I did a bit of WebRTC development for video chat, and the state machine for that is one of the most complicated I've ever dealt with. Even household name-brand commercial providers don't handle all of the edge cases. It took me about a week to get it right. Session negotiation gone wrong can easily cause audio to be heard before the call is established (and this will certainly happen with the naive implementation -- even Google's own reference implementation had issues). If you're curious, check out this flowchart slide from a Google I/O WebRTC talk: https://image.slidesharecdn.com/2014q2-geekandkranky-scalability-150620155937-lva1-app6891/95/webrtc-infrastructure-scalability-notes-geekn-kranky-june-2014-google-sf-9-638.jpg?cb=1450690287 https://image.slidesharecdn.com/2014q2-geekandkranky-scalabi...

mcpherrinm 8y ago

This seems like a state machine bug to me. If you consider what happens when you add a third person to a call: You start sending them everyone's audio! That's the desired behavior and exactly what happened here. Except the client should have probably checked if the call had been accepted first. That's why I say it's a state machine bug: The "Send audio" function should have never been activated in state "waiting to accept".

auiya 8y ago
> If you consider what happens when you add a third person to a call: You start sending them everyone's audio! Not before they accept the call connection request you don't.
- mcpherrinm 8y ago
  Well clearly. That's the bug.

yalok 8y ago

My guess is that it’s an unfortunate combination of several problems: - audio and video capture has to start going before call is actually established at signaling level, in order to minimize call establishment delay. Audio maybe going through Bluetooth, for example, and waking up Handsfree mode of BT may take 1-2 sec - most of the group calling functionality was developed by a separate team, and group calling signaling may be loosely integrated at UI level, where, once UI triggers a switch to a group call - internally, the whole new library may kick in and get the current 1-1 call state transferred to it. - when this “transfer” happens, the state of the first 1-1 call gets affected (at either local or remote side (due to signaling), which leads to either remote side think that the call was answered (a lack of protection in the call signaling state machine to ensure it was users UI action) or local side thinks it’s ok that remote users answers the call (in this case FT must have streamed audio even during 1-1 call establishment phase) - lack of a check for your own phone number added to a call. This, due to having the same IDs/tokens twice in a group call, may lead to unexpected call signaling state machine switch - lack of manual testing with focus on edge cases (like the described flow to repro the bug may not be the main flow for how users start group calls on FT) I never worked at Apple, but I built VoIP stuff for the past 20 years.

nolok 8y ago
> audio and video capture has to start going before call is actually established at signaling level, in order to minimize call establishment delay. Audio maybe going through Bluetooth, for example, and waking up Handsfree mode of BT may take 1-2 sec As a user, while I can accept capture starting before I answer, I cannot accept sending. I understand how it helps the speed of establishing calls. But it means the only thing needed to spy on me from that is a software change ON THE OTHER SIDE. No way to know from my side if I'm good or not.
- akvadrako 8y ago
  I don't think that's what he meant. It can be sending empty audio, but the whole path needs to be up and running.
- enedil 8y ago

rahoulb 8y ago

Although others have mentioned state machines I wonder if it’s a deliberate feature with an unforeseen side effect. The Apple Watch walkie talkie feature allows you to talk with anyone who has approved you and has walkie talkie switched on. But it appears to use FaceTime under the hood as when it was launched it failed if FaceTime was switched off on your phone. So the “auto answer” mechanism is already in there, just getting triggered at the wrong time?

TomK32 8y ago

I'm pretty sure that person also wrote a few lines so the phone being called won't show there's a call... /tinfoil

StreamBright 8y ago

Lack of formal methods in the design phase. It would be trivial to express a rule that you cannot hear audio from the called person before they accept it with it.

nvarsj 8y ago

You’re curious how a complex piece of software can have a bug? Come on now. You’re either naive or being hyperbolic.

blablabla123 8y ago

> I'm not suggesting malice but I do wonder at the lack of defensive programming. I've never worked at Apple but neither have I in 10 years seen people truly appreciate any attempts I did in defensive programming. On the contrary, usually I hear - sometimes loud - complaints about that. Usually from fellow engineers when they see it but on occasions also from non technical people. It's a joke that everybody is so sad about bad application security but at the same time virtually nobody cares about it at all when involved in an actual development project. Classical case of NIMBY. FWIW Apple's security is far over average compared to other companies. But I guess they cannot isolate themselves 100%.

C14L 8y ago

If the implementation is complicated enough, the code that opens the mic and the code that accepts the call are so far away from each other that its impossible to see. That's why keeping things simple is the greatest art in programming.

Haga 8y ago

When in doubt collect all the data.

JustSomeNobody 8y ago

I agree this should have been caught in QA/dev testing. However, I would bet a dollar this wasn't "Some Engineer" deciding to do it this way. This design was the culmination of some back and forth discussions and it is this way for a sound (no pun) reason.

skywhopper 8y ago

The code involved is a lot more complicated, involving a lot more pieces written by different teams working in concert with each other. There's not a single routine that takes the steps you describe that got a few lines out of order or something.

scraft 8y ago

Possibly a product owner trying out the latest build, receiving a call, accepting it, and then waiting for the call initiator to receive the message that the call has been accepted, and then start sending data and asking: "Why doesn't it take X seconds before I can start talking". To which the engineers possibly explained the reasons and the product owner saying: "But I want it instant, let's bypass all this extra stuff and get a proof of concept instant answer working" To which the engineer said: "But we'd technically be sending data before the call has even been accepted" To which the product owner said: "That's okay, the user can't actually see that data, let us just get this in for now, we can worry about the security/privacy side later". To which the engineer said "but, but, but" saw the product owners eyes glaze over and just made the commit: Commit 1279: Remove very important security/privacy feature of ensuring no data is transmitted until the call has been accepted. This is again my best judgement, do not come to me when this blows up, please speak to the product owner. Then went to the pub in despair.

whamlastxmas 8y ago
I think it's much more likely their code is a mess and it's not super obvious when stuff starts and ends and this is just an unintentional mistake.
- pbhjpbhj 8y ago
  You hear this excuse all the time, don't FAANG employ the world's very best developers? Maybe their code is a mess for orthogonal reasons - management, profit-motive? Aside: I thought I'd heard devs have automated analysers that step through and find all possible code paths, allowing complex code to be audited for security issues and such? Presumably that's how these sorts of bugs should be found in testing.
  - MagicPropmaker 8y ago
    Judging from their quality issues, I'm not so sure they're attracting the "world's very best developers" any more.

jeromebaek 8y ago

An old product like FaceTime comes with tons of legacy code. Mistakes like this while fiddling with legacy code is not unreasonable.

JangoSteve 8y ago

I'm wondering if it's something really simple, like comparing the unique number of callers versus the total number who have accepted the call. Imagine a method for group calling that returns whether or not the call should be considered active or not by seeing if the number of callers is equal to the number who have accepted the call. But now imagine the method that tallies up the number of callers does a unique count by phone number, while the method that tallies how many people have accepted the call does not. Since adding your own number is adding a caller who has already accepted a call, you end up with unique_callers (2, you and them) == total_accepted (2, you and you). This could be tested by adding a third person to a group call twice (if iOS will let you do that) instead of adding yourself to the call. Furthermore, if the interface on the recipient's phone only looks at whether or not they've accepted the call, that would explain why the call doesn't auto-accept on their end and go into the call before they've accepted or rejected the call.

update 8y ago

> I mean QA & Testing should catch it, sure. I believe they have been working on FaceTime Group for 5+ years now. In 2015, Modern Family had an episode where the entire plot focused around the family using FaceTime on their iPhones --- but they had the Group feature[1]! I remember being blown away when I saw that, figured it'd be coming in the next iOS update. It didn't. 4 years later, they finally released it..then recalled it immediately [1] https://www.mlive.com/entertainment/index.ssf/2015/02/tech_review_modern_family_epis.html https://www.mlive.com/entertainment/index.ssf/2015/02/tech_r... edit: managed to find a short clip from said Modern Family episode: https://www.youtube.com/watch?v=vy3jUOBxQuI https://www.youtube.com/watch?v=vy3jUOBxQuI Though in the full episode you see a lot more of the Group feature

m3kw9 8y ago

Is rarely that they didn’t test it, it was likely caused by a new scenario that they have not thought of that brings out this bug.

foobaw 8y ago

Agreed - looks like something very simple to add as a test-case. If not manual, an automated QA process could've easily caught this if it was even remotely robust enough. The steps to reproduce is trivial so I'm shocked.

iforgotpassword 8y ago

Our company's phone system, or our provider, had a problem with outgoing calls once: you couldn't call users of a certain mobile provider, but only if they were currently using GSM. Anything newer would make it work. The symptom was that even when the cell user picked up, you'd still here the sound like their phone is still ringing. That alone was already mindboggling to me, but the crazy part was that with certain phone models (at our company), the cell user would be able to hear you while you still thought their phone was ringing.