4 ms·

> “These recordings are accompanied by user data showing location, contact details, and app data.” This should be in the title. Media was focusing on contract

> “These recordings are accompanied by user data showing location, contact details, and app data.” This should be in the title. Media was focusing on contractors listening to Google Assistant snippets not long ago — and those were anonymized. Private and identifying information was in the snippets, not metadata (the article mentioned this).

zaroth 7y ago
I think the story is possibly confusing this bit. When recorded, and transmitted to Apple, the location, Apple ID, etc. of the recording device is known. The same (similar) would be true for any voice assistant. When the contractors are reviewing snippets for training purposes, this data is anonymized. In the case of Apple; > Apple has said that it takes steps to protect users from being connected with the recordings sent to contractors. The audio is not linked to an Apple ID and less than 1% of daily Siri activations are reviewed."
- romwell 7y ago
  I wish the article was more clear on this. It takes a lot of work to properly anonymize the data. The question is: what information does the operator see?
  - panpanna 7y ago
    As I understand it, the data was not anonymized. They simply asked the contractors to sign a non-disclosure contract.
- Buge 7y ago
  It's a direct quote from the whistleblower: > The whistleblower said: “There have been countless instances of recordings featuring private discussions between doctors and patients, business deals, seemingly criminal dealings, sexual encounters and so on. These recordings are accompanied by user data showing location, contact details, and app data.” > That accompanying information may be used to verify whether a request was successfully dealt with. In its privacy documents, Apple says the Siri data “is not linked to other data that Apple may have from your use of other Apple services”. There is no specific name or identifier attached to a record and no individual recording can be easily linked to other recordings. > Apple told the Guardian: “A small portion of Siri requests are analysed to improve Siri and dictation. User requests are not associated with the user’s Apple ID. Siri responses are analysed in secure facilities and all reviewers are under the obligation to adhere to Apple’s strict confidentiality requirements.” Strangely, the whistleblower's quote directly contradicts Apple's privacy documents. The whistleblower says the recordings are linked to some other information, but the privacy documents say the recordings are not linked to other information. But the whistleblower's quote does not contradict Apple's quote to the Guardian. The whistleblower never says the recordings are linked to the user's Apple ID. https://www.theguardian.com/technology/2019/jul/26/apple-contractors-regularly-hear-confidential-details-on-siri-recordings https://www.theguardian.com/technology/2019/jul/26/apple-con...
  - deleted 7y ago
    [deleted]
- vatueil 7y ago
  > When recorded, and transmitted to Apple, the location, Apple ID, etc. of the recording device is known. Actually, according to Apple's statements Siri does not link recordings to your Apple ID. That's a bit of a double-edged sword, though, since it means users cannot easily check what Apple has recorded either. Judging from public reactions, it seems many users think Apple doesn't keep Siri records at all simply because they can't access the audio history. Reportedly, the only way to avoid letting Apple manually review audio was to not use Siri at all, unlike other voice assistants. If so, I wonder if that was also due to the lack of Apple ID, since if audio isn't linked to a user then it follows that it isn't linked to user preferences either. I suppose you could still do something like send data from users that allow QA testing to one system and from those that opt-out to another, but apparently that wasn't done.
- occamrazor 7y ago
  1% is an unexpectedly high number.
  - panpanna 7y ago
    It's a huge number. I am surprised they have the manpower.
  - eitland 7y ago
    In fairness gp says less than 1%.
    - benj111 7y ago
      I would expect 'less than 1%' to be 0.1% to 0.99%. 0.1% is still a high figure.
      - schuke 7y ago
        I would bet it’s closer to 0.99% otherwise wouldn’t they have phrased it something like “slightly more than 0.1%” to make it sound smaller?
        Symmetry 7y ago
        If I was talking to a reporter and the real number was .01% I would still say "less than 1%" to make sure I was properly understood. The whole reason we use percentages and say "one percent" instead of "point zero one" is that some people tend to get confused with decimals but not with integers. EDIT: And part of this is just how we say things. "Twenty" is just easier to parse in a conversation than "Two Oh" even though the later is perfectly comprehensible when written out.
        jklein11 7y ago
        1% vs .01% would be overstating the truth by 100x. If you were a spokesperson for apple you would want to make that number seem as small as possible.
ocdtrekkie 7y ago
Note that while the Google recording data was arguably "anonymized", the reviewers of the leaked recordings were able to in many cases find enough personal information to contact the people recorded. Which means they were not, in fact, anonymized. "We let ordinary Flemish people hear some of their own recordings. ‘This is undeniably my own voice’, says one man, clearly surprised." "A couple from Waasmunster immediately recognise the voice of their son and their grandchild." https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are-eavesdropping-even-in-flemish-living-rooms/ https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are...
- luckylion 7y ago
  It's largely impossible to anonymize data you don't understand, isn't it? They have somebody transcribe the recording because they don't know what it said, so they can't know whether PII is in there. I suppose they might try to split it into multiple parts and have different people work on each part, but that may lead to problems, as you need the full sentence as context in some cases.
  - dash2 7y ago
    This. There's a nice paradox. Until the AI is good enough to understand the voice data, it can't know that it isn't privacy-violating -- up to and including extremes like somebody telling their most intimate secrets. Exposing this data to any other human, whether a subcontractor or internal employee, is a potential privacy violation, full stop.
    - laumars 7y ago
      I would rather the voice processing happened locally on your device. We must be getting to the stage where that's becoming viable? If not entirely in software then via a custom chip. Even if that transcript then still needs to be sent to the cloud for processing (eg for translating human sentences into logical actions might still be out of the grasp of home devices), developers can at least then send the data tokenised (which wouldn't be possible if it was a compressed audio stream) and then get a server response referencing those tokenised place holders. A simplistic example of my point might be Message: I need to book a doctors appointment for erectile dysfunction. Sent: I need to book a $ARG[0] appointment for $ARG[1]. Received: { "action": "ring"; "subject": "$ARG[0]"; "message": "appointment for $ARG[1]" } I appreciate something like that wouldn't be infallible but it would still go a long way to redacting sensitive or identifiable information and thus a huge step forward for privacy.
paulcarroty 7y ago
> and those were anonymized Hard to believe for Google context. They always trying to get the max possible data.
- kelnos 7y ago
  When they give the recordings to reviewers, they anonymize it, but they still have all the data associated with it.
- orbital-decay 7y ago
  It's a very relative concept. You can always omit some part of the record pretending you're "anonymizing" it, while the rest is easily correlated with a particular person.

zaroth 7y ago

I think the story is possibly confusing this bit. When recorded, and transmitted to Apple, the location, Apple ID, etc. of the recording device is known. The same (similar) would be true for any voice assistant. When the contractors are reviewing snippets for training purposes, this data is anonymized. In the case of Apple; > Apple has said that it takes steps to protect users from being connected with the recordings sent to contractors. The audio is not linked to an Apple ID and less than 1% of daily Siri activations are reviewed."

romwell 7y ago
I wish the article was more clear on this. It takes a lot of work to properly anonymize the data. The question is: what information does the operator see?
- panpanna 7y ago
  As I understand it, the data was not anonymized. They simply asked the contractors to sign a non-disclosure contract.
Buge 7y ago
It's a direct quote from the whistleblower: > The whistleblower said: “There have been countless instances of recordings featuring private discussions between doctors and patients, business deals, seemingly criminal dealings, sexual encounters and so on. These recordings are accompanied by user data showing location, contact details, and app data.” > That accompanying information may be used to verify whether a request was successfully dealt with. In its privacy documents, Apple says the Siri data “is not linked to other data that Apple may have from your use of other Apple services”. There is no specific name or identifier attached to a record and no individual recording can be easily linked to other recordings. > Apple told the Guardian: “A small portion of Siri requests are analysed to improve Siri and dictation. User requests are not associated with the user’s Apple ID. Siri responses are analysed in secure facilities and all reviewers are under the obligation to adhere to Apple’s strict confidentiality requirements.” Strangely, the whistleblower's quote directly contradicts Apple's privacy documents. The whistleblower says the recordings are linked to some other information, but the privacy documents say the recordings are not linked to other information. But the whistleblower's quote does not contradict Apple's quote to the Guardian. The whistleblower never says the recordings are linked to the user's Apple ID. https://www.theguardian.com/technology/2019/jul/26/apple-contractors-regularly-hear-confidential-details-on-siri-recordings https://www.theguardian.com/technology/2019/jul/26/apple-con...
- deleted 7y ago
  [deleted]
vatueil 7y ago
> When recorded, and transmitted to Apple, the location, Apple ID, etc. of the recording device is known. Actually, according to Apple's statements Siri does not link recordings to your Apple ID. That's a bit of a double-edged sword, though, since it means users cannot easily check what Apple has recorded either. Judging from public reactions, it seems many users think Apple doesn't keep Siri records at all simply because they can't access the audio history. Reportedly, the only way to avoid letting Apple manually review audio was to not use Siri at all, unlike other voice assistants. If so, I wonder if that was also due to the lack of Apple ID, since if audio isn't linked to a user then it follows that it isn't linked to user preferences either. I suppose you could still do something like send data from users that allow QA testing to one system and from those that opt-out to another, but apparently that wasn't done.
occamrazor 7y ago
1% is an unexpectedly high number.
- panpanna 7y ago
  It's a huge number. I am surprised they have the manpower.
- eitland 7y ago
  In fairness gp says less than 1%.
  - benj111 7y ago
    I would expect 'less than 1%' to be 0.1% to 0.99%. 0.1% is still a high figure.
    - schuke 7y ago
      I would bet it’s closer to 0.99% otherwise wouldn’t they have phrased it something like “slightly more than 0.1%” to make it sound smaller?
      - Symmetry 7y ago
        If I was talking to a reporter and the real number was .01% I would still say "less than 1%" to make sure I was properly understood. The whole reason we use percentages and say "one percent" instead of "point zero one" is that some people tend to get confused with decimals but not with integers. EDIT: And part of this is just how we say things. "Twenty" is just easier to parse in a conversation than "Two Oh" even though the later is perfectly comprehensible when written out.
        jklein11 7y ago
        1% vs .01% would be overstating the truth by 100x. If you were a spokesperson for apple you would want to make that number seem as small as possible.

ocdtrekkie 7y ago

Note that while the Google recording data was arguably "anonymized", the reviewers of the leaked recordings were able to in many cases find enough personal information to contact the people recorded. Which means they were not, in fact, anonymized. "We let ordinary Flemish people hear some of their own recordings. ‘This is undeniably my own voice’, says one man, clearly surprised." "A couple from Waasmunster immediately recognise the voice of their son and their grandchild." https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are-eavesdropping-even-in-flemish-living-rooms/ https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are...

luckylion 7y ago
It's largely impossible to anonymize data you don't understand, isn't it? They have somebody transcribe the recording because they don't know what it said, so they can't know whether PII is in there. I suppose they might try to split it into multiple parts and have different people work on each part, but that may lead to problems, as you need the full sentence as context in some cases.
- dash2 7y ago
  This. There's a nice paradox. Until the AI is good enough to understand the voice data, it can't know that it isn't privacy-violating -- up to and including extremes like somebody telling their most intimate secrets. Exposing this data to any other human, whether a subcontractor or internal employee, is a potential privacy violation, full stop.
  - laumars 7y ago
    I would rather the voice processing happened locally on your device. We must be getting to the stage where that's becoming viable? If not entirely in software then via a custom chip. Even if that transcript then still needs to be sent to the cloud for processing (eg for translating human sentences into logical actions might still be out of the grasp of home devices), developers can at least then send the data tokenised (which wouldn't be possible if it was a compressed audio stream) and then get a server response referencing those tokenised place holders. A simplistic example of my point might be Message: I need to book a doctors appointment for erectile dysfunction. Sent: I need to book a $ARG[0] appointment for $ARG[1]. Received: { "action": "ring"; "subject": "$ARG[0]"; "message": "appointment for $ARG[1]" } I appreciate something like that wouldn't be infallible but it would still go a long way to redacting sensitive or identifiable information and thus a huge step forward for privacy.

paulcarroty 7y ago

> and those were anonymized Hard to believe for Google context. They always trying to get the max possible data.

kelnos 7y ago
When they give the recordings to reviewers, they anonymize it, but they still have all the data associated with it.
orbital-decay 7y ago
It's a very relative concept. You can always omit some part of the record pretending you're "anonymizing" it, while the rest is easily correlated with a particular person.