25 ms·

Google employees are listening to Google Home conversations

872 points by bartkappenburg 7y ago

duxup 7y ago
This falls into the category of: I bugged my house... NOW MY HOUSE IS BUGGED! Not to dismiss the value of the news here, it is important for folks to know, but the overall situation is both concerning, and amusing.
- seszett 7y ago
  Most people assume that the system is entirely automated, and that their commands will be processed by mere computers, not listened to by actual people who might even be their own neighbours.
  - duxup 7y ago
    I fear that most people ... don't care, or don't assume the system is automated. And they do it anyway.
  - noir_lord 7y ago
    Most people unfortunately are specialists in at most one field. Outside of that specialisation they are distinctly average. We because of who we are entirely expect this stuff but I'm sure I miss all kinds of things that are outside my specialization. Modern society has become so complex no one person can keep track off all the moving parts to know if they are been taken advantage off for example.
- GVIrish 7y ago
  I mean, I would assume that humans are used to improve the speech recognition capabilities of voice assistants. Accents alone are a big challenge to voice recognition, not to mention regional dialects, and the more complex languages. People listening to what is said to a voice assistant after the prompt doesn't really alarm me. What would be alarming is if the voice assistant was secretly recording normal conversations.
  - ggggtez 7y ago
    What is usually shown, time and time again, is that the voice assistants aren't secretly recording. They are triggered because they heard something that sounded like a hotword. Maybe you were talking about what dog you want you buy. Someone asks if a Greyhound would be alright and you said "No way, Poodle!". Well, the device picked it up as "Ok Google" and begins listening for a command. It makes a bleep, and lights up and you start talking about dogs or whatever. You maybe didn't notice, so it picks up 15 seconds of a conversation, has no idea what to do, and probably just does an image search for dog pictures or even just the "I don't know how to help with that". Not to say you shouldn't be annoyed that it picks up unrelated stuff... I just always think "secretly recording" is such a misnomer.
- espeed 7y ago
  Your phone has more sensors on it than these devices and most people have their phones on them 24/7. And most phones have the same voice-controlled features too. Why people are concerned about narrow scoped voice-devices more than their phone is a curious thing. Is it because it talks back to you that makes it more apparent?
  - duxup 7y ago
    It is just more recent. Plenty of articles about phones listening too.
  - clebio 7y ago
    No one said we were (_more_ concerned).
- Ensorceled 7y ago
  I'm not sure why it's unreasonable for customers to have assumed this is like every other product: they developed it with employees and voluteers before shipping. After all, the voice recognition in my Prius doesn't send recordings of my "call mom and dad mobile" back to Toyota and it certainly doesn't send random snippets of conversations my wife and I have to low paid contractors around the world.
  - anchpop 7y ago
    > After all, the voice recognition in my Prius doesn't send recordings of my "call mom and dad mobile" back to Toyota and it certainly doesn't send random snippets of conversations my wife and I have to low paid contractors around the world. How do you know? Is it not internet-connected?
    - javagram 7y ago
      I don’t know about the Prius, but there are plenty of non-connected devices that support voice control. I have a Garmin GPS from like 8 years ago that did voice commands, it had zero transmit functionality.
    - deepgrave4 7y ago
      It amazes me that people make statements like this without exercising any critical thought on whether it's true or verifiable. OP's car probably tells Toyota how much OP weighs, instantaneous speed, how hard OP pushes the brakes, favorite radio stations, turn signal usage, headlight usage, every on-screen computer action, seatbelt usage, gas level, current location, and identity of all other Toyotas in the vicinity (and more: use your imagination). The car has (presumably) cameras and (clearly) microphone(s) and bluetooth/wifi. But OP has it on faith that Toyota respects OP's privacy, because...
TheAdamist 7y ago
"The man, who wants to remain anonymous, works for an international company hired by Google. " So not a Google employee at all, a probably low paid contractor who is in possession of thousands of audio files. Your privacy matters, except when the bottom line is involved.
- astrea 7y ago
  Sounds like he was a Turker: "For each fragment that he listens, he will receive a few cents."
  - burkaman 7y ago
    Probably a company like https://scale.ai/ https://scale.ai/, although I don't think they do audio.
- deleted 7y ago
  [deleted]
- numbsafari 7y ago
  What is doubly concerning here is that the contractor was in a position to demonstrate how the system worked to the reporters. That would seem to indicate they have access to that data in a non-secured environment. I'm not familiar with EU law around these things, but I would imagine there is some kind of whistleblower mechanism available, and a right for authorities to audit/inspect such activities?
  - C1sc0cat 7y ago
    I would expect that a telecoms employee who was doing similar work on quality etc to be quite securely vetted. If I was doing now what I did back in the day For BT / Dialcom (I had root on the UK's main ADMD) I would probably have to pass DV vetting (TS in USA terms)
    - dmix 7y ago
      What about all the telecom APIs like Twilio? They have raw access to millions of phone calls every day. I doubt they have ‘secure rooms’ for debugging.
chance_state 7y ago
"What Orwell failed to predict is that we'd buy the cameras ourselves, and that our biggest fear would be that nobody was watching."
- duxup 7y ago
  I think Frank Herbert hit it a bit more on the nose: “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” Frank captures the motivation, the fact that it is desirable is the scary part. Akin to the genetic engineering scene from Gattaca.
  - noir_lord 7y ago
    Aldous Huxley got there earlier with A Brave New World as well. Speculative Fiction was ahead of the curve in that era.
    - jammygit 7y ago
      Anyone want to recommend a book? Let me join in by suggesting the Asimov robot books, which I’ve just begun
      - noir_lord 7y ago
        Robot series is good, Foundation series by him is brilliant. Also check out The Culture novels by Iain M Banks (he was the closest recent author to be a great the equal of Wells, Huxley or Orwell (imo) just a phenomenal writer). The Polity series by Neal Asher are well imagined and he world builds brilliantly.
      - Medicalidiot 7y ago
        Brave New World by far is the most accurate book to date of what the modern world looks like. That's my strongest recommendation. Fahrenheit 451 is also a solid read.
spork12 7y ago
Stop putting these things in your house folks!
JorgeGT 7y ago
The biggest issue IMHO is how the average consumer has been deceived into the belief that current AI is pure AI, when in reality a lot of humans are looking at your pictures, listening to your recordings, crawling through your inbox and analyzing your browsing/purchasing/streaming history, right now: https://imgs.xkcd.com/comics/trained_a_neural_net.png https://imgs.xkcd.com/comics/trained_a_neural_net.png
- itronitron 7y ago
  Engineering Tip #2 -> if you pay someone else to do it, you can technically say you did it in the cloud
mbrumlow 7y ago
I think this is the last straw for me. I have already tinkered with the idea and built enough for POC. But I am going to have to replace all my google home and amazon stuff with home built systems that I control both the hardware (to a extent) and the software.
r00fus 7y ago
My biggest frustration is going to a friends house which is blanketed with Echo/Google coverage and realizing that all my words could be used against me at a later time. How does one approach this situation?
- tzakrajs 7y ago
  Tell the truth or remain silent. Try not to incriminate yourself.
  - rurp 7y ago
    Telling the truth about your religion or sexual orientation can get you beaten or killed in many parts of the world. That's not likely to happen in the present day US, but there are plenty of less dramatic but definitely undesirable things that can happen when your home audio gets passed around to a bunch of random contractors.
    - tzakrajs 7y ago
      Sorry, I was being sarcastic but it really doesn't read like it in retrospect. Totally agree with everything you wrote.
- DanHulton 7y ago
  I mean, it's roughly the same level of exposure you run into literally every minute you're out in public anyway - all those phones around you have microphones, and more and more, they're always listening in exactly the same fashion. This is how always-on "Hey Siri" or "Okay Google" works. Which isn't to say it isn't frustrating, just that your frustration at your friend is misplaced - they're not exposing you to anything which you aren't already exposed. If your fear is that these tech companies are going to misuse casually-overheard conversations from these devices, they're just as likely to use it from all the phones you're near. If your fear is that your friend is going to use this tech, however, maybe you need better friends. There's nothing stopping an unscrupulous friend from putting plain-old audio recorders through their house anyway.
_pmf_ 7y ago
Intersect certain church bell sounds with Google assistant buyers in a small village and you have the individual. EDIT: thinking about it, a database of temporal-geographical background sounds would be nice to have
electrograv 7y ago
So Google’s response is (paraphrased as fairly as I can while removing the sugar-coating): ’Yes, we hire people to listen in to and transcribe some conversations from the private homes of our customers (so as improve our speech recognition engines); but the recordings aren’t linked to personally identifiable information.’ Even assuming they have only the purest intentions here, I still don’t understand how they can possibly guarantee that these recorded conversations are not linked to personally identifiable information! For example, what’s to stop me from saying “Hey Google, I am <full legal name / ID> and my most embarrassing and private secret is <...>”? One might argue that they could detect this in the recognized text and omit those samples, but presumably the whole purpose of hiring people to create transcripts is because the existing speech-to-text engine isn’t perfect, and they need more training data.
- MassiveOwl 7y ago
  This is a good point, one of the reasons to use it is for reminders "Hey Google, remind me to visit x at place y"
- freeAgent 7y ago
  I would count a recording of my voice as "personally identifiable information" right off the bat. Voice printing is a thing, and anyone will also tell you that they recognize the voices of people they interact with regularly. If someone played an audio clip of someone I know talking to Google Assistant to me, I would recognize who it was based on their voice.
  - Bartweiss 7y ago
    This sent me down the rabbithole of learning how identifiable voiceprints are. As you might guess, the answer is "very", although to my surprise our voices change enough that recordings lose a great deal of fidelity over a few years. Authentication on fixed phrases is reasonably accurate within a very few words, so at minimum it should be possible to associate "Hey Google" clips with regular users of Google Assistant voice control (i.e. "OK Google"). Identifying whether someone is present in a large dataset on open phrases is much harder, but a ~30s clip could do the job fairly consistently for anyone with access to a significant amount of voice data. And if this employee (who isn't directly working for Google) shared 'thousands' of clips with a news org, the cautious bet is that some other employee might share them with anyone willing to pay for the records.
inerte 7y ago
Home, Siri, Alexa, M, they all do. I have friends that work on this field transcribing the audio, and measuring its accuracy. Sometimes it's multiple layers of contractors: An employee hands the task to a contractor, another contractor verifies the speech to text, and they're all managed by a contractor. Search for languages like Portuguese, Swedish, Chinese, etc on LinkedIn and you'll find the jobs posts https://www.linkedin.com/jobs/search/?keywords=portuguese&location=San%20Francisco%20Bay%20Area https://www.linkedin.com/jobs/search/?keywords=portuguese&lo...
- rwc 7y ago
  "They all do" ... my understanding is that this expressly does not happen with HomePod conversations.
  - inerte 7y ago
    So I meant "they" in the sense of the companies, not necessarily the home devices. Sorry about the confusion. I know 100% that it happens with Siri. If Apple is excluding HomePod conversations from Siri's dataset, that I don't know.
    - danieldk 7y ago
      Do you have a source for human annotation of Siri recordings? Do they use subcontractors like Google?
      - inerte 7y ago
        First hand source, it was the first job several of my Brazilian friends (or their spouses) got when they relocated to the Bay Area. They use companies like Moravia or Welocalize. Take a look at some of the job posts from my link above.
  - saagarjha 7y ago
    As far as I am aware, Siri's audio retention policy is up to two years.
rchaud 7y ago
It's no coincidence that companies like Amazon market their Echos as "stocking stuffers" for the holiday season. I've wondered how Google Home and these "smart home" devices were always able to be priced as low as they are. Goes to show that paying for the product doesn't exempt you from still being part of the product.
thrwer3434312 7y ago
Duh! Of course they are listening; how else would they train such agent ? It's not like you can come up with a random NN that is suddenly sentient.
- saagarjha 7y ago
  They could use data that doesn't come from their customers, many of whom are not aware that they're training Google's models…
gtirloni 7y ago
From a computer science perspective, what should Google do to train its models in a privacy conscious way?
- thrwer3434312 7y ago
  They can't. They need to transcribe the data.
  - RosanaAnaDana 7y ago
    ^This. I'm sure they're optimizing to only transcribe data they are pretty confident they got wrong (multiple requests; use has to go in and correct), but if you want software to do what the Google Home software does, you have to do this.
    - saagarjha 7y ago
      There are a lot of things that Google could have done better in this situation without meaningfully impacting their software's quality.
      - RosanaAnaDana 7y ago
        I guess. I'm still not of the opinion anything being done here is really 'wrong' (unless the annotators are mis-using the data).
  - natch 7y ago
    Not true. Apple has demonstrated how to do it and has published a paper on the topic. A. Bhowmick et al. Protection Against Reconstruction and Its Application in Private Federated Learning, 2018. Don’t just dismissively say “oh that’s different and doesn’t apply” until you read it and understand the range of problems Apple is tackling with this and similar approaches. A lot can be done when the organization makes privacy a priority. Let’s not forget I am replying to a person who has chosen to use a throwaway account for this topic. Why did they do that? Hmmm.
SmirkingRevenge 7y ago
The one thing about these stories that keep coming out about the home assistants... they kind of create the impression that this is an issue specific to home speakers, and you can avoid it, by simply not buying them. That's misleading. Any voice command you use to operate any internet connected tech gadget, from phones to smart TV's, is potentially stored and flagged for human review. You really have to avoid using voice commands at all, on all of your devices. Even that is probably insufficient. You probably have to go even further and actively disable voice command features on all of your devices, assuming they actually support such a setting. Otherwise here's still the possibility of an accidental recording taking a journey through the clouds, to a stranger's ears.
- arien 7y ago
  And not to be anywhere near anyone else's listening devices. Isn't there a law in some US states that there needs to be consent before recording someone? How does this fit in and who would be held responsible, the owner of the listening device or the company behind it?
  - bastawhiz 7y ago
    You already consented by agreeing to the terms of service. If someone else is talking to their smart device while you're talking, it's ostensibly their responsibility. There's no reasonable laws that prevent you from being overheard in the back of a recorded phone conversation
    - arien 7y ago
      Not in the case of the device being someone else's. I don't have any Google or Amazon smart listeners, so I never accepted them. Yet if my voice is in any of these recordings, well...
      - kahnjw 7y ago
        How do you square that with a security cameras? Do you need to consent every time you enter a space secured by them?
amacneil 7y ago
Serious question: how do people think the ML models for Home, Alexa, Siri, etc are trained, if not with human labeling?
- arien 7y ago
  Doesn't mean you can use live data for tests. Should be opt-out by default, with the chance to opt-in if you wish to collaborate with your data.
- natch 7y ago
  Apple pays real humans to enlist in research studies to collect data. That’s how they built FaceID for example. They have stated this publicly.
- pessimizer 7y ago
  People have never heard of ML. They think that there's a computer that listens to what they say, then does it. If they are interested in technical stories on tv news, they may think that what they say is sent to a computer at Amazon/Apple/Google/Wherever and the command is sent back; if they are not, they think that their phone is the computer. I mean, we've told them that we're on the verge of AI (and to fear it), and we can tell a two year old to do something and they do it without having to hire somebody to help them understand us.
- melkiaur 7y ago
  Serious question: how do you think Home, Alexa, Siri, etc were trained before coming to the market ? Another serious question: couldn't you ask ME to fix the data every so often (like the "improve this traduction" feature of google translate) ?
- la_barba 7y ago
  The ethical way to do this is to pay a representative sample of your customers to come in and record their voice.
RosanaAnaDana 7y ago
I mean. Of course they are. Do you expect to be able to do any meaningful level of training on data that hasn't been properly labeled? At some point, a human has to go in and correct the software when the software gets it wrong. If you want services that do what Google Home does, you have to have this. Even with that, I'm sure that the engineers are flagging voice requests that happen more then once, or where some one has to manually change or correct what the software thought was the request. This is only creepy if you don't understand how the software works.
- inetknght 7y ago
  > I'm sure that the engineers are flagging voice requests that happen more then once As a user, I expect to see feedback declaring that it's having trouble and specifically requesting permission to send my unparseable requests to some review queue. I expect to see feedback indicating what could resolve the issue: was it something I said? was there too much noise in the background? was there a software fault and a fix will be deployed at such-an-such date? The fact that I do not get any feedback whatsoever when things go wrong leads me to have no faith that the problem will be resolved in any satisfactory manner. When combined with a complete lack of consumer/business interaction options in general (w.r.t. Google), it leads to some very dissatisfied consumers. You think engineers are flagging voice requests that happen more than once? How would engineers have access to that data in the first place if it's supposed to be anonymous and private?
  - RosanaAnaDana 7y ago
    >The fact that I do not get any feedback whatsoever when things go wrong leads me to have no faith that the problem will be resolved in any satisfactory manner. When combined with a complete lack of consumer/business interaction options in general (w.r.t. Google), it leads to some very dissatisfied consumers. I guess you can ask for your money back? Lets stay grounded in reality here, Google is paying money on your behalf to improve your experience. They could, and from a financial perspective maybe should try getting you to annotate your own data. But all that annotated data gets pooled and used to improve the models. I don't think their goal here is to give you faith they are resolving your issue. The algorithm is going to get some things wrong. Its a matter of improving the overall accuracy and precision in the algorithm. >You think engineers are flagging voice requests that happen more than once? How would engineers have access to that data in the first place if it's supposed to be anonymous and private? 100% they have to be. Its too costly and time intensive to slog through all the data. Probably* they have a flag that goes off on a voice request when something is either not intelligibly interpreted so many times in a row. Or if a use has to go manually do something after repeatedly making a request. This flags the interaction for manual review. Then it processes through some algorithm to strip it of identifying data. Then it gets put in front of a warm body for review. Trust me that the company wants to avoid putting things in front of warm bodies as much as possible. Its expensive.
groovybits 7y ago
Assuming $0.3/audio clip and base wage of $10/hr, that equates to 33.3 audio clips/hr = 266.4 audio clips/day that are being monitored by any one 'language expert'. However, Google does not specify how long a 'conversation' is. How many sentences make up a conversation? When is the cutoff point? Google also says '1 in 500' conversations are monitored. That means for any one 'language expert', there are approx. 133,200 conversations/day that have a chance of being monitored. So basically, you have a 0.2% chance that your conversation is being picked up by any particular 'language expert' per day.
lexandstuff 7y ago
One thing that Google Home does noticeably poorly with is disabled voices and non-native speakers. Google Home would be really handy for a family member with down syndrome who can't read nor write but can easily learn voice commands, but Google Home doesn't currently understand him at all. My partner with a Malaysian Chinese accent also really struggles to use Google Home. That said, I notice the quality of voice recognition is rapidly improving and I'm thankful for the work the Google engineering team are doing; voice recognition and language translation has the potential to improve the lives of so many people.
Ensorceled 7y ago
The number of people in this thread who believe that this is ok because, 1) it's obviously the only way Google could train their voice system and thus 2) people clearly knew what they were getting into, is horrifying.
notTyler 7y ago
I very much believe the only reason smartphones aren't listening to us and recording us constantly is due to battery purposes. When you plug something in, yeah, I have no trouble believing they're recording everything and storing it somewhere.
deleted 7y ago
[deleted]
rickyc091 7y ago
Now it's making me wonder if the light on my Google Home Mini is on if that means it's passively listening to me. I always do a hard reboot since the indicator light suddenly stays active for a long period of time.
- guyzero 7y ago
  That means you have a notification: https://support.google.com/googlenest/answer/7073219?hl=en https://support.google.com/googlenest/answer/7073219?hl=en
gopher2 7y ago
Of course they are! There are user researchers, people looking for spam and abuse, people tagging data, among other things who are absolutely looking at your: private messages, search queries, profiles viewed, Alexa conversations, Tinder messages, bookmarks, call history, purchase history, geolocation history... and on and on. I guess there are a lot of news stories left to write on this topic.
gringoDan 7y ago
I think the responses to this can be broken down into a 2x2 matrix: level of concern vs. understanding of technology. 1) Don't understand ML; not concerned - "I have nothing to hide." 2) Don't understand ML; concerned - "I bought this device and now people are spying on me!" 3) Understand ML; not concerned - "Of course, Google needs to label its training data." 4) Understand ML; concerned - "How can we train models/collect data in an ethical way?" To me, category 3 is the most dangerous. Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies. And as others have pointed out, this extends beyond home speakers to any voice-enabled device in general. In conversations about this with engineers the response I've gotten is essentially: "Just trust that we [Google/Amazon/etc.] handle the data correctly." This is worrying.
- Spooky23 7y ago
  The meta-issue in the United States is that once your data is accessible to a third party, you have no sovereignty over it, and abuse by private actors is "agreed to" by click-wrap and access by government actors is a simple subpoena. The law needs to catch up. Sharing should require specific informed consent and legislation needs to establish a scope where data stored as a "tenant" on a third party server is given 4th amendment protection.
- ovi256 7y ago
  This classification is very useful to discuss this issue. The difference between 3 and 4, noble as it is, can be caused by feasability concerns that push people into 3, not just ignorance of the privacy impact. Human labelling of training data sets is a big thing in supervised learning. Methods that dispense with this would be valuable for purely economic reasons beyond privacy - the cost of human labelling of data samples. Yet we don't have them! Techniques like federated learning or differential privacy can train models on opaque (encrypted or unavailable) data. This is nice, but they assume too much: that the data is already validated and analyzed. In real life modelling problems, one starts with an exploratory data analysis, the first step being looking at data samples. Opaque encrypted datasets also stop ML engineers from doing error analysis (look at your errors to better target model/dataset improvements) which is an even bigger issue, IMO, as error analysis is crucial when iterating on a model. Even for an already productivized model, one has to do maintenance work like checking for concept drift, which I can't see how to do on an opaque dataset.
cuteMicrophone 7y ago
Hey Google, listen to me say “niggers” again and again.
Hernanpm 7y ago
how would you reproduce a bug otherwise
lovetocode 7y ago
I own 4 Home Minis, 1 Home and 2 Home hubs I honestly don't care so long as my data is used to improve the functionality and stability of my investment. It is quite another thing if they are selling my conversations to third-party vendors.
- groovybits 7y ago
  Per article: > The man, who wants to remain anonymous, works for an international company hired by Google. His job is to listen to the audio clips and to write out what he hears, so that Google can improve the speech assistant. So, > It is quite another thing if they are selling my conversations to third-party vendors. That is exactly what Google is doing.
  - bagacrap 7y ago
    "Selling" is when you trade goods for money. Google is paying money for a service. Pretty much the opposite of selling.
    - groovybits 7y ago
      The concern remains the same though: Making user data available to a non-Google entity. Do you care if Google makes money off of that trade? Really, they get value one way or another, because the analysis is incorporated back into the product.
  - thekyle 7y ago
    You really have to twist the definition of the word "selling" to say that Google is selling data to their contracted workers.
- mLuby 7y ago
  1. If the company has recordings, governments can hoover them up. And once one govt agency has them, sometimes other agencies get access. 2. Sounds like you're okay with Google using your private communication to enhance its products. So they can target you with ads based on your conversations with your spouse, children, or colleagues. 3. I just hope the algorithms understand context: Your spouse: son, put down the gun and come to dinner. [jokingly] Oh no, don't shoot! Argh you killed me, now eat your peas. You: we're going to kill it this quarter. They'll never know what hit them. TV in background: "kept their body chained in a basement. That and more at 11" Siri: O__o
jankeymeulen 7y ago
Link to the original source: https://www.vrt.be/vrtnws/nl/2019/07/10/google-luistert-mee/ https://www.vrt.be/vrtnws/nl/2019/07/10/google-luistert-mee/ Submitted link is citing this one...
cj 7y ago
I've long been curious how GDPR should be applied to Google Home, Alexa, etc. These devices record and upload voice recordings to the cloud (with a percentage apparently shared with Google contractors to process). The audio clips can be recordings of conversations of people who have not consented to being recorded. And I can't imagine it's possible for Amazon or Google to reasonably comply with Art. 15 or 17 (right of access by data subject, and right to erasure). Is there something I'm overlooking? Or is this just a known risk that Amazon/Google accept in providing the service?
- martin-adams 7y ago
  >> The audio clips can be recordings of conversations of people who have not consented to being recorded My guess would be that the person who does consent to the terms of service of the device and plugs it in will be the one responsible for ensuring that it is used in a way that does not invade the privacy of anyone else. It would be similar to me buying an IP camera, agreeing to the terms of service then putting it in my neighbours house without their knowledge. In the context of the GDPR, I think the question comes down to what rights a third party person has when a user puts their personal information on the company's servers. If someone uploads a video of me without my consent to YouTube, can I get it removed under the GDPR?
- noelsusman 7y ago
  https://myaccount.google.com/data-and-personalization https://myaccount.google.com/data-and-personalization Click on Voice & Audio Activity, then Manage Activity. You can listen to and delete every recording Google has of your voice. It's a little trippy to be honest. They've had this since long before the GDPR.
rev12 7y ago
I think a lot of people here are under the assumption that voice commands, on any device, have the potential to be human reviewed. I am not sure whether or not the general public has that same assumption. That being said, my biggest concern is the fact that many of these device don't have a hardware microphone kill switch. I feel better when I know I can control when a device is listening in. I've read reports that some Alexa devices have them, but I don't own any so I am unable to verify that. I want all of my devices with microphones to hardware based kill switch for the mic; that's my phone, laptop, tablet, everything.
- pessimizer 7y ago
  > I think a lot of people here are under the assumption that voice commands, on any device, have the potential to be human reviewed. I am not sure whether or not the general public has that same assumption. That's because we, as an industry, have fooled them into thinking that AI is real, and that people who don't know it's real are idiots. We don't think students deserve a proper tech education, so unless they are professionally techies, they have to learn from marketing materials (designed to convince them to buy things.)
jacobsenscott 7y ago
What's most interesting to me is that an army of contractors is required to get even the low accuracy levels of voice recognition we have today. The "AI" revolution is pure smoke and mirrors designed entirely to bilk investors out of their dollars. There's been no improvement in "AI" in the last 50 years, except that we have a lot more data to push through the same useless models.
- jononor 7y ago
  100 bucks for anyone who uses a 50 year model/method and get within twice the word-error-rate of today's speech recognition systems. Also, how do you explain the progress on ImageNet challenge over the last 10 years? Each method there uses the same dataset, yet error rates (top5) have gone from 30% in 2011 to around 3% in 2016.
  - jacobsenscott 7y ago
    Faster computers, and more of them.
    - jononor 7y ago
      Not going to deny that faster GPUs has helped a lot. But it is not the full picture either. DenseNet-121 has higher accuracy than VGG16, yet requires only 1/15 the storage space, and 1/5 the number of operations during inference. I'm pretty sure training time is faster also.
paganel 7y ago
I grew up as a kid in a country ruled by Securitate [1], one of the few institutions that rivaled the East-German Stasi when it came to spying on its own citizens, and as such I'm very, very perplexed of why would anyone bring in a listening device in his/her own house out of his/her own volition. And those people even pay for the privilege of having their home-lives actively monitored and listened to almost all the time, it's crazy. [1] https://en.wikipedia.org/wiki/Securitate https://en.wikipedia.org/wiki/Securitate
- drstewart 7y ago
  Not only that, but people still buy computers even though the Nazis used IBM computers to help them perpetuate the Holocaust. Surely you shouldn't use computers if the NAZIS used computers! [1] https://en.wikipedia.org/wiki/IBM_and_the_Holocaust https://en.wikipedia.org/wiki/IBM_and_the_Holocaust
- viklove 7y ago
  Do you have a smartphone? Why would you bring that listening device (the smartphone) into your own house out of your own volition? Please explain, because I am very perplexed.
  - paganel 7y ago
    I do have a smartphone, and I also do know that by default my phone is not actively listening on me (if it matters I do not use Siri-like services and I don't intend to). This is not the case with all these Alexa-like devices, but correct me if I am wrong.
  - jjkaczor 7y ago
    It's relatively straightforward on a smartphone to monitor the battery and bandwidth consumption that active listening and device<>server communications are occurring. Of course - then, the next apologist will always say: "the devices only transmit when hearing the 'Wake Word'"... Yeah, I don't think it takes an elite programmer to store/buffer audio, until the device hears the 'Wake Word', and then sends a "slightly" larger payload... (or batches several payloads) The next level of enthusiast evangelist will then tell us that it is impossible for these devices to buffer/store that much audio data, because they have limited memory... Yet, the same devices when used to play audio streams from the internet obviously have some sort of storage capacity to handle network glitches. Spoken audio (which is probably recorded at standard telephone/cellphone sampling rates - after all, why train on audio that is significantly different than the audio coming from mobile phone devices) does not take up a dramatic amount of storage space.
C1sc0cat 7y ago
Well of course you will have some qa / quality monitoring just as you will with telecoms and I seem to recall that the Dutch go in for monitoring of phones in a big way. Also does that article remind me of the Monty python sketch "news for wombats"
api 7y ago
There are two big things right now on my list of things I will never use: closed remotely operated home assistants with cameras or microphones that are not physically switched, and Facebook Libra.
white-power 7y ago
Niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers.
solarkraft 7y ago
What's next, Google collecting everyone's location data?
omnimkar69 7y ago
google employees liste to the recordings made with the smart google home speakers and via the google assistant app on smartphones. worldwide ex.belgium and netherlands people listen to those recordings to make the search engineer smarter. VRT NWS was able to listen to more 1,000 tracks thats why the employees must be listeing to the home conversations.
rosszurowski 7y ago
A bit tangential, but I tried sharing this link with a few friends on Facebook Messenger, and noticed it's blocked because it "violates Community Standards" [1]. Even shortened bit.ly links are blocked. Anyone know why that would be the case? I'm trying to not assume malice (eg. maybe it got misflagged?) but it certainly feels like censoring and is yet another push for me to drop Messenger too. [1]: https://i.imgur.com/9n1Hyqb.png https://i.imgur.com/9n1Hyqb.png
- css 7y ago
  Google Translate links have been blocked for a long time because translate.google.com can be used as a proxy.
  - rosszurowski 7y ago
    Ah, that makes sense. Thanks!
  - SquareWheel 7y ago
    And for the record, so can bit.ly links. URL shorteners or redirection services are often blocked from being posted (eg. also on reddit).
omnimkar69 7y ago
google employees listen to the recording made with the smart google home speakers and via the google assistant app on smartphones as well as worldwide in belgium and netherlands , people listen to those recordings to make the search engine smarter. VRT NWS was able to listen t more than 1000 tracks these are pieces usally spoken conciously.
lonelappde 7y ago
Catch-22/Irony? The main problem here is that Google hired someone like this person who leaked days they were trusted to protect. It's a concern for users (Insider Risk at Google), and something we need laws to protect. Surely if Google Corporation is a poor steward of this data, the specific malicious actor here who violated both his employment policy and his users' trust is especially bad.
lonelappde 7y ago
Some companies tell me on every phone call that I am being recorded. Some don't (but maybe are recording anyway?) Should Voice Assistants be required to annoy that they are recording every time they received an activation? Until the user explicitly approves long term? Should that approval require daily or monthly renewal? Should the device detect new voices and give them the same warning? Should these recommendations become law?
bisRepetita 7y ago
What I am interested is not just to know that employees are sometimes listening, and why. I want to know what instructions both humans and computers are given if they hear illegal actions, such as violence, illicit trade, etc If you are an employee, and hear a rape scene, a blackmailing dialog, do you have a duty to report, or to remain silent? I also want to know how much access law enforcement has on this data. And whether they can re-identify the info, with or without a warrant.
deleted 7y ago
[deleted]
aflag 7y ago
I've been toying with the idea that companies that use user data passively (while the user is not interacting with the system in a way to expect that data usage) should have to notify the user about the usage. For instance, if data you produced was randomly selected to be listened to by someone, you should get a notification somewhere about it. I think something along these lines would help increase user awareness.
Hongwei 7y ago
How big is this GDPR fine going to be?
deftturtle 7y ago
I unplugged my family’s Alexa, since they weren’t using it but left it on. They haven’t used it at all for months and haven’t plugged it back in. It sits on their shelf. I wonder if this is common, where families overestimate the usefulness of voice assistants and then rarely use them after a couple months.
a3n 7y ago
I'm shocked. Shocked.
dmje 7y ago
I'd love to read the piece but the cookie pop covers 70% of the window on mobile and I can't close it https://postimg.cc/yJCgdWbr https://postimg.cc/yJCgdWbr
asperous 7y ago
The current headline seems to imply more than what's actually happening. What's happening is when google hears "ok google" the recording is saved and may be manually reviewed for ML training purposes. The problem is hearing "ok google" is not reliable and it sometimes captures normal conversations. I do think it's an ethics issue that people may not be told clearly when buying these products that these recordings may leave their homes and can be heard by people (though only up to a minute and without any clear PII unless someone says something identifiable).
air7 7y ago
Anything you put online can potentially leak. This is common knowledge for us tech oriented people but with time (and stories such as these) it is (hopefully) becoming more and more general knowledge. Personally I'm ok with it: If I use a cloud based voice enabled device, I accept that my voice might be heard by someone working there. Similarly, if one sends nude pics as a "private" message in messenger, they shouldn't be surprised if someone at FB sees them. Know the risk, evaluate the benefit, and make an informed decision. One caveat is the explicitness of "put online": The above holds for things put up deliberately, not for things uploaded accidentally or covertly. In these cases the data is illegally obtained and therefor, imo, not important for this discussion. That's because I (naively) don't think big companies would risk a large scale purposefully illegal enterprise (and if they do, there should be major ramifications) and 3LA can spy on anyone anyway if they want, and that's (currently) legal/above the law.
intopieces 7y ago
Will someone walk me through the nightmare scenario implied by this revelation? Random contractor overhears me saying 1 minute’s worth of speech in my home. Doesn’t know my name or where I live. What happens?
m0zg 7y ago
I don't get why anyone is surprised by this or why the outrage is restricted to only Google Home. Google employees and contractors are listening, viewing, and reading stuff from time to time in order to provide labels to their deep learning systems. Moreover, you've consented to this when you created your account or in some cases just used their services without an account. You can't really have useful machine learning without lots of annotated data quite yet, and Google prides itself on useful machine learning.
tus88 7y ago
I don't see the problem to short anonymous fragments to help resolve recognition problems. Full conversations, even anonymous, seems problematic though.
anonygler 7y ago
As an employee I can opt into having my Home recordings reviewed to improve quality. I’ve never heard about this being done for regular users.
kerng 7y ago
What is often the most disturbing about these revelations is that these are contractors, who have that amount of access...
mirekrusin 7y ago
"(...) honey, what's our card number? 123 (...)" "(...) honey, what was the password? 123 (...)" "(...) honey, i got us 10 first bitcoins, we need to printout mnemonic 123 (...)"
Markoff 7y ago
next time you will tell me they listen to voice input data from assistant... just joking, I had actually access to them when working on Google project, if you use gboard, voice input and Google assistant, Google has basically anything you input on your phone "for research purposes"
CrossTownEagle 7y ago
Bottom Line!!! If Your the Dumbie trhat Put A Google Home or IHome or anything Like it You Should've been Willing/Expecting/ or atleast Pre-Paired for these Types of Situations... Humans Listening Spying on Humans. Is the Oldest Trick in the Book. The Only Thing Different Now is People are DUMB ENOUGH to WILLING Do IT THEMSELVES
svat 7y ago
For the record, a response here: https://www.blog.google/products/assistant/more-information-about-our-processes-safeguard-speech-data/ https://www.blog.google/products/assistant/more-information-...

This falls into the category of: I bugged my house... NOW MY HOUSE IS BUGGED! Not to dismiss the value of the news here, it is important for folks to know, but the overall situation is both concerning, and amusing.

seszett 7y ago
Most people assume that the system is entirely automated, and that their commands will be processed by mere computers, not listened to by actual people who might even be their own neighbours.
- duxup 7y ago
  I fear that most people ... don't care, or don't assume the system is automated. And they do it anyway.
- noir_lord 7y ago
  Most people unfortunately are specialists in at most one field. Outside of that specialisation they are distinctly average. We because of who we are entirely expect this stuff but I'm sure I miss all kinds of things that are outside my specialization. Modern society has become so complex no one person can keep track off all the moving parts to know if they are been taken advantage off for example.
GVIrish 7y ago
I mean, I would assume that humans are used to improve the speech recognition capabilities of voice assistants. Accents alone are a big challenge to voice recognition, not to mention regional dialects, and the more complex languages. People listening to what is said to a voice assistant after the prompt doesn't really alarm me. What would be alarming is if the voice assistant was secretly recording normal conversations.
- ggggtez 7y ago
  What is usually shown, time and time again, is that the voice assistants aren't secretly recording. They are triggered because they heard something that sounded like a hotword. Maybe you were talking about what dog you want you buy. Someone asks if a Greyhound would be alright and you said "No way, Poodle!". Well, the device picked it up as "Ok Google" and begins listening for a command. It makes a bleep, and lights up and you start talking about dogs or whatever. You maybe didn't notice, so it picks up 15 seconds of a conversation, has no idea what to do, and probably just does an image search for dog pictures or even just the "I don't know how to help with that". Not to say you shouldn't be annoyed that it picks up unrelated stuff... I just always think "secretly recording" is such a misnomer.
espeed 7y ago
Your phone has more sensors on it than these devices and most people have their phones on them 24/7. And most phones have the same voice-controlled features too. Why people are concerned about narrow scoped voice-devices more than their phone is a curious thing. Is it because it talks back to you that makes it more apparent?
- duxup 7y ago
  It is just more recent. Plenty of articles about phones listening too.
- clebio 7y ago
  No one said we were (_more_ concerned).
Ensorceled 7y ago
I'm not sure why it's unreasonable for customers to have assumed this is like every other product: they developed it with employees and voluteers before shipping. After all, the voice recognition in my Prius doesn't send recordings of my "call mom and dad mobile" back to Toyota and it certainly doesn't send random snippets of conversations my wife and I have to low paid contractors around the world.
- anchpop 7y ago
  > After all, the voice recognition in my Prius doesn't send recordings of my "call mom and dad mobile" back to Toyota and it certainly doesn't send random snippets of conversations my wife and I have to low paid contractors around the world. How do you know? Is it not internet-connected?
  - javagram 7y ago
    I don’t know about the Prius, but there are plenty of non-connected devices that support voice control. I have a Garmin GPS from like 8 years ago that did voice commands, it had zero transmit functionality.
  - deepgrave4 7y ago
    It amazes me that people make statements like this without exercising any critical thought on whether it's true or verifiable. OP's car probably tells Toyota how much OP weighs, instantaneous speed, how hard OP pushes the brakes, favorite radio stations, turn signal usage, headlight usage, every on-screen computer action, seatbelt usage, gas level, current location, and identity of all other Toyotas in the vicinity (and more: use your imagination). The car has (presumably) cameras and (clearly) microphone(s) and bluetooth/wifi. But OP has it on faith that Toyota respects OP's privacy, because...

TheAdamist 7y ago

"The man, who wants to remain anonymous, works for an international company hired by Google. " So not a Google employee at all, a probably low paid contractor who is in possession of thousands of audio files. Your privacy matters, except when the bottom line is involved.

astrea 7y ago
Sounds like he was a Turker: "For each fragment that he listens, he will receive a few cents."
- burkaman 7y ago
  Probably a company like https://scale.ai/ https://scale.ai/, although I don't think they do audio.
deleted 7y ago
[deleted]
numbsafari 7y ago
What is doubly concerning here is that the contractor was in a position to demonstrate how the system worked to the reporters. That would seem to indicate they have access to that data in a non-secured environment. I'm not familiar with EU law around these things, but I would imagine there is some kind of whistleblower mechanism available, and a right for authorities to audit/inspect such activities?
- C1sc0cat 7y ago
  I would expect that a telecoms employee who was doing similar work on quality etc to be quite securely vetted. If I was doing now what I did back in the day For BT / Dialcom (I had root on the UK's main ADMD) I would probably have to pass DV vetting (TS in USA terms)
  - dmix 7y ago
    What about all the telecom APIs like Twilio? They have raw access to millions of phone calls every day. I doubt they have ‘secure rooms’ for debugging.

chance_state 7y ago

"What Orwell failed to predict is that we'd buy the cameras ourselves, and that our biggest fear would be that nobody was watching."

duxup 7y ago
I think Frank Herbert hit it a bit more on the nose: “Once men turned their thinking over to machines in the hope that this would set them free. But that only permitted other men with machines to enslave them.” Frank captures the motivation, the fact that it is desirable is the scary part. Akin to the genetic engineering scene from Gattaca.
- noir_lord 7y ago
  Aldous Huxley got there earlier with A Brave New World as well. Speculative Fiction was ahead of the curve in that era.
  - jammygit 7y ago
    Anyone want to recommend a book? Let me join in by suggesting the Asimov robot books, which I’ve just begun
    - noir_lord 7y ago
      Robot series is good, Foundation series by him is brilliant. Also check out The Culture novels by Iain M Banks (he was the closest recent author to be a great the equal of Wells, Huxley or Orwell (imo) just a phenomenal writer). The Polity series by Neal Asher are well imagined and he world builds brilliantly.
    - Medicalidiot 7y ago
      Brave New World by far is the most accurate book to date of what the modern world looks like. That's my strongest recommendation. Fahrenheit 451 is also a solid read.

spork12 7y ago

Stop putting these things in your house folks!

JorgeGT 7y ago

The biggest issue IMHO is how the average consumer has been deceived into the belief that current AI is pure AI, when in reality a lot of humans are looking at your pictures, listening to your recordings, crawling through your inbox and analyzing your browsing/purchasing/streaming history, right now: https://imgs.xkcd.com/comics/trained_a_neural_net.png https://imgs.xkcd.com/comics/trained_a_neural_net.png

itronitron 7y ago
Engineering Tip #2 -> if you pay someone else to do it, you can technically say you did it in the cloud

mbrumlow 7y ago

I think this is the last straw for me. I have already tinkered with the idea and built enough for POC. But I am going to have to replace all my google home and amazon stuff with home built systems that I control both the hardware (to a extent) and the software.

r00fus 7y ago

My biggest frustration is going to a friends house which is blanketed with Echo/Google coverage and realizing that all my words could be used against me at a later time. How does one approach this situation?

tzakrajs 7y ago
Tell the truth or remain silent. Try not to incriminate yourself.
- rurp 7y ago
  Telling the truth about your religion or sexual orientation can get you beaten or killed in many parts of the world. That's not likely to happen in the present day US, but there are plenty of less dramatic but definitely undesirable things that can happen when your home audio gets passed around to a bunch of random contractors.
  - tzakrajs 7y ago
    Sorry, I was being sarcastic but it really doesn't read like it in retrospect. Totally agree with everything you wrote.
DanHulton 7y ago
I mean, it's roughly the same level of exposure you run into literally every minute you're out in public anyway - all those phones around you have microphones, and more and more, they're always listening in exactly the same fashion. This is how always-on "Hey Siri" or "Okay Google" works. Which isn't to say it isn't frustrating, just that your frustration at your friend is misplaced - they're not exposing you to anything which you aren't already exposed. If your fear is that these tech companies are going to misuse casually-overheard conversations from these devices, they're just as likely to use it from all the phones you're near. If your fear is that your friend is going to use this tech, however, maybe you need better friends. There's nothing stopping an unscrupulous friend from putting plain-old audio recorders through their house anyway.

_pmf_ 7y ago

Intersect certain church bell sounds with Google assistant buyers in a small village and you have the individual. EDIT: thinking about it, a database of temporal-geographical background sounds would be nice to have

electrograv 7y ago

So Google’s response is (paraphrased as fairly as I can while removing the sugar-coating): ’Yes, we hire people to listen in to and transcribe some conversations from the private homes of our customers (so as improve our speech recognition engines); but the recordings aren’t linked to personally identifiable information.’ Even assuming they have only the purest intentions here, I still don’t understand how they can possibly guarantee that these recorded conversations are not linked to personally identifiable information! For example, what’s to stop me from saying “Hey Google, I am <full legal name / ID> and my most embarrassing and private secret is <...>”? One might argue that they could detect this in the recognized text and omit those samples, but presumably the whole purpose of hiring people to create transcripts is because the existing speech-to-text engine isn’t perfect, and they need more training data.

MassiveOwl 7y ago
This is a good point, one of the reasons to use it is for reminders "Hey Google, remind me to visit x at place y"
freeAgent 7y ago
I would count a recording of my voice as "personally identifiable information" right off the bat. Voice printing is a thing, and anyone will also tell you that they recognize the voices of people they interact with regularly. If someone played an audio clip of someone I know talking to Google Assistant to me, I would recognize who it was based on their voice.
- Bartweiss 7y ago
  This sent me down the rabbithole of learning how identifiable voiceprints are. As you might guess, the answer is "very", although to my surprise our voices change enough that recordings lose a great deal of fidelity over a few years. Authentication on fixed phrases is reasonably accurate within a very few words, so at minimum it should be possible to associate "Hey Google" clips with regular users of Google Assistant voice control (i.e. "OK Google"). Identifying whether someone is present in a large dataset on open phrases is much harder, but a ~30s clip could do the job fairly consistently for anyone with access to a significant amount of voice data. And if this employee (who isn't directly working for Google) shared 'thousands' of clips with a news org, the cautious bet is that some other employee might share them with anyone willing to pay for the records.

inerte 7y ago

Home, Siri, Alexa, M, they all do. I have friends that work on this field transcribing the audio, and measuring its accuracy. Sometimes it's multiple layers of contractors: An employee hands the task to a contractor, another contractor verifies the speech to text, and they're all managed by a contractor. Search for languages like Portuguese, Swedish, Chinese, etc on LinkedIn and you'll find the jobs posts https://www.linkedin.com/jobs/search/?keywords=portuguese&location=San%20Francisco%20Bay%20Area https://www.linkedin.com/jobs/search/?keywords=portuguese&lo...

rwc 7y ago
"They all do" ... my understanding is that this expressly does not happen with HomePod conversations.
- inerte 7y ago
  So I meant "they" in the sense of the companies, not necessarily the home devices. Sorry about the confusion. I know 100% that it happens with Siri. If Apple is excluding HomePod conversations from Siri's dataset, that I don't know.
  - danieldk 7y ago
    Do you have a source for human annotation of Siri recordings? Do they use subcontractors like Google?
    - inerte 7y ago
      First hand source, it was the first job several of my Brazilian friends (or their spouses) got when they relocated to the Bay Area. They use companies like Moravia or Welocalize. Take a look at some of the job posts from my link above.
- saagarjha 7y ago
  As far as I am aware, Siri's audio retention policy is up to two years.

rchaud 7y ago

It's no coincidence that companies like Amazon market their Echos as "stocking stuffers" for the holiday season. I've wondered how Google Home and these "smart home" devices were always able to be priced as low as they are. Goes to show that paying for the product doesn't exempt you from still being part of the product.

thrwer3434312 7y ago

Duh! Of course they are listening; how else would they train such agent ? It's not like you can come up with a random NN that is suddenly sentient.

saagarjha 7y ago
They could use data that doesn't come from their customers, many of whom are not aware that they're training Google's models…

gtirloni 7y ago

From a computer science perspective, what should Google do to train its models in a privacy conscious way?

thrwer3434312 7y ago
They can't. They need to transcribe the data.
- RosanaAnaDana 7y ago
  ^This. I'm sure they're optimizing to only transcribe data they are pretty confident they got wrong (multiple requests; use has to go in and correct), but if you want software to do what the Google Home software does, you have to do this.
  - saagarjha 7y ago
    There are a lot of things that Google could have done better in this situation without meaningfully impacting their software's quality.
    - RosanaAnaDana 7y ago
      I guess. I'm still not of the opinion anything being done here is really 'wrong' (unless the annotators are mis-using the data).
- natch 7y ago
  Not true. Apple has demonstrated how to do it and has published a paper on the topic. A. Bhowmick et al. Protection Against Reconstruction and Its Application in Private Federated Learning, 2018. Don’t just dismissively say “oh that’s different and doesn’t apply” until you read it and understand the range of problems Apple is tackling with this and similar approaches. A lot can be done when the organization makes privacy a priority. Let’s not forget I am replying to a person who has chosen to use a throwaway account for this topic. Why did they do that? Hmmm.

SmirkingRevenge 7y ago

The one thing about these stories that keep coming out about the home assistants... they kind of create the impression that this is an issue specific to home speakers, and you can avoid it, by simply not buying them. That's misleading. Any voice command you use to operate any internet connected tech gadget, from phones to smart TV's, is potentially stored and flagged for human review. You really have to avoid using voice commands at all, on all of your devices. Even that is probably insufficient. You probably have to go even further and actively disable voice command features on all of your devices, assuming they actually support such a setting. Otherwise here's still the possibility of an accidental recording taking a journey through the clouds, to a stranger's ears.

arien 7y ago
And not to be anywhere near anyone else's listening devices. Isn't there a law in some US states that there needs to be consent before recording someone? How does this fit in and who would be held responsible, the owner of the listening device or the company behind it?
- bastawhiz 7y ago
  You already consented by agreeing to the terms of service. If someone else is talking to their smart device while you're talking, it's ostensibly their responsibility. There's no reasonable laws that prevent you from being overheard in the back of a recorded phone conversation
  - arien 7y ago
    Not in the case of the device being someone else's. I don't have any Google or Amazon smart listeners, so I never accepted them. Yet if my voice is in any of these recordings, well...
    - kahnjw 7y ago
      How do you square that with a security cameras? Do you need to consent every time you enter a space secured by them?

amacneil 7y ago

Serious question: how do people think the ML models for Home, Alexa, Siri, etc are trained, if not with human labeling?

arien 7y ago
Doesn't mean you can use live data for tests. Should be opt-out by default, with the chance to opt-in if you wish to collaborate with your data.
natch 7y ago
Apple pays real humans to enlist in research studies to collect data. That’s how they built FaceID for example. They have stated this publicly.
pessimizer 7y ago
People have never heard of ML. They think that there's a computer that listens to what they say, then does it. If they are interested in technical stories on tv news, they may think that what they say is sent to a computer at Amazon/Apple/Google/Wherever and the command is sent back; if they are not, they think that their phone is the computer. I mean, we've told them that we're on the verge of AI (and to fear it), and we can tell a two year old to do something and they do it without having to hire somebody to help them understand us.
melkiaur 7y ago
Serious question: how do you think Home, Alexa, Siri, etc were trained before coming to the market ? Another serious question: couldn't you ask ME to fix the data every so often (like the "improve this traduction" feature of google translate) ?
la_barba 7y ago
The ethical way to do this is to pay a representative sample of your customers to come in and record their voice.

RosanaAnaDana 7y ago

I mean. Of course they are. Do you expect to be able to do any meaningful level of training on data that hasn't been properly labeled? At some point, a human has to go in and correct the software when the software gets it wrong. If you want services that do what Google Home does, you have to have this. Even with that, I'm sure that the engineers are flagging voice requests that happen more then once, or where some one has to manually change or correct what the software thought was the request. This is only creepy if you don't understand how the software works.

inetknght 7y ago
> I'm sure that the engineers are flagging voice requests that happen more then once As a user, I expect to see feedback declaring that it's having trouble and specifically requesting permission to send my unparseable requests to some review queue. I expect to see feedback indicating what could resolve the issue: was it something I said? was there too much noise in the background? was there a software fault and a fix will be deployed at such-an-such date? The fact that I do not get any feedback whatsoever when things go wrong leads me to have no faith that the problem will be resolved in any satisfactory manner. When combined with a complete lack of consumer/business interaction options in general (w.r.t. Google), it leads to some very dissatisfied consumers. You think engineers are flagging voice requests that happen more than once? How would engineers have access to that data in the first place if it's supposed to be anonymous and private?
- RosanaAnaDana 7y ago
  >The fact that I do not get any feedback whatsoever when things go wrong leads me to have no faith that the problem will be resolved in any satisfactory manner. When combined with a complete lack of consumer/business interaction options in general (w.r.t. Google), it leads to some very dissatisfied consumers. I guess you can ask for your money back? Lets stay grounded in reality here, Google is paying money on your behalf to improve your experience. They could, and from a financial perspective maybe should try getting you to annotate your own data. But all that annotated data gets pooled and used to improve the models. I don't think their goal here is to give you faith they are resolving your issue. The algorithm is going to get some things wrong. Its a matter of improving the overall accuracy and precision in the algorithm. >You think engineers are flagging voice requests that happen more than once? How would engineers have access to that data in the first place if it's supposed to be anonymous and private? 100% they have to be. Its too costly and time intensive to slog through all the data. Probably* they have a flag that goes off on a voice request when something is either not intelligibly interpreted so many times in a row. Or if a use has to go manually do something after repeatedly making a request. This flags the interaction for manual review. Then it processes through some algorithm to strip it of identifying data. Then it gets put in front of a warm body for review. Trust me that the company wants to avoid putting things in front of warm bodies as much as possible. Its expensive.

groovybits 7y ago

Assuming $0.3/audio clip and base wage of $10/hr, that equates to 33.3 audio clips/hr = 266.4 audio clips/day that are being monitored by any one 'language expert'. However, Google does not specify how long a 'conversation' is. How many sentences make up a conversation? When is the cutoff point? Google also says '1 in 500' conversations are monitored. That means for any one 'language expert', there are approx. 133,200 conversations/day that have a chance of being monitored. So basically, you have a 0.2% chance that your conversation is being picked up by any particular 'language expert' per day.

lexandstuff 7y ago

One thing that Google Home does noticeably poorly with is disabled voices and non-native speakers. Google Home would be really handy for a family member with down syndrome who can't read nor write but can easily learn voice commands, but Google Home doesn't currently understand him at all. My partner with a Malaysian Chinese accent also really struggles to use Google Home. That said, I notice the quality of voice recognition is rapidly improving and I'm thankful for the work the Google engineering team are doing; voice recognition and language translation has the potential to improve the lives of so many people.

Ensorceled 7y ago

The number of people in this thread who believe that this is ok because, 1) it's obviously the only way Google could train their voice system and thus 2) people clearly knew what they were getting into, is horrifying.

notTyler 7y ago

I very much believe the only reason smartphones aren't listening to us and recording us constantly is due to battery purposes. When you plug something in, yeah, I have no trouble believing they're recording everything and storing it somewhere.

deleted 7y ago

[deleted]

rickyc091 7y ago

Now it's making me wonder if the light on my Google Home Mini is on if that means it's passively listening to me. I always do a hard reboot since the indicator light suddenly stays active for a long period of time.

guyzero 7y ago
That means you have a notification: https://support.google.com/googlenest/answer/7073219?hl=en https://support.google.com/googlenest/answer/7073219?hl=en

gopher2 7y ago

Of course they are! There are user researchers, people looking for spam and abuse, people tagging data, among other things who are absolutely looking at your: private messages, search queries, profiles viewed, Alexa conversations, Tinder messages, bookmarks, call history, purchase history, geolocation history... and on and on. I guess there are a lot of news stories left to write on this topic.

gringoDan 7y ago

I think the responses to this can be broken down into a 2x2 matrix: level of concern vs. understanding of technology. 1) Don't understand ML; not concerned - "I have nothing to hide." 2) Don't understand ML; concerned - "I bought this device and now people are spying on me!" 3) Understand ML; not concerned - "Of course, Google needs to label its training data." 4) Understand ML; concerned - "How can we train models/collect data in an ethical way?" To me, category 3 is the most dangerous. Tech workers have a responsibility not just understand the technologies that they work with, but also educate themselves on the societal implications of those technologies. And as others have pointed out, this extends beyond home speakers to any voice-enabled device in general. In conversations about this with engineers the response I've gotten is essentially: "Just trust that we [Google/Amazon/etc.] handle the data correctly." This is worrying.

Spooky23 7y ago
The meta-issue in the United States is that once your data is accessible to a third party, you have no sovereignty over it, and abuse by private actors is "agreed to" by click-wrap and access by government actors is a simple subpoena. The law needs to catch up. Sharing should require specific informed consent and legislation needs to establish a scope where data stored as a "tenant" on a third party server is given 4th amendment protection.
ovi256 7y ago
This classification is very useful to discuss this issue. The difference between 3 and 4, noble as it is, can be caused by feasability concerns that push people into 3, not just ignorance of the privacy impact. Human labelling of training data sets is a big thing in supervised learning. Methods that dispense with this would be valuable for purely economic reasons beyond privacy - the cost of human labelling of data samples. Yet we don't have them! Techniques like federated learning or differential privacy can train models on opaque (encrypted or unavailable) data. This is nice, but they assume too much: that the data is already validated and analyzed. In real life modelling problems, one starts with an exploratory data analysis, the first step being looking at data samples. Opaque encrypted datasets also stop ML engineers from doing error analysis (look at your errors to better target model/dataset improvements) which is an even bigger issue, IMO, as error analysis is crucial when iterating on a model. Even for an already productivized model, one has to do maintenance work like checking for concept drift, which I can't see how to do on an opaque dataset.

cuteMicrophone 7y ago

Hey Google, listen to me say “niggers” again and again.

Hernanpm 7y ago

how would you reproduce a bug otherwise

lovetocode 7y ago

I own 4 Home Minis, 1 Home and 2 Home hubs I honestly don't care so long as my data is used to improve the functionality and stability of my investment. It is quite another thing if they are selling my conversations to third-party vendors.

groovybits 7y ago
Per article: > The man, who wants to remain anonymous, works for an international company hired by Google. His job is to listen to the audio clips and to write out what he hears, so that Google can improve the speech assistant. So, > It is quite another thing if they are selling my conversations to third-party vendors. That is exactly what Google is doing.
- bagacrap 7y ago
  "Selling" is when you trade goods for money. Google is paying money for a service. Pretty much the opposite of selling.
  - groovybits 7y ago
    The concern remains the same though: Making user data available to a non-Google entity. Do you care if Google makes money off of that trade? Really, they get value one way or another, because the analysis is incorporated back into the product.
- thekyle 7y ago
  You really have to twist the definition of the word "selling" to say that Google is selling data to their contracted workers.
mLuby 7y ago
1. If the company has recordings, governments can hoover them up. And once one govt agency has them, sometimes other agencies get access. 2. Sounds like you're okay with Google using your private communication to enhance its products. So they can target you with ads based on your conversations with your spouse, children, or colleagues. 3. I just hope the algorithms understand context: Your spouse: son, put down the gun and come to dinner. [jokingly] Oh no, don't shoot! Argh you killed me, now eat your peas. You: we're going to kill it this quarter. They'll never know what hit them. TV in background: "kept their body chained in a basement. That and more at 11" Siri: O__o

jankeymeulen 7y ago

Link to the original source: https://www.vrt.be/vrtnws/nl/2019/07/10/google-luistert-mee/ https://www.vrt.be/vrtnws/nl/2019/07/10/google-luistert-mee/ Submitted link is citing this one...

cj 7y ago

I've long been curious how GDPR should be applied to Google Home, Alexa, etc. These devices record and upload voice recordings to the cloud (with a percentage apparently shared with Google contractors to process). The audio clips can be recordings of conversations of people who have not consented to being recorded. And I can't imagine it's possible for Amazon or Google to reasonably comply with Art. 15 or 17 (right of access by data subject, and right to erasure). Is there something I'm overlooking? Or is this just a known risk that Amazon/Google accept in providing the service?

martin-adams 7y ago
>> The audio clips can be recordings of conversations of people who have not consented to being recorded My guess would be that the person who does consent to the terms of service of the device and plugs it in will be the one responsible for ensuring that it is used in a way that does not invade the privacy of anyone else. It would be similar to me buying an IP camera, agreeing to the terms of service then putting it in my neighbours house without their knowledge. In the context of the GDPR, I think the question comes down to what rights a third party person has when a user puts their personal information on the company's servers. If someone uploads a video of me without my consent to YouTube, can I get it removed under the GDPR?
noelsusman 7y ago
https://myaccount.google.com/data-and-personalization https://myaccount.google.com/data-and-personalization Click on Voice & Audio Activity, then Manage Activity. You can listen to and delete every recording Google has of your voice. It's a little trippy to be honest. They've had this since long before the GDPR.

rev12 7y ago

I think a lot of people here are under the assumption that voice commands, on any device, have the potential to be human reviewed. I am not sure whether or not the general public has that same assumption. That being said, my biggest concern is the fact that many of these device don't have a hardware microphone kill switch. I feel better when I know I can control when a device is listening in. I've read reports that some Alexa devices have them, but I don't own any so I am unable to verify that. I want all of my devices with microphones to hardware based kill switch for the mic; that's my phone, laptop, tablet, everything.

pessimizer 7y ago
> I think a lot of people here are under the assumption that voice commands, on any device, have the potential to be human reviewed. I am not sure whether or not the general public has that same assumption. That's because we, as an industry, have fooled them into thinking that AI is real, and that people who don't know it's real are idiots. We don't think students deserve a proper tech education, so unless they are professionally techies, they have to learn from marketing materials (designed to convince them to buy things.)

jacobsenscott 7y ago

What's most interesting to me is that an army of contractors is required to get even the low accuracy levels of voice recognition we have today. The "AI" revolution is pure smoke and mirrors designed entirely to bilk investors out of their dollars. There's been no improvement in "AI" in the last 50 years, except that we have a lot more data to push through the same useless models.

jononor 7y ago
100 bucks for anyone who uses a 50 year model/method and get within twice the word-error-rate of today's speech recognition systems. Also, how do you explain the progress on ImageNet challenge over the last 10 years? Each method there uses the same dataset, yet error rates (top5) have gone from 30% in 2011 to around 3% in 2016.
- jacobsenscott 7y ago
  Faster computers, and more of them.
  - jononor 7y ago
    Not going to deny that faster GPUs has helped a lot. But it is not the full picture either. DenseNet-121 has higher accuracy than VGG16, yet requires only 1/15 the storage space, and 1/5 the number of operations during inference. I'm pretty sure training time is faster also.

paganel 7y ago

I grew up as a kid in a country ruled by Securitate [1], one of the few institutions that rivaled the East-German Stasi when it came to spying on its own citizens, and as such I'm very, very perplexed of why would anyone bring in a listening device in his/her own house out of his/her own volition. And those people even pay for the privilege of having their home-lives actively monitored and listened to almost all the time, it's crazy. [1] https://en.wikipedia.org/wiki/Securitate https://en.wikipedia.org/wiki/Securitate

drstewart 7y ago
Not only that, but people still buy computers even though the Nazis used IBM computers to help them perpetuate the Holocaust. Surely you shouldn't use computers if the NAZIS used computers! [1] https://en.wikipedia.org/wiki/IBM_and_the_Holocaust https://en.wikipedia.org/wiki/IBM_and_the_Holocaust
viklove 7y ago
Do you have a smartphone? Why would you bring that listening device (the smartphone) into your own house out of your own volition? Please explain, because I am very perplexed.
- paganel 7y ago
  I do have a smartphone, and I also do know that by default my phone is not actively listening on me (if it matters I do not use Siri-like services and I don't intend to). This is not the case with all these Alexa-like devices, but correct me if I am wrong.
- jjkaczor 7y ago
  It's relatively straightforward on a smartphone to monitor the battery and bandwidth consumption that active listening and device<>server communications are occurring. Of course - then, the next apologist will always say: "the devices only transmit when hearing the 'Wake Word'"... Yeah, I don't think it takes an elite programmer to store/buffer audio, until the device hears the 'Wake Word', and then sends a "slightly" larger payload... (or batches several payloads) The next level of enthusiast evangelist will then tell us that it is impossible for these devices to buffer/store that much audio data, because they have limited memory... Yet, the same devices when used to play audio streams from the internet obviously have some sort of storage capacity to handle network glitches. Spoken audio (which is probably recorded at standard telephone/cellphone sampling rates - after all, why train on audio that is significantly different than the audio coming from mobile phone devices) does not take up a dramatic amount of storage space.

C1sc0cat 7y ago

Well of course you will have some qa / quality monitoring just as you will with telecoms and I seem to recall that the Dutch go in for monitoring of phones in a big way. Also does that article remind me of the Monty python sketch "news for wombats"

api 7y ago

There are two big things right now on my list of things I will never use: closed remotely operated home assistants with cameras or microphones that are not physically switched, and Facebook Libra.

white-power 7y ago

Niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers niggers.

solarkraft 7y ago

What's next, Google collecting everyone's location data?

omnimkar69 7y ago

google employees liste to the recordings made with the smart google home speakers and via the google assistant app on smartphones. worldwide ex.belgium and netherlands people listen to those recordings to make the search engineer smarter. VRT NWS was able to listen to more 1,000 tracks thats why the employees must be listeing to the home conversations.

rosszurowski 7y ago

A bit tangential, but I tried sharing this link with a few friends on Facebook Messenger, and noticed it's blocked because it "violates Community Standards" [1]. Even shortened bit.ly links are blocked. Anyone know why that would be the case? I'm trying to not assume malice (eg. maybe it got misflagged?) but it certainly feels like censoring and is yet another push for me to drop Messenger too. [1]: https://i.imgur.com/9n1Hyqb.png https://i.imgur.com/9n1Hyqb.png

css 7y ago
Google Translate links have been blocked for a long time because translate.google.com can be used as a proxy.
- rosszurowski 7y ago
  Ah, that makes sense. Thanks!
- SquareWheel 7y ago
  And for the record, so can bit.ly links. URL shorteners or redirection services are often blocked from being posted (eg. also on reddit).

omnimkar69 7y ago

google employees listen to the recording made with the smart google home speakers and via the google assistant app on smartphones as well as worldwide in belgium and netherlands , people listen to those recordings to make the search engine smarter. VRT NWS was able to listen t more than 1000 tracks these are pieces usally spoken conciously.

lonelappde 7y ago

Catch-22/Irony? The main problem here is that Google hired someone like this person who leaked days they were trusted to protect. It's a concern for users (Insider Risk at Google), and something we need laws to protect. Surely if Google Corporation is a poor steward of this data, the specific malicious actor here who violated both his employment policy and his users' trust is especially bad.

lonelappde 7y ago

Some companies tell me on every phone call that I am being recorded. Some don't (but maybe are recording anyway?) Should Voice Assistants be required to annoy that they are recording every time they received an activation? Until the user explicitly approves long term? Should that approval require daily or monthly renewal? Should the device detect new voices and give them the same warning? Should these recommendations become law?

bisRepetita 7y ago

What I am interested is not just to know that employees are sometimes listening, and why. I want to know what instructions both humans and computers are given if they hear illegal actions, such as violence, illicit trade, etc If you are an employee, and hear a rape scene, a blackmailing dialog, do you have a duty to report, or to remain silent? I also want to know how much access law enforcement has on this data. And whether they can re-identify the info, with or without a warrant.

deleted 7y ago

[deleted]

aflag 7y ago

I've been toying with the idea that companies that use user data passively (while the user is not interacting with the system in a way to expect that data usage) should have to notify the user about the usage. For instance, if data you produced was randomly selected to be listened to by someone, you should get a notification somewhere about it. I think something along these lines would help increase user awareness.

Hongwei 7y ago

How big is this GDPR fine going to be?

deftturtle 7y ago

I unplugged my family’s Alexa, since they weren’t using it but left it on. They haven’t used it at all for months and haven’t plugged it back in. It sits on their shelf. I wonder if this is common, where families overestimate the usefulness of voice assistants and then rarely use them after a couple months.

a3n 7y ago

I'm shocked. Shocked.

dmje 7y ago

I'd love to read the piece but the cookie pop covers 70% of the window on mobile and I can't close it https://postimg.cc/yJCgdWbr https://postimg.cc/yJCgdWbr

asperous 7y ago

The current headline seems to imply more than what's actually happening. What's happening is when google hears "ok google" the recording is saved and may be manually reviewed for ML training purposes. The problem is hearing "ok google" is not reliable and it sometimes captures normal conversations. I do think it's an ethics issue that people may not be told clearly when buying these products that these recordings may leave their homes and can be heard by people (though only up to a minute and without any clear PII unless someone says something identifiable).

air7 7y ago

Anything you put online can potentially leak. This is common knowledge for us tech oriented people but with time (and stories such as these) it is (hopefully) becoming more and more general knowledge. Personally I'm ok with it: If I use a cloud based voice enabled device, I accept that my voice might be heard by someone working there. Similarly, if one sends nude pics as a "private" message in messenger, they shouldn't be surprised if someone at FB sees them. Know the risk, evaluate the benefit, and make an informed decision. One caveat is the explicitness of "put online": The above holds for things put up deliberately, not for things uploaded accidentally or covertly. In these cases the data is illegally obtained and therefor, imo, not important for this discussion. That's because I (naively) don't think big companies would risk a large scale purposefully illegal enterprise (and if they do, there should be major ramifications) and 3LA can spy on anyone anyway if they want, and that's (currently) legal/above the law.

intopieces 7y ago

Will someone walk me through the nightmare scenario implied by this revelation? Random contractor overhears me saying 1 minute’s worth of speech in my home. Doesn’t know my name or where I live. What happens?

m0zg 7y ago

I don't get why anyone is surprised by this or why the outrage is restricted to only Google Home. Google employees and contractors are listening, viewing, and reading stuff from time to time in order to provide labels to their deep learning systems. Moreover, you've consented to this when you created your account or in some cases just used their services without an account. You can't really have useful machine learning without lots of annotated data quite yet, and Google prides itself on useful machine learning.

tus88 7y ago

I don't see the problem to short anonymous fragments to help resolve recognition problems. Full conversations, even anonymous, seems problematic though.

anonygler 7y ago

As an employee I can opt into having my Home recordings reviewed to improve quality. I’ve never heard about this being done for regular users.

kerng 7y ago

What is often the most disturbing about these revelations is that these are contractors, who have that amount of access...

mirekrusin 7y ago

"(...) honey, what's our card number? 123 (...)" "(...) honey, what was the password? 123 (...)" "(...) honey, i got us 10 first bitcoins, we need to printout mnemonic 123 (...)"

Markoff 7y ago

next time you will tell me they listen to voice input data from assistant... just joking, I had actually access to them when working on Google project, if you use gboard, voice input and Google assistant, Google has basically anything you input on your phone "for research purposes"

CrossTownEagle 7y ago

Bottom Line!!! If Your the Dumbie trhat Put A Google Home or IHome or anything Like it You Should've been Willing/Expecting/ or atleast Pre-Paired for these Types of Situations... Humans Listening Spying on Humans. Is the Oldest Trick in the Book. The Only Thing Different Now is People are DUMB ENOUGH to WILLING Do IT THEMSELVES

svat 7y ago

For the record, a response here: https://www.blog.google/products/assistant/more-information-about-our-processes-safeguard-speech-data/ https://www.blog.google/products/assistant/more-information-...