4 ms·

Overlords who casually murder non-supporters is not far-fetched by any stretch, such regimes have risen dozens of times just in the last 100 years. These overlo

Overlords who casually murder non-supporters is not far-fetched by any stretch, such regimes have risen dozens of times just in the last 100 years. These overlords haven't quite reached "worldwide" status yet, but use your imagination. A device that instantaneously and irrevocably transmits anything spoken to it to HQ, and then automatically transcribes it and makes it available for analysis, is absolutely a dangerous tool for censorship and repression. The fact that it looks nice on your counter and also spits out weather reports doesn't automatically invalidate those risks. These BigCos already compose complex, fairly accurate estimations of our interests and beliefs based on the data they read out of our emails, search history, and yes, voice commands. This data is used to make advertising decisions, but there's no reason that the same data can't be applied to "pre-moderate" the opinions of those who've expressed "dangerous views" in private, or worse. It just hasn't been used that way yet (to our knowledge, though it wouldn't surprise me to learn that Facebook or Google are already punishing/down-ranking accounts that express sentiments they dislike). Inviting these companies into your house to sit in every room and listen for "wake words", and then send up snippets of audio to HQ, should not be dismissed as a casual thing, even if the tradeoff is considered acceptable by some.

dpark 9y ago
This is absurd. When Bezos becomes the dark lord you imagine, just unplug your Echo and throw it away. Problem solved. In the meantime, your use or non-use of an Echo seems to have no meaningful impact on the rise of dark lord Bezos. Also you well know that the Echo is not transmitting everything it hears to HQ. It transmits only when invoked, which can be spurious but is not constant. Same for Apple and Google equivalents.
- IntronExon 9y ago
  I don’t think Bezos = Sauron... but this is just wrong: This is absurd. When Bezos becomes the dark lord you imagine, just unplug your Echo and throw it away. Problem solved. How much data will have been collected and stored by then? Let’s just put Bezos aside for a moment, and think about foreign and domestic intelligence orgs which will get their fingers into this. Let’s remember a bit of WWII, when harmless census data empowered the Nazis to track down people with Jewish ancestry. It’s not about what you think Bezos or today’s govt will do with the information, but what the corporate raiders and govts will do with it for the rest of your life.
  - dpark 9y ago
    > How much data will have been collected and stored by then? It doesn’t matter. If the concern is that Bezos will flip the echo to be a political surveillance device, there is an immediate fix. Yes, the existing cache of data may still be around, but that wasn’t the concern posed by the original poster. The concern was specifically that after Bezos becomes the dark lord, an “innocent” question could result in politically motivated murder. If your concern with the echo is that you don’t want anything recording anything in your house, that is reasonable and you should not get an echo. Of course, you should probably also not carry a phone. > Let’s remember a bit of WWII, when harmless census data empowered the Nazis to track down people with Jewish ancestry. And yet we still do the census because the value is high and the concern low. The fact that a thing can be abused does not mean it should not be used.
    - IntronExon 9y ago
      It doesn’t matter. If the concern is that Bezos will flip the echo to be a political surveillance device, there is an immediate fix. Yes, the existing cache of data may still be around, but that wasn’t the concern posed by the original poster. Well it’s my concern, and I suspect others have it too. I’m not trying to weigh in on the side of “Bezos is evil” by any means, I’m just adding a dimension that seemed to be neglected.
      - dpark 9y ago
        I totally understand that. It’s just not really the scenario proposed. Personally I’m not very concerned about the Echo, which is why I own it. But I do understand the reasonable concern that records could be subpoenaed or the data feed could be tapped.
- cookiecaper 9y ago
  It transmits everything after the wake word. My example of the Echo being used to monitor the political education of children is very plausible. It depends on a child using the wake word and asking the Echo an innocent question without understanding the broader implications, e.g., that something bad can happen if you let some big powerful group know that you're asking questions that they don't want people to ask. Last I checked, the Echo had 3 potential wake words: "Alexa", "Echo", and "Amazon". I don't use "Alexa" because I knew a kid named Alexa growing up and I didn't like her. I don't want to remember her every time I talk to the machine. I use "Echo" but when I'm listening to audiobooks or podcasts and the word "Echo" comes up, the thing activates, and maybe I _don't want_ Amazon to know what podcasts or books I listen to. It also happens on words that sound kinda-sorta like "Echo". Random "accidental" invocation and the loss of control over the listener then becomes a real thing, especially with a wake word like "Amazon". All they'd have to do is say "We've found 'Amazon' is the best wake word and 90% of our people use it so it's the default now", and suddenly, they'd be hearing a lot of casual conversations and opinions about what people think of this 'Amazon'. Now imagine that this really does become a tool for political oppression and they add a few "silent wake words" that don't make the ring light up. I have 3 Echos. My kids ask it questions all the time. If I were to teach my children some politically incorrect belief or some religious tenet that counters the dominant religious tenets of the day, and they ask the Echo for more information about it, someone at Amazon may hear that, and a computer at Amazon would certainly process and analyze it. Some day, the result of that analysis may be flagging for further investigation or addition to dissident watchlists. They may even apply this to archived recordings, all of which you yourself can play back (for now). Imagine someone from the other side of the aisle guessed $Prominent_Politician's Amazon password. What would happen if there was a voice recording of that politician asking something that doesn't look so good? Would the attacker just say "Oh boy I better not share this, that wouldn't be very nice?" In this age when people like Jo Cox and Steve Scalise are targeted by radical citizen-assassins, how can we pretend there is no risk? How can we pretend like this data is somehow intrinsically impregnable to technically sophisticated political extremists? And what happens when a demagogue mainstreams that extremist perspective, and writes laws that require Google or Amazon to submit these recordings for the good of the people? I'm not trying to be alarmist, and I obviously think that the tradeoff is not horrible right now since I have 3 of these spy devices (and a Nest cam), but we have to understand that these are real things, real possibilities. It is very naive to presume that this couldn't or wouldn't happen (or even that it doesn't on some scale -- I'm sure a subpoena or NSL could get this data).

dpark 9y ago

This is absurd. When Bezos becomes the dark lord you imagine, just unplug your Echo and throw it away. Problem solved. In the meantime, your use or non-use of an Echo seems to have no meaningful impact on the rise of dark lord Bezos. Also you well know that the Echo is not transmitting everything it hears to HQ. It transmits only when invoked, which can be spurious but is not constant. Same for Apple and Google equivalents.

IntronExon 9y ago
I don’t think Bezos = Sauron... but this is just wrong: This is absurd. When Bezos becomes the dark lord you imagine, just unplug your Echo and throw it away. Problem solved. How much data will have been collected and stored by then? Let’s just put Bezos aside for a moment, and think about foreign and domestic intelligence orgs which will get their fingers into this. Let’s remember a bit of WWII, when harmless census data empowered the Nazis to track down people with Jewish ancestry. It’s not about what you think Bezos or today’s govt will do with the information, but what the corporate raiders and govts will do with it for the rest of your life.
- dpark 9y ago
  > How much data will have been collected and stored by then? It doesn’t matter. If the concern is that Bezos will flip the echo to be a political surveillance device, there is an immediate fix. Yes, the existing cache of data may still be around, but that wasn’t the concern posed by the original poster. The concern was specifically that after Bezos becomes the dark lord, an “innocent” question could result in politically motivated murder. If your concern with the echo is that you don’t want anything recording anything in your house, that is reasonable and you should not get an echo. Of course, you should probably also not carry a phone. > Let’s remember a bit of WWII, when harmless census data empowered the Nazis to track down people with Jewish ancestry. And yet we still do the census because the value is high and the concern low. The fact that a thing can be abused does not mean it should not be used.
  - IntronExon 9y ago
    It doesn’t matter. If the concern is that Bezos will flip the echo to be a political surveillance device, there is an immediate fix. Yes, the existing cache of data may still be around, but that wasn’t the concern posed by the original poster. Well it’s my concern, and I suspect others have it too. I’m not trying to weigh in on the side of “Bezos is evil” by any means, I’m just adding a dimension that seemed to be neglected.
    - dpark 9y ago
      I totally understand that. It’s just not really the scenario proposed. Personally I’m not very concerned about the Echo, which is why I own it. But I do understand the reasonable concern that records could be subpoenaed or the data feed could be tapped.
cookiecaper 9y ago
It transmits everything after the wake word. My example of the Echo being used to monitor the political education of children is very plausible. It depends on a child using the wake word and asking the Echo an innocent question without understanding the broader implications, e.g., that something bad can happen if you let some big powerful group know that you're asking questions that they don't want people to ask. Last I checked, the Echo had 3 potential wake words: "Alexa", "Echo", and "Amazon". I don't use "Alexa" because I knew a kid named Alexa growing up and I didn't like her. I don't want to remember her every time I talk to the machine. I use "Echo" but when I'm listening to audiobooks or podcasts and the word "Echo" comes up, the thing activates, and maybe I _don't want_ Amazon to know what podcasts or books I listen to. It also happens on words that sound kinda-sorta like "Echo". Random "accidental" invocation and the loss of control over the listener then becomes a real thing, especially with a wake word like "Amazon". All they'd have to do is say "We've found 'Amazon' is the best wake word and 90% of our people use it so it's the default now", and suddenly, they'd be hearing a lot of casual conversations and opinions about what people think of this 'Amazon'. Now imagine that this really does become a tool for political oppression and they add a few "silent wake words" that don't make the ring light up. I have 3 Echos. My kids ask it questions all the time. If I were to teach my children some politically incorrect belief or some religious tenet that counters the dominant religious tenets of the day, and they ask the Echo for more information about it, someone at Amazon may hear that, and a computer at Amazon would certainly process and analyze it. Some day, the result of that analysis may be flagging for further investigation or addition to dissident watchlists. They may even apply this to archived recordings, all of which you yourself can play back (for now). Imagine someone from the other side of the aisle guessed $Prominent_Politician's Amazon password. What would happen if there was a voice recording of that politician asking something that doesn't look so good? Would the attacker just say "Oh boy I better not share this, that wouldn't be very nice?" In this age when people like Jo Cox and Steve Scalise are targeted by radical citizen-assassins, how can we pretend there is no risk? How can we pretend like this data is somehow intrinsically impregnable to technically sophisticated political extremists? And what happens when a demagogue mainstreams that extremist perspective, and writes laws that require Google or Amazon to submit these recordings for the good of the people? I'm not trying to be alarmist, and I obviously think that the tradeoff is not horrible right now since I have 3 of these spy devices (and a Nest cam), but we have to understand that these are real things, real possibilities. It is very naive to presume that this couldn't or wouldn't happen (or even that it doesn't on some scale -- I'm sure a subpoena or NSL could get this data).