7 ms·

I blame Star Trek. Simply speaking out loud to ask the computer anytime something was needed, it seemed like magical tech. We just didn't anticipate the compani

I blame Star Trek. Simply speaking out loud to ask the computer anytime something was needed, it seemed like magical tech. We just didn't anticipate the companies and extra consequences that would come along with it. I would love to see a parody of Star Trek where all the tech is run by an advertising company exploiting all the crew members individually by capturing all their private data around the clock.

api 7y ago
No relationship. Why does the device have to record and send everything to the cloud to provide this experience? I suspect the answer is a mix of "we want to invade privacy" and "it makes the devices cheaper to build since you can put really wimpy chips in them."
- shadowgovt 7y ago
  In the short run, it gives the company three direct benefits for making the technology work (besides the "They can advertise at you better" one that people like to drumbeat on). 1) Context is nearly invisible, and is constructed from dozens to hundreds of data points. Holding state on the user (not just from direct interaction with the agent, but from other services the agent can be aware of) makes a lot more context possible. My assistant knows about my calendar, where I've been recently, and my recent emails; it can make much better educated guesses about what I mean if I say "What's travel time to Oakland?" 2) Getting the voice recognition tech itself from 90% accurate to 99.9% accurate requires awareness of the tens of thousands of real-world complicators that never show up in laboratory testing. As Google showed off by bootstrapping their voice recognition tech via Goog-411, they know how to take megabytes / gigabytes of "dirty signal" and use it to refine their algorithms against unforeseen corner cases. 3) Releasing new assistant features can be done atop existing collected data, without requiring months of lag for enough state to build up for the feature to be useful.
  - api 7y ago
    All of these are addressable engineering challenges if privacy were a priority. It's just that privacy is not a priority. (1) is not that hard, just subscribe to feeds or query cloud services from the device. (2) is harder but could be done by treating the devices as a cluster or sending data up only when it can't be easily recognized and storing it only long enough to be useful in training (and storing it anonymously), and as for (3) the device could store data encrypted in the cloud and that would be fine. A new or updated device just downloads and decrypts the index to its data. There are other workable approaches. I think the best way to change that is legislation. We need HIPAA-type liability laws applying to all privacy-sensitive data: recorded audio, recorded video, location, etc. Leak someone's location or a recording of audio from their house? That'll be $10,000 per person per incident. That would instantly transform privacy invasion from an asset to a liability and incentivize companies to minimize data collected and when it must be collected to provide the service minimize the length of the time it is stored. As it stands literally all economic incentives jump up and down and scream "invade the user's privacy s much as possible!" These are: (1) Data is valuable and advertisers will pay for it or pay for data-driven ad services. (2) Engineering privacy-invasive systems is easier than engineering privacy-respecting systems. This is a subset of "engineering insecure systems is easier than engineering secure systems" as privacy is a dimension of security. (3) Privacy sensitive approaches usually require more compute and storage at the edge, increasing device build cost.
    - TeMPOraL 7y ago
      Let's not forget the other important issue: aligned with privacy-invasive technology is the business incentive towards turning products into services. This is what drives most of the bad engineering in IoT products - perhaps even more than the secondary revenue stream of selling users' data. When you make your device depend on the Internet for its most basic functions, you secure yourself a recurring revenue stream. I'd like to see regulations addressing this issue directly, in addition to more strict pro-privacy regulation. For instance, I'd love to see it so that data, software and compute become independent - instead of current vertical integration. That is, I want to a) own my data, b) choose where it'll be processed, and c) have the software processing it come to the data - this way the vendor supplying software for c) won't be able to take the data and sell it or hold it for ransom.
  - reaperducer 7y ago
    All of these items can be used through using test panels, or even dogfooding. There's no need to violate the privacy of millions of people.
    - yellow_postit 7y ago
      That’s not going to produce a good model. Test panels and dogfooding are both small scale and introduce biases based on who uses them. Until the gains stop coming primarily from supervised deep learning we won’t move away from the winner being whomever has the largest, most diverse dataset and compute resources.
  - Enginerrrd 7y ago
    I feel like another motivation is to keep the neural net parameters, AKA the most valuable part of the speech recognition algorithm, off-device. It'd have to be huge otherwise, and could be easily duplicated then used as a starting place for a competing product.
  - heavyset_go 7y ago
    "Violate the user's privacy in order to build a product" is lazy engineering. There are plenty of avenues to collect the data needed at that scale, it's just that none of them are as cheap as simply spying on users.
    - shadowgovt 7y ago
      Lazy engineering gets to market first, and if most users don't find it harmful, the company wins?
      - heavyset_go 7y ago
        Yes, spying on users is a cheap way to get results. The fact that incentives exist to violate user privacy is not an excuse to violate user privacy.
        shadowgovt 7y ago
        Giant piles of cash can buy many excuses. ;)
      - TeMPOraL 7y ago
        Lazy, half-assed, corner-cutting everything gets to market first. This is a problem. It's also why markets accrue regulations over time.
        shadowgovt 7y ago
        I agree, and it's likely that's the most correct answer if people want to see the market change. The second most correct answer is to just opt-out if you feel spied on and get to feel very smug watching people who didn't opt-out get burned by their choices. Honestly, I recommend that solution unless one encounters a situation where opt-out is impossible.
        api 7y ago
        We seem to have no good solution to this. Capitalism/cowboy development: ship crap fast, break stuff, accrue regulations. Socialism/planned economy: have meetings to plan meetings to discuss requirements forever and nothing ever happens. While both were government funded, the "cowboy" approach of ArpaNet and IP vs the OSI network protocol consortia are a great case study.
- Buge 7y ago
  > Why does the device have to record and send everything to the cloud to provide this experience? It's everything after the hotword, not everything.
  - RhodesianHunter 7y ago
    You're getting downvoted, but this is literally baked into the hardware.
    - TeMPOraL 7y ago
      Except when it isn't, or another "convenient" bug makes everything a hot word, turning the device into an always-recording Internet stream.
      - Buge 7y ago
        A "bug" in your phone could do the same.
mojuba 7y ago
I think it looks comfortable in the Star Trek universe because it is implied that voice activated stuff is controlled by a super-AI. That is, the AI won't give humans access to the audio. Which in turn means we are going to have a super-AI that can restrict our rights and itself have access to vast amounts of information about us. I.e. Star Trek is a society essentially controlled by the AI, as it will be above us.
- shadowgovt 7y ago
  It looks comfortable in the Star Trek universe because everyone trusts Starfleet and the ship's computer. I mean, if you don't trust the computer to look after your interests with regards to your voice interactions, why are you trusting it when it also controls life support and airlocks? ;)
  - sdenton4 7y ago
    Another classic SF example of a voice interface: "I'm sorry, Dave..."
    - K0SM0S 7y ago
      But what's an AI, what's 'artificial', when does something become so if it's made by/of 'natural' things? Which reminds me of BSG, they have airlocks too.
- pixelbath 7y ago
  I disagree. Given it takes place hundreds of years into our future, I'd assume speech recognition is just a "solved problem" and not something to worry about. Plus, there's also the Federation -> navy comparison. If speech recognition is mandated in Federation vessels, why would the crew worry about who's listening?
- api 7y ago
  I think it looks comfortable in the Star Trek universe because Star Trek portrays a society that is not run by mobsters, warlords, delusional ideologues, and sociopaths. We are still very much savages. Clinical psychopaths and delusional narcissists run our institutions and major decisions are made on the basis of fanatical superstitious ideologies and mindless herd behavior. In short I won't put an Internet connected microphone in my house and I'm very wary of privacy leakage in general because I don't trust the people who run our world (including both governments and corporations). I don't think this is irrational. Glancing around it seems like a decent number of world government and corporate leaders belong in prisons and mental institutions, not powerful offices.
TeMPOraL 7y ago
It works in Star Trek, because as 'api says, their society is sane. United Federation of Planets is an utopia, where people don't try to abuse and profit off each other. For a proper Star Trek comparison, imagine how comfortable Federation citizens would be with using ambient computing technology created and sold by the Ferengi, who are also known to be performing surveillance for Romulans in exchange for money. (For non-Trekkies here, Ferengi is a species of extremely greedy and exploitative capitalists; Romulans are what you'd get if you reified the Five Eyes into a species, and put them into state of cold war with the Federation.)
K0SM0S 7y ago
That parody needs to happen on Ferenginar.

api 7y ago

No relationship. Why does the device have to record and send everything to the cloud to provide this experience? I suspect the answer is a mix of "we want to invade privacy" and "it makes the devices cheaper to build since you can put really wimpy chips in them."

shadowgovt 7y ago
In the short run, it gives the company three direct benefits for making the technology work (besides the "They can advertise at you better" one that people like to drumbeat on). 1) Context is nearly invisible, and is constructed from dozens to hundreds of data points. Holding state on the user (not just from direct interaction with the agent, but from other services the agent can be aware of) makes a lot more context possible. My assistant knows about my calendar, where I've been recently, and my recent emails; it can make much better educated guesses about what I mean if I say "What's travel time to Oakland?" 2) Getting the voice recognition tech itself from 90% accurate to 99.9% accurate requires awareness of the tens of thousands of real-world complicators that never show up in laboratory testing. As Google showed off by bootstrapping their voice recognition tech via Goog-411, they know how to take megabytes / gigabytes of "dirty signal" and use it to refine their algorithms against unforeseen corner cases. 3) Releasing new assistant features can be done atop existing collected data, without requiring months of lag for enough state to build up for the feature to be useful.
- api 7y ago
  All of these are addressable engineering challenges if privacy were a priority. It's just that privacy is not a priority. (1) is not that hard, just subscribe to feeds or query cloud services from the device. (2) is harder but could be done by treating the devices as a cluster or sending data up only when it can't be easily recognized and storing it only long enough to be useful in training (and storing it anonymously), and as for (3) the device could store data encrypted in the cloud and that would be fine. A new or updated device just downloads and decrypts the index to its data. There are other workable approaches. I think the best way to change that is legislation. We need HIPAA-type liability laws applying to all privacy-sensitive data: recorded audio, recorded video, location, etc. Leak someone's location or a recording of audio from their house? That'll be $10,000 per person per incident. That would instantly transform privacy invasion from an asset to a liability and incentivize companies to minimize data collected and when it must be collected to provide the service minimize the length of the time it is stored. As it stands literally all economic incentives jump up and down and scream "invade the user's privacy s much as possible!" These are: (1) Data is valuable and advertisers will pay for it or pay for data-driven ad services. (2) Engineering privacy-invasive systems is easier than engineering privacy-respecting systems. This is a subset of "engineering insecure systems is easier than engineering secure systems" as privacy is a dimension of security. (3) Privacy sensitive approaches usually require more compute and storage at the edge, increasing device build cost.
  - TeMPOraL 7y ago
    Let's not forget the other important issue: aligned with privacy-invasive technology is the business incentive towards turning products into services. This is what drives most of the bad engineering in IoT products - perhaps even more than the secondary revenue stream of selling users' data. When you make your device depend on the Internet for its most basic functions, you secure yourself a recurring revenue stream. I'd like to see regulations addressing this issue directly, in addition to more strict pro-privacy regulation. For instance, I'd love to see it so that data, software and compute become independent - instead of current vertical integration. That is, I want to a) own my data, b) choose where it'll be processed, and c) have the software processing it come to the data - this way the vendor supplying software for c) won't be able to take the data and sell it or hold it for ransom.
- reaperducer 7y ago
  All of these items can be used through using test panels, or even dogfooding. There's no need to violate the privacy of millions of people.
  - yellow_postit 7y ago
    That’s not going to produce a good model. Test panels and dogfooding are both small scale and introduce biases based on who uses them. Until the gains stop coming primarily from supervised deep learning we won’t move away from the winner being whomever has the largest, most diverse dataset and compute resources.
- Enginerrrd 7y ago
  I feel like another motivation is to keep the neural net parameters, AKA the most valuable part of the speech recognition algorithm, off-device. It'd have to be huge otherwise, and could be easily duplicated then used as a starting place for a competing product.
- heavyset_go 7y ago
  "Violate the user's privacy in order to build a product" is lazy engineering. There are plenty of avenues to collect the data needed at that scale, it's just that none of them are as cheap as simply spying on users.
  - shadowgovt 7y ago
    Lazy engineering gets to market first, and if most users don't find it harmful, the company wins?
    - heavyset_go 7y ago
      Yes, spying on users is a cheap way to get results. The fact that incentives exist to violate user privacy is not an excuse to violate user privacy.
      - shadowgovt 7y ago
        Giant piles of cash can buy many excuses. ;)
    - TeMPOraL 7y ago
      Lazy, half-assed, corner-cutting everything gets to market first. This is a problem. It's also why markets accrue regulations over time.
      - shadowgovt 7y ago
        I agree, and it's likely that's the most correct answer if people want to see the market change. The second most correct answer is to just opt-out if you feel spied on and get to feel very smug watching people who didn't opt-out get burned by their choices. Honestly, I recommend that solution unless one encounters a situation where opt-out is impossible.
      - api 7y ago
        We seem to have no good solution to this. Capitalism/cowboy development: ship crap fast, break stuff, accrue regulations. Socialism/planned economy: have meetings to plan meetings to discuss requirements forever and nothing ever happens. While both were government funded, the "cowboy" approach of ArpaNet and IP vs the OSI network protocol consortia are a great case study.
Buge 7y ago
> Why does the device have to record and send everything to the cloud to provide this experience? It's everything after the hotword, not everything.
- RhodesianHunter 7y ago
  You're getting downvoted, but this is literally baked into the hardware.
  - TeMPOraL 7y ago
    Except when it isn't, or another "convenient" bug makes everything a hot word, turning the device into an always-recording Internet stream.
    - Buge 7y ago
      A "bug" in your phone could do the same.

mojuba 7y ago

I think it looks comfortable in the Star Trek universe because it is implied that voice activated stuff is controlled by a super-AI. That is, the AI won't give humans access to the audio. Which in turn means we are going to have a super-AI that can restrict our rights and itself have access to vast amounts of information about us. I.e. Star Trek is a society essentially controlled by the AI, as it will be above us.

shadowgovt 7y ago
It looks comfortable in the Star Trek universe because everyone trusts Starfleet and the ship's computer. I mean, if you don't trust the computer to look after your interests with regards to your voice interactions, why are you trusting it when it also controls life support and airlocks? ;)
- sdenton4 7y ago
  Another classic SF example of a voice interface: "I'm sorry, Dave..."
  - K0SM0S 7y ago
    But what's an AI, what's 'artificial', when does something become so if it's made by/of 'natural' things? Which reminds me of BSG, they have airlocks too.
pixelbath 7y ago
I disagree. Given it takes place hundreds of years into our future, I'd assume speech recognition is just a "solved problem" and not something to worry about. Plus, there's also the Federation -> navy comparison. If speech recognition is mandated in Federation vessels, why would the crew worry about who's listening?
api 7y ago
I think it looks comfortable in the Star Trek universe because Star Trek portrays a society that is not run by mobsters, warlords, delusional ideologues, and sociopaths. We are still very much savages. Clinical psychopaths and delusional narcissists run our institutions and major decisions are made on the basis of fanatical superstitious ideologies and mindless herd behavior. In short I won't put an Internet connected microphone in my house and I'm very wary of privacy leakage in general because I don't trust the people who run our world (including both governments and corporations). I don't think this is irrational. Glancing around it seems like a decent number of world government and corporate leaders belong in prisons and mental institutions, not powerful offices.

TeMPOraL 7y ago

It works in Star Trek, because as 'api says, their society is sane. United Federation of Planets is an utopia, where people don't try to abuse and profit off each other. For a proper Star Trek comparison, imagine how comfortable Federation citizens would be with using ambient computing technology created and sold by the Ferengi, who are also known to be performing surveillance for Romulans in exchange for money. (For non-Trekkies here, Ferengi is a species of extremely greedy and exploitative capitalists; Romulans are what you'd get if you reified the Five Eyes into a species, and put them into state of cold war with the Federation.)

K0SM0S 7y ago

That parody needs to happen on Ferenginar.