5 ms·

I have an open ended question-- mostly born out of ignorance; But why is this a bad thing? Isn't an artificial assistant that not only knows and understands us

I have an open ended question-- mostly born out of ignorance; But why is this a bad thing? Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful? In the process, sure they'll collect your info for better advertising, but short of Totalitarian Surveillance or Data Breach Concerns (The former is a bit of a reach if you live in the west, and they can survey you anyway if they really want to, the latter also seems somewhat unlikely)-- whats the issue here? Genuinely asking because I'm trying to understand.

danso 10y ago
Not an ignorant question, but a reasonable one. I think the skeptics believe that society overall does not yet understand the tradeoff. And I agree with them there. I think on a deeper level, people are creeped out that data paints such a deterministic picture of our lives. Sure, I leave home the same time every day, and I leave work about the same time every day. That Google knows where I live and work without me ever explicitly saying shouldn't be a surprise, but not everyone enjoys seeing how their daily lives are so easily circumscribed, even with just passively-collected data.
jonnathanson 10y ago
There are three levels of discomfort some people feel with this situation: 1. Concern that a single, third-party entity (Google, in this case) might peer into every aspect of our lives, and/or reverse-engineer an exhaustive catalog of our entire lives, by virtue of data collection. 2. Concern that many consumers will unwittingly opt into such control, unaware of the privacy they're relinquishing, and unable to make informed decisions about the possible applications and consequences of the tradeoff. 3. Concern that the custodian of all this personal data (Google) might use, sell, transmit, or turn over the data in ways we had not anticipated or believed we'd consented to. Personally speaking, I understand these concerns but also understand the potential upside. I'm not 100% sure where I stand just yet. The aforementioned bullet points are presented without editorial comment; just trying my best to articulate what I believe to be the crux of people's concerns here.
falcolas 10y ago
> But why is this a bad thing? Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large. Totalitarian Surveillance is here. In the west. Secure document releases aside, it's too easy to do to imagine a state actor not doing it. Data breaches of differing severities occur every day, at nearly every company. I would have thought Yahoo was big enough and smart enough to avoid it; but no. Not Yahoo, not Sony, not security contractors, not credit bureaus, not Apple (a'la celebrity photo leaks), not Google (stories abound of individual GMail accounts being hacked).
- chillingeffect 10y ago
  I agree with you. To help convince people, I realize that we often imagine benevolent leadership, so it helps to give an example such as, "Imagine if you were a Muslim or illegal and Donald Trump were elected president. What could he do with your data?" E.g. find you, search your residence based on your purchasing and travel habits and send you home. E.g. Wakes up at 5:30 am, travels to a construction site, lives in a house with a large number of people -> signals possible immigrant. Or this: Detecting Islamic Calendar Effects on U.S. Meat Consumption: Is the Muslim Population Larger than Widely Assumed? https://mpra.ub.uni-muenchen.de/41554/ https://mpra.ub.uni-muenchen.de/41554/ We have to think about data not just in terms of our relative safety, but in terms of what could happen in adverse circumstances. And not even just in terms of our own government, but foreign governments.
- zardeh 10y ago
  >Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large. (Have worked at google in the past, may in the future, am not currently). You say this as though anyone at Google (or Microsoft or whatever) can go in and search for 'falcolas' and look through your GPS history. I'm honestly not sure if there is a single individual at the company who had that power. I honestly think that the best thing Google could to is publicize their internal training and documents on personal information, because the regulations and such made me a lot more comfortable with giving Google the sort of amorphous entity my data, because no person is going to be looking at that data. >, not Google (stories abound of individual GMail accounts being hacked). One of these is not like the others, unless you're talking about something I'm not aware of. Hacking an individual GMail account requires guessing/taking someone's password, which is not an attack on Google's infrastructure (Unlike the yahoo, sony, apple, etc. examples), its an attack on a bad password.
probably_wrong 10y ago
The way you phrased the question sounds to me like "what are the shortcomings, leaving aside these terrible shortcomings?" Having said that: Google is not in the business of making your life easier, but in the business of selling you ads. The data that Google collects about you is incredibly powerful, allowing them to go from a "simple" manipulation to sell you stuff you wouldn't otherwise buy, to full-scale blackmailing you if they see it fit (not saying that this is happening, but if they wanted to, who would stop them?). It's putting too much power in the hands of a single, amoral entity (like all corporations). That's not good.
- euyyn 10y ago
  > but if they wanted to, who would stop them? The law, and the economic interest of all the rich shareholders that care about the company's reputation.
deleted 10y ago
[deleted]
barrkel 10y ago
I wouldn't mind an artificial assistant. I don't want a company that employs tens of thousands of people, along with a government in a foreign country, along with all the governments on the data route in between, and their employees, civil servants and assorted snoopers of all shades, to have access to the artificial assistant's communications and thoughts relating to me. All these organisations are made out of people. People with power are inherently untrustworthy; they need enforcement mechanisms to be kept in line, and enforcement mechanisms need to be activated every now and then to stay in working order. That is, occasional abuses are required to keep abuse in line. The thin blue line wavers like a pendulum: it's how we know it's working.
dhimes 10y ago
Part of what I fear is that Totalitarian Surveillance is only a "bit of a reach in the west" because we put such a high value on privacy (and personal liberty) that we're willing to defend it. When that goes away then the "reach" will be far easier. Edit: Interesting comment in another thread: https://news.ycombinator.com/item?id=12639530 https://news.ycombinator.com/item?id=12639530
pdkl95 10y ago
> incredibly useful Sure it would be useful. Sell the assistant as a locally-installed app that guarantees personal data never leaves the LAN and will sell. > sure they'll collect your info Only if you let them. Demand better behavior from their software and business practices. > Totalitarian Surveillance or Data Breach Concerns What you seem to be missing is that the concern isn't about today's level of surveillance or today's data breach risk. Data generally persists indefinitely once it makes its way into a database or logfile. To make a claim that these are low risk requires that at no time in the future will surveillance risk increase or data breaches become more common, ... or that the company will run into financial trouble and need to sell your data, ... or that a breach will be forced by a government (not necessarily your's or Google's), ... or that your data will be aggregated into other databases, increasing the "predictive" power and attack surface, ... or any of the other unknown ways your data could be used in the future. Humans are already known to be terrible at assessing risk, especially when there is a very large separation between the cause and effect. Smoking today giving you cancer many years later is a traditional example. We already know data breaches happen, well meaning employees make mistakes or succumb to corruption, and external powers such as governments or organized crime occasionally take away your agency. Do you really want to claim that none of these risks will ever happen? Because that's the actual wager you're making when when you use Google's products.
ams6110 10y ago
For most of history, most humans lived under tyranny or domination if not outright slavery. It's only been the past few hundred years that this mostly stopped in some places. Maybe we've turned a corner and will never go back to that. But I don't have confidence yet.
eli_gottlieb 10y ago
>Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful? No, not really. Restaurant recommendations and traffic reports are simply not that hard for me to find on Yelp or Waze myself. The "anticipation" here doesn't really help me in any material way.

danso 10y ago

Not an ignorant question, but a reasonable one. I think the skeptics believe that society overall does not yet understand the tradeoff. And I agree with them there. I think on a deeper level, people are creeped out that data paints such a deterministic picture of our lives. Sure, I leave home the same time every day, and I leave work about the same time every day. That Google knows where I live and work without me ever explicitly saying shouldn't be a surprise, but not everyone enjoys seeing how their daily lives are so easily circumscribed, even with just passively-collected data.

jonnathanson 10y ago

There are three levels of discomfort some people feel with this situation: 1. Concern that a single, third-party entity (Google, in this case) might peer into every aspect of our lives, and/or reverse-engineer an exhaustive catalog of our entire lives, by virtue of data collection. 2. Concern that many consumers will unwittingly opt into such control, unaware of the privacy they're relinquishing, and unable to make informed decisions about the possible applications and consequences of the tradeoff. 3. Concern that the custodian of all this personal data (Google) might use, sell, transmit, or turn over the data in ways we had not anticipated or believed we'd consented to. Personally speaking, I understand these concerns but also understand the potential upside. I'm not 100% sure where I stand just yet. The aforementioned bullet points are presented without editorial comment; just trying my best to articulate what I believe to be the crux of people's concerns here.

falcolas 10y ago

> But why is this a bad thing? Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large. Totalitarian Surveillance is here. In the west. Secure document releases aside, it's too easy to do to imagine a state actor not doing it. Data breaches of differing severities occur every day, at nearly every company. I would have thought Yahoo was big enough and smart enough to avoid it; but no. Not Yahoo, not Sony, not security contractors, not credit bureaus, not Apple (a'la celebrity photo leaks), not Google (stories abound of individual GMail accounts being hacked).

chillingeffect 10y ago
I agree with you. To help convince people, I realize that we often imagine benevolent leadership, so it helps to give an example such as, "Imagine if you were a Muslim or illegal and Donald Trump were elected president. What could he do with your data?" E.g. find you, search your residence based on your purchasing and travel habits and send you home. E.g. Wakes up at 5:30 am, travels to a construction site, lives in a house with a large number of people -> signals possible immigrant. Or this: Detecting Islamic Calendar Effects on U.S. Meat Consumption: Is the Muslim Population Larger than Widely Assumed? https://mpra.ub.uni-muenchen.de/41554/ https://mpra.ub.uni-muenchen.de/41554/ We have to think about data not just in terms of our relative safety, but in terms of what could happen in adverse circumstances. And not even just in terms of our own government, but foreign governments.
zardeh 10y ago
>Do you trust every single person at Google, and every single person at every third party company Google shares your data with, now and in perpetuity, to never abuse the data collected on you? Personally, my circle of trust is not that large. (Have worked at google in the past, may in the future, am not currently). You say this as though anyone at Google (or Microsoft or whatever) can go in and search for 'falcolas' and look through your GPS history. I'm honestly not sure if there is a single individual at the company who had that power. I honestly think that the best thing Google could to is publicize their internal training and documents on personal information, because the regulations and such made me a lot more comfortable with giving Google the sort of amorphous entity my data, because no person is going to be looking at that data. >, not Google (stories abound of individual GMail accounts being hacked). One of these is not like the others, unless you're talking about something I'm not aware of. Hacking an individual GMail account requires guessing/taking someone's password, which is not an attack on Google's infrastructure (Unlike the yahoo, sony, apple, etc. examples), its an attack on a bad password.

probably_wrong 10y ago

The way you phrased the question sounds to me like "what are the shortcomings, leaving aside these terrible shortcomings?" Having said that: Google is not in the business of making your life easier, but in the business of selling you ads. The data that Google collects about you is incredibly powerful, allowing them to go from a "simple" manipulation to sell you stuff you wouldn't otherwise buy, to full-scale blackmailing you if they see it fit (not saying that this is happening, but if they wanted to, who would stop them?). It's putting too much power in the hands of a single, amoral entity (like all corporations). That's not good.

euyyn 10y ago
> but if they wanted to, who would stop them? The law, and the economic interest of all the rich shareholders that care about the company's reputation.

deleted 10y ago

[deleted]

barrkel 10y ago

I wouldn't mind an artificial assistant. I don't want a company that employs tens of thousands of people, along with a government in a foreign country, along with all the governments on the data route in between, and their employees, civil servants and assorted snoopers of all shades, to have access to the artificial assistant's communications and thoughts relating to me. All these organisations are made out of people. People with power are inherently untrustworthy; they need enforcement mechanisms to be kept in line, and enforcement mechanisms need to be activated every now and then to stay in working order. That is, occasional abuses are required to keep abuse in line. The thin blue line wavers like a pendulum: it's how we know it's working.

dhimes 10y ago

Part of what I fear is that Totalitarian Surveillance is only a "bit of a reach in the west" because we put such a high value on privacy (and personal liberty) that we're willing to defend it. When that goes away then the "reach" will be far easier. Edit: Interesting comment in another thread: https://news.ycombinator.com/item?id=12639530 https://news.ycombinator.com/item?id=12639530

pdkl95 10y ago

> incredibly useful Sure it would be useful. Sell the assistant as a locally-installed app that guarantees personal data never leaves the LAN and will sell. > sure they'll collect your info Only if you let them. Demand better behavior from their software and business practices. > Totalitarian Surveillance or Data Breach Concerns What you seem to be missing is that the concern isn't about today's level of surveillance or today's data breach risk. Data generally persists indefinitely once it makes its way into a database or logfile. To make a claim that these are low risk requires that at no time in the future will surveillance risk increase or data breaches become more common, ... or that the company will run into financial trouble and need to sell your data, ... or that a breach will be forced by a government (not necessarily your's or Google's), ... or that your data will be aggregated into other databases, increasing the "predictive" power and attack surface, ... or any of the other unknown ways your data could be used in the future. Humans are already known to be terrible at assessing risk, especially when there is a very large separation between the cause and effect. Smoking today giving you cancer many years later is a traditional example. We already know data breaches happen, well meaning employees make mistakes or succumb to corruption, and external powers such as governments or organized crime occasionally take away your agency. Do you really want to claim that none of these risks will ever happen? Because that's the actual wager you're making when when you use Google's products.

ams6110 10y ago

For most of history, most humans lived under tyranny or domination if not outright slavery. It's only been the past few hundred years that this mostly stopped in some places. Maybe we've turned a corner and will never go back to that. But I don't have confidence yet.

eli_gottlieb 10y ago

>Isn't an artificial assistant that not only knows and understands us but anticipates our needs incredibly useful? No, not really. Restaurant recommendations and traffic reports are simply not that hard for me to find on Yelp or Waze myself. The "anticipation" here doesn't really help me in any material way.