6 ms·

Are there any examples so far around gross mis-use of this data? Building a large list of potential suspects and/or getting some direction for further study bas

Are there any examples so far around gross mis-use of this data? Building a large list of potential suspects and/or getting some direction for further study based on a face match sounds very useful, but sending a swat team to someone's house and ruining their lives based on a terrible match on a crappy picture is a whole other side of the scale. I understand why a company collecting this information is annoying / frustrating, but it was also inevitable, was it not? EDIT: Example, LinkedIn just got told by a court that they must allow crawling. A quick crawl of profiles and photos would yield a beginning of one of these types of databases, etc - one it's on the web, you have to assume it can be used for facial recognition. Either way I am happy that people are recognizing the import of what they post online - even if it's just a picture.

llamataboot 7y ago
I think I'd refer to it more as "terrifying" and "unpreventable without regulation" more than annoying myself.
- sailfast 7y ago
  I agree it is terrifying. Is it really preventable with regulation though? You can't really put the data back in the box.
Scoundreller 7y ago
> Building a large list of potential suspects and/or getting some direction for further study based on a face match sounds very useful, but sending a swat team to someone's house and ruining their lives based on a terrible match on a crappy picture is a whole other side of the scale. How do we know they were inputting pictures of suspects? Could have inputted people they have a beef with, ex-girlfriends/boyfriends, ex-spouses, online dating profiles, annoying non-criminal activists, citizens/politicians that threaten police budgets, etc.
sp332 7y ago
Clearview encouraged customers and potential customers to abuse their access. https://www.buzzfeednews.com/article/ryanmac/clearview-ai-cops-run-wild-facial-recognition-lawsuits https://www.buzzfeednews.com/article/ryanmac/clearview-ai-co... "Clearview has also previously instructed police to act in direct violation of the company's code of conduct, which was outlined in a blog post on Monday."
mLuby 7y ago
> Are there any examples so far around gross mis-use of this data? To flip the question: how many databases like this haven't been abused? When a corporation or government claims new powers over citizens/customers, the burden of proof falls on them to show how they can't/won't abuse it, rather than on the little people to show that it's been abused.
- sailfast 7y ago
  Agreed - I was trying to get a sense of the problem to-date, mostly. It's extremely important to ensure proper controls and I agree the burden of proof falls on the government to ensure they have controls in place to prevent abuses. Perhaps I am merely surprised this creates so much outrage / surprise in this community. The folks on this website (we) have created so many platforms for sharing personal information publicly on the internet that it of course makes sense that people have been scraping it for years and it's likely stored somewhere and will be used for non-original purposes at some point because of course. As you say, how many data sets have not been abused? Users have published this data themselves, on purpose, on web platforms meant for sharing. I don't think this can be put back in the box. I'm not really sure how you regulate it in any effective way. There are some laws on the books and some corporate penalties but I don't see how you stop this if the model works using public data short of making the data so noisy and useless that it stops being effective (which may have to be the end result)
  - mLuby 7y ago
    I think it's the same outrage as when someone uses a telephoto lens to takes pictures of the inside of your home from the street. It's all technically possible, legal(ish), and should have been anticipated by anyone who leaves their windows unblocked, but it's still a clear violation of expectations of a home and of the purpose for those windows. Difference is that we've had hundreds of years to reach consensus on home windows, but barely a decade for social media. Laws are not keeping up with society's expectations, which in turn aren't keeping up with technology.
testvox 7y ago
Yes Clearview has been abused as specified TFA. Also Clearview itself has been found breaking the law repeatedly. https://techcrunch.com/2020/02/14/class-action-suit-against-clearview-ai-cites-illinois-law-that-cost-facebook-550m/ https://techcrunch.com/2020/02/14/class-action-suit-against-...
- sailfast 7y ago
  Thanks
pdkl95 7y ago
> so far The really clever ways to abuse this kind of data haven't even been invented yet. We are still fumbling around with these technologies. Most of the currently visible examples of data misuse are "high tech"/"on the internet" variation of the scams and abuses humans have always done. Current data misuse is a minor concern. The larger concern is that these databases will probably sill exist 20/30/50 years from now when someone invents a really clever way to abuse data. While it isn't possible[1] to predicting how technology will be used in the future, if the trends in technology over the last few decades suggest that this clever new abuse of data will be difficult for the average person to understand, highly disruptive to existing institutions, and the damage will cascade across the interdependencies[2] we've been adding to everything. > Building a large list of potential suspects ... sounds very useful Increasing the number of "potential suspects" is a terrible idea, because most of those suspects are innocent. Adding people to the suspect pool lowers S/N (actually-useful-targets/"potential suspects"). The ideal (magic) tool would produce a minimal list that simply identified the actual criminal. Adding more people to that list makes it less useful. Also, see the Base Rate Fallacy[3] and the Birthday Paradox[4]. [1] https://en.wikipedia.org/wiki/Connections_%28TV_series%29#Content https://en.wikipedia.org/wiki/Connections_%28TV_series%29#Co... [2] http://geer.tinho.net/geer.blackhat.6viii14.txt http://geer.tinho.net/geer.blackhat.6viii14.txt (section 10, "Convergence") [3] https://www.schneier.com/blog/archives/2012/05/criminal_intent.html https://www.schneier.com/blog/archives/2012/05/criminal_inte... [4] https://en.wikipedia.org/wiki/Birthday_problem https://en.wikipedia.org/wiki/Birthday_problem
- sailfast 7y ago
  Thanks. This is very informative. I completely agree on future misuse, but I'm also not sure how to put any of this back in the bottle. What regulation are people proposing that would prevent compilation and sale of this kind of data that is accessible publicly on the internet? Is it the government use of the tech, or the fact that the company exists in the first place that is upsetting?
  - pdkl95 7y ago
    > What regulation are people proposing that would prevent compilation and sale of this kind of data Liability! (with the option of avoiding that liability by explicitly not allowing yourself access to content/personal-data, aka a common carrier) As Dan Geer said in my previous [2] (section 2, "Net neutrality"): Hello, Uncle Sam here. You can charge whatever you like based on the contents of what you are carrying, but you are responsible for that content if it is hurtful; inspecting brings with it a responsibility for what you learn. -or- You can enjoy common carrier protections at all times, but you can neither inspect nor act on the contents of what you are carrying and can only charge for carriage itself. Bits are bits. Choose wisely. No refunds or exchanges at this window. While he was talking about ISP spying, the general principle can be adapted easily. Inspection (or building databases of the results of other people's inspections) must be tied to liability for the problems and negative externalities produced by that inspection (or database). Obviously this is a high-level description that ignores the messy details that are important in any actual plan. There is room for negotiation and modification. The goal is to create a situation that disincentives creating tomorrow's problems, just like we do for other types of hazardous technology. Data and spyware needs to be seen as toxic that requires special handling, storage, and disposal procedures. > This is very informative. Dan Geer's keynote - "Cybersecurity as Realpolitik" (video: [5], transcript: previous [2]) - is incredibly informative and should be mandatory viewing/reading for anyone interesting in trying to create the "least worst" Grim Meathook Future that technology is pushing us towards. I also recommend Dan Geer BSides DC 2018 keynote[6][7] as an addendum/update to "Cybersecurity as Realpolitik". In both talks he provides a very concise description of the core problems that are defining our future. We have to soon choose what we want to happen when stolen data is, for example, not just exposed but also put on a blockchain from which it cannot be erased. Put differently, assured data deletion is far harder than permanent data retention, yet many civilized goals, including but hardly limited to a right to be forgotten, require the sealing of records or their outright destruction. Which do we give up, the slick usefulness of immutability or information crime being unmitigatable? What does consent of the governed mean when a technology trumps a Court Order? [5] https://www.youtube.com/watch?v=nT-TGvYOBpI https://www.youtube.com/watch?v=nT-TGvYOBpI [6] https://www.youtube.com/watch?v=gbDEbfijxNY https://www.youtube.com/watch?v=gbDEbfijxNY [7] http://www.bsidesdc.org/history/geer.html http://www.bsidesdc.org/history/geer.html
freeflight 7y ago
> Building a large list of potential suspects and/or getting some direction for further study based on a face match sounds very useful Useful and sensible can be two very different things. Total surveillance, down to every private space, could be useful to prevent crimes and save lives in accidents, that still doesn't make it sensible.

llamataboot 7y ago

I think I'd refer to it more as "terrifying" and "unpreventable without regulation" more than annoying myself.

sailfast 7y ago
I agree it is terrifying. Is it really preventable with regulation though? You can't really put the data back in the box.

Scoundreller 7y ago

> Building a large list of potential suspects and/or getting some direction for further study based on a face match sounds very useful, but sending a swat team to someone's house and ruining their lives based on a terrible match on a crappy picture is a whole other side of the scale. How do we know they were inputting pictures of suspects? Could have inputted people they have a beef with, ex-girlfriends/boyfriends, ex-spouses, online dating profiles, annoying non-criminal activists, citizens/politicians that threaten police budgets, etc.

sp332 7y ago

Clearview encouraged customers and potential customers to abuse their access. https://www.buzzfeednews.com/article/ryanmac/clearview-ai-cops-run-wild-facial-recognition-lawsuits https://www.buzzfeednews.com/article/ryanmac/clearview-ai-co... "Clearview has also previously instructed police to act in direct violation of the company's code of conduct, which was outlined in a blog post on Monday."

mLuby 7y ago

> Are there any examples so far around gross mis-use of this data? To flip the question: how many databases like this haven't been abused? When a corporation or government claims new powers over citizens/customers, the burden of proof falls on them to show how they can't/won't abuse it, rather than on the little people to show that it's been abused.

sailfast 7y ago
Agreed - I was trying to get a sense of the problem to-date, mostly. It's extremely important to ensure proper controls and I agree the burden of proof falls on the government to ensure they have controls in place to prevent abuses. Perhaps I am merely surprised this creates so much outrage / surprise in this community. The folks on this website (we) have created so many platforms for sharing personal information publicly on the internet that it of course makes sense that people have been scraping it for years and it's likely stored somewhere and will be used for non-original purposes at some point because of course. As you say, how many data sets have not been abused? Users have published this data themselves, on purpose, on web platforms meant for sharing. I don't think this can be put back in the box. I'm not really sure how you regulate it in any effective way. There are some laws on the books and some corporate penalties but I don't see how you stop this if the model works using public data short of making the data so noisy and useless that it stops being effective (which may have to be the end result)
- mLuby 7y ago
  I think it's the same outrage as when someone uses a telephoto lens to takes pictures of the inside of your home from the street. It's all technically possible, legal(ish), and should have been anticipated by anyone who leaves their windows unblocked, but it's still a clear violation of expectations of a home and of the purpose for those windows. Difference is that we've had hundreds of years to reach consensus on home windows, but barely a decade for social media. Laws are not keeping up with society's expectations, which in turn aren't keeping up with technology.

testvox 7y ago

Yes Clearview has been abused as specified TFA. Also Clearview itself has been found breaking the law repeatedly. https://techcrunch.com/2020/02/14/class-action-suit-against-clearview-ai-cites-illinois-law-that-cost-facebook-550m/ https://techcrunch.com/2020/02/14/class-action-suit-against-...

sailfast 7y ago
Thanks

pdkl95 7y ago

> so far The really clever ways to abuse this kind of data haven't even been invented yet. We are still fumbling around with these technologies. Most of the currently visible examples of data misuse are "high tech"/"on the internet" variation of the scams and abuses humans have always done. Current data misuse is a minor concern. The larger concern is that these databases will probably sill exist 20/30/50 years from now when someone invents a really clever way to abuse data. While it isn't possible[1] to predicting how technology will be used in the future, if the trends in technology over the last few decades suggest that this clever new abuse of data will be difficult for the average person to understand, highly disruptive to existing institutions, and the damage will cascade across the interdependencies[2] we've been adding to everything. > Building a large list of potential suspects ... sounds very useful Increasing the number of "potential suspects" is a terrible idea, because most of those suspects are innocent. Adding people to the suspect pool lowers S/N (actually-useful-targets/"potential suspects"). The ideal (magic) tool would produce a minimal list that simply identified the actual criminal. Adding more people to that list makes it less useful. Also, see the Base Rate Fallacy[3] and the Birthday Paradox[4]. [1] https://en.wikipedia.org/wiki/Connections_%28TV_series%29#Content https://en.wikipedia.org/wiki/Connections_%28TV_series%29#Co... [2] http://geer.tinho.net/geer.blackhat.6viii14.txt http://geer.tinho.net/geer.blackhat.6viii14.txt (section 10, "Convergence") [3] https://www.schneier.com/blog/archives/2012/05/criminal_intent.html https://www.schneier.com/blog/archives/2012/05/criminal_inte... [4] https://en.wikipedia.org/wiki/Birthday_problem https://en.wikipedia.org/wiki/Birthday_problem

sailfast 7y ago
Thanks. This is very informative. I completely agree on future misuse, but I'm also not sure how to put any of this back in the bottle. What regulation are people proposing that would prevent compilation and sale of this kind of data that is accessible publicly on the internet? Is it the government use of the tech, or the fact that the company exists in the first place that is upsetting?
- pdkl95 7y ago
  > What regulation are people proposing that would prevent compilation and sale of this kind of data Liability! (with the option of avoiding that liability by explicitly not allowing yourself access to content/personal-data, aka a common carrier) As Dan Geer said in my previous [2] (section 2, "Net neutrality"): Hello, Uncle Sam here. You can charge whatever you like based on the contents of what you are carrying, but you are responsible for that content if it is hurtful; inspecting brings with it a responsibility for what you learn. -or- You can enjoy common carrier protections at all times, but you can neither inspect nor act on the contents of what you are carrying and can only charge for carriage itself. Bits are bits. Choose wisely. No refunds or exchanges at this window. While he was talking about ISP spying, the general principle can be adapted easily. Inspection (or building databases of the results of other people's inspections) must be tied to liability for the problems and negative externalities produced by that inspection (or database). Obviously this is a high-level description that ignores the messy details that are important in any actual plan. There is room for negotiation and modification. The goal is to create a situation that disincentives creating tomorrow's problems, just like we do for other types of hazardous technology. Data and spyware needs to be seen as toxic that requires special handling, storage, and disposal procedures. > This is very informative. Dan Geer's keynote - "Cybersecurity as Realpolitik" (video: [5], transcript: previous [2]) - is incredibly informative and should be mandatory viewing/reading for anyone interesting in trying to create the "least worst" Grim Meathook Future that technology is pushing us towards. I also recommend Dan Geer BSides DC 2018 keynote[6][7] as an addendum/update to "Cybersecurity as Realpolitik". In both talks he provides a very concise description of the core problems that are defining our future. We have to soon choose what we want to happen when stolen data is, for example, not just exposed but also put on a blockchain from which it cannot be erased. Put differently, assured data deletion is far harder than permanent data retention, yet many civilized goals, including but hardly limited to a right to be forgotten, require the sealing of records or their outright destruction. Which do we give up, the slick usefulness of immutability or information crime being unmitigatable? What does consent of the governed mean when a technology trumps a Court Order? [5] https://www.youtube.com/watch?v=nT-TGvYOBpI https://www.youtube.com/watch?v=nT-TGvYOBpI [6] https://www.youtube.com/watch?v=gbDEbfijxNY https://www.youtube.com/watch?v=gbDEbfijxNY [7] http://www.bsidesdc.org/history/geer.html http://www.bsidesdc.org/history/geer.html

freeflight 7y ago

> Building a large list of potential suspects and/or getting some direction for further study based on a face match sounds very useful Useful and sensible can be two very different things. Total surveillance, down to every private space, could be useful to prevent crimes and save lives in accidents, that still doesn't make it sensible.