9 ms·

Spoiler warning. Article punchline ahead. "The whole point of a dimension reduction model is to mathematically represent the data in simpler form. It’s as if C

by etiam 9y ago

Spoiler warning. Article punchline ahead. "The whole point of a dimension reduction model is to mathematically represent the data in simpler form. It’s as if Cambridge Analytica took a very high-resolution photograph, resized it to be smaller, and then deleted the original. The photo still exists — and as long as Cambridge Analytica’s models exist, the data effectively does too." That's an eloquent piece of explanation of a very important point. And apropos the discussion about privacy legislation, it's also going to be a very interesting point. Will the Cambridge Analyticas of the world be able to claim they have held on to no personal data, when strictly speaking the raw data has indeed been deleted after being used to create a derivative work that can for all important purposes be used to recreate the original? Assuming I find out I'm being profiled and demand to have my data removed, will society grant me rights to have derivative forms removed or adjusted too? I'm somewhat pessimistic that legal hairsplitting about matters like these will make enforcement very difficult.

autokad 9y ago
> "will society grant me rights to have derivative forms removed or adjusted too? " I am in favor of no. Imagine I build a gender classification model off public tweets, and then you later delete your twitter account and demand my model not be used because it was trained off 'your data'. I am in the camp that so long as the data isn't traceable back to you specifically, then don't put any information out there you are not OK with sticking around.
salty_biscuits 9y ago
I guess the question is what are you trying to protect? The model is fundamentally lossy as it is a rank reduction method so your original data is gone (i.e. no one would be able to accuse you of liking a particular controversial post, just that you are likely to like that post). So it sort of has the differential privacy thing going on. I guess it is another question as to if such models should be built at all. I think the fidelity of the models will answer that in time, if they work really well it is scary, if they are poor models they will cease to be used. I suspect that it will be in the middle and highly sensitive to the quality of the original data and the quality of the implementation like all ml applications.
darawk 9y ago
> when strictly speaking the raw data has indeed been deleted after being used to create a derivative work that can for all important purposes be used to recreate the original? To be precise, you almost certainly cannot use this data to recreate anything remotely resembling the original dataset. This type of dimensionality reduction would throw away enormous volumes of data. There is no meaningful sense in which you can reconstruct the data from it. What they have done is distill some insights about people from this data. It's arguable whether they should be allowed to keep those insights, but there's no privacy risk there really. It's honestly kind of disingenuous to describe dimensionality reduction in the way that they do here. It is like reducing the resolution of a photo, but it'd best be described as reducing that resolution to say, the 20 most representative pixels. There's no real sense in which the photo still exists.
- anf 9y ago
  > It's arguable whether they should be allowed to keep those insights, but there's no privacy risk there really. So if Google has distilled someone's emails over the years into "closeted homosexual with a deeply repressed leather fetish", that's not an invasion of their privacy as long as they throw away the source materials?
  - spydum 9y ago
    As long as they retain no data which could specifically identify the original person, yes. There is nothing wrong with building segmentation models as long as they aren't specific enough to identify a specific person. My concern would be, how granular is too granular? What if we added "and live in zip code 12355 and is registered Green Party"? This now gets eerily specific, and might be sufficient to identify an individual.
    - anf 9y ago
      Why would they ever discard that? Why would there be a granularity where ML suddenly stops working? Why would you even stop at one model per person, instead of one model per mood, or modes of thought at different stress points?
pbalau 9y ago
If I tell you I like furry porn, is there a way for me to make sure you forget that? This has lots of implications, many of them placing the "blame" on me for telling you this.
- flippy2 9y ago
  That’s strangely specific.... Hmmm
- etiam 9y ago
  I think that's a fair point, but I question if it's an apt comparison to the situation with some big players of the surveillance economy. Let me make that a bit more convoluted for you: Let's say, very hypothetically, that you, like most of the general population suck at things technological and just switch off mentally when someone mentions phrases like "social graph" or "Javascript", and I'm a semi-psychopath who plans to make a fortune by scamming dumb fucks like you. I take on my mask of sanity and most endearing nerd T-shirt and appear at your door to give you a FREE robot servant. Except to keep it FREE the robot servant is going to pause what it's doing sometimes and whisper subliminal messages to you from my sponsors. But you're not afraid of stuff like that are you? And all you have to do is sign my brick of legal documents in complexified Legalese, which of course you don't have mental stamina to read through. "But, hey", you think "people are nice and trustworthy and if there was something really bad going on here that would be illegal and punished, and besides what's the worst thing that could happen", and I get your signature and you got your FREE robot. If you had bothered to learn complexified Legalese and do your reading you'd have noticed you also just approved that the robot spends it's spare computation cycles surreptitiously watching you and getting to know you, and one of the many things it does is it glances over your shoulder when you get your porn fix, and in collaboration with my team of highly trained robot masters it concludes not only that you're into furry porn but also which particular furries really push your buttons. We catalogue this away for future use. Years later, in spite of your technical ineptitude, but maybe let's say because you're really a good people person, you've risen to become the highly respected mayor of your town. Then a business partner of mine who used to work covert operations over at CI6 but now has switched to lucrative private contracting, comes to me asking for the files collected from your robot for a influence gig he's taken on from Toxico, and I sell them to him the data for a suitably juicy sum for a man of you stature. Ex-CI6-guy studies the file with interest and uses it to select two skilled furries he can tell you'll be incapable of resisting and sends them to cross your path at a representation dinner. They very convincingly persuade you to have them over the next weekend your wife's away, and it's WILD(!). And of course "your" robot is carefully documenting the whole thing, which I also sell to my partner, for an additional cut of the profits from the Toxico job. While you still wrestle with your conscience about whether last weekend was really a Good Thing, someone appears at your office door to propose you use your influence to switch the city energy supply over to a 90-year contract on Toxico's patented owl-burning power plant (with levels of carcinogen emissions they'll never be able to get us for!). Incidentally that someone at your office door has probably also been chosen according to your robot file to be a kind of person you'd have a harder-than-usual time to say no to for one reason or the other, but in the end you still refuse because despite some personal weaknesses you're a decent man, who values and protects your town, it's people and it's environmental surroundings. And then you're informed that someone may have videos you'd really rather not become public. Unless you agree to the Toxico proposal and do so generously, your comfy little life may meet with a sudden and radical change of fortune. So who's to "blame" in this scenario? I'm sure there's a point where ignorance should be illegal, but I think generally society looks with some leniency on getting deceived. And winding back to the early parts, what does this do for your moral rights to have me remove your data? It was right there on page 200 in crisp, clear letters that you allowed me to watch your porn surfing habits and share that data and derived works with selected partners. You agreed to this! We haven't done anything that we're not allowed to do according to the contract. ...
jobigoud 9y ago
This analogy immediately reminds me that with enough high res / low res pairs we can rebuild a high resolution image from its low res version with fairly good results. Wonder if the same could be done here.
mercer 9y ago
To take a completely different approach in terms of 'derivative works': Say we have a bunch of profile images, and then describe them in text. "Blonde, caucasian, large nose, curls, receding hairline, strong jaw, big ears", or perhaps even more specific stuff like "has a mole on the left cheek at the same height as the right earlobe" and "right nostril is larger than the left" and "dimple in chink". Based on a description like this, we could identify an individual in probably a short paragraph. Nonetheless, on the data side, this is a lot less information than is represented by the raw pixels. When it comes to the topic of CA's tools, and 'psychosocial' targeting, we can't separate the broader context and the way in which one single term can encode tons of data ("looks like George Clooney with a bigger forehead"). I'd argue the same princple applies to political views, and personality.
deleted 9y ago
[deleted]
kurthr 9y ago
Based on that description I'd say it's closer to taking a RAW format and converting it into a JPG, which contains most of the (human visual) relevant data from a DCT while removing the remaining "noise"... which is a bit different from simply resizing (eg subsampling). What I would find a bit satisfying about this description is that you probably wouldn't get away with claiming that a JPG of an image was un-copyrighted and only a BMP (of the same resolution) was. To be fair, it does depend on the amount of compression before it is not recognizable, but if you can still squint and see the Mona Lisa (when you also have her phone #)... have you not violated her privacy?

by etiam 9y ago

autokad 9y ago

> "will society grant me rights to have derivative forms removed or adjusted too? " I am in favor of no. Imagine I build a gender classification model off public tweets, and then you later delete your twitter account and demand my model not be used because it was trained off 'your data'. I am in the camp that so long as the data isn't traceable back to you specifically, then don't put any information out there you are not OK with sticking around.

salty_biscuits 9y ago

I guess the question is what are you trying to protect? The model is fundamentally lossy as it is a rank reduction method so your original data is gone (i.e. no one would be able to accuse you of liking a particular controversial post, just that you are likely to like that post). So it sort of has the differential privacy thing going on. I guess it is another question as to if such models should be built at all. I think the fidelity of the models will answer that in time, if they work really well it is scary, if they are poor models they will cease to be used. I suspect that it will be in the middle and highly sensitive to the quality of the original data and the quality of the implementation like all ml applications.

darawk 9y ago

> when strictly speaking the raw data has indeed been deleted after being used to create a derivative work that can for all important purposes be used to recreate the original? To be precise, you almost certainly cannot use this data to recreate anything remotely resembling the original dataset. This type of dimensionality reduction would throw away enormous volumes of data. There is no meaningful sense in which you can reconstruct the data from it. What they have done is distill some insights about people from this data. It's arguable whether they should be allowed to keep those insights, but there's no privacy risk there really. It's honestly kind of disingenuous to describe dimensionality reduction in the way that they do here. It is like reducing the resolution of a photo, but it'd best be described as reducing that resolution to say, the 20 most representative pixels. There's no real sense in which the photo still exists.

anf 9y ago
> It's arguable whether they should be allowed to keep those insights, but there's no privacy risk there really. So if Google has distilled someone's emails over the years into "closeted homosexual with a deeply repressed leather fetish", that's not an invasion of their privacy as long as they throw away the source materials?
- spydum 9y ago
  As long as they retain no data which could specifically identify the original person, yes. There is nothing wrong with building segmentation models as long as they aren't specific enough to identify a specific person. My concern would be, how granular is too granular? What if we added "and live in zip code 12355 and is registered Green Party"? This now gets eerily specific, and might be sufficient to identify an individual.
  - anf 9y ago
    Why would they ever discard that? Why would there be a granularity where ML suddenly stops working? Why would you even stop at one model per person, instead of one model per mood, or modes of thought at different stress points?

pbalau 9y ago

If I tell you I like furry porn, is there a way for me to make sure you forget that? This has lots of implications, many of them placing the "blame" on me for telling you this.

flippy2 9y ago
That’s strangely specific.... Hmmm
etiam 9y ago
I think that's a fair point, but I question if it's an apt comparison to the situation with some big players of the surveillance economy. Let me make that a bit more convoluted for you: Let's say, very hypothetically, that you, like most of the general population suck at things technological and just switch off mentally when someone mentions phrases like "social graph" or "Javascript", and I'm a semi-psychopath who plans to make a fortune by scamming dumb fucks like you. I take on my mask of sanity and most endearing nerd T-shirt and appear at your door to give you a FREE robot servant. Except to keep it FREE the robot servant is going to pause what it's doing sometimes and whisper subliminal messages to you from my sponsors. But you're not afraid of stuff like that are you? And all you have to do is sign my brick of legal documents in complexified Legalese, which of course you don't have mental stamina to read through. "But, hey", you think "people are nice and trustworthy and if there was something really bad going on here that would be illegal and punished, and besides what's the worst thing that could happen", and I get your signature and you got your FREE robot. If you had bothered to learn complexified Legalese and do your reading you'd have noticed you also just approved that the robot spends it's spare computation cycles surreptitiously watching you and getting to know you, and one of the many things it does is it glances over your shoulder when you get your porn fix, and in collaboration with my team of highly trained robot masters it concludes not only that you're into furry porn but also which particular furries really push your buttons. We catalogue this away for future use. Years later, in spite of your technical ineptitude, but maybe let's say because you're really a good people person, you've risen to become the highly respected mayor of your town. Then a business partner of mine who used to work covert operations over at CI6 but now has switched to lucrative private contracting, comes to me asking for the files collected from your robot for a influence gig he's taken on from Toxico, and I sell them to him the data for a suitably juicy sum for a man of you stature. Ex-CI6-guy studies the file with interest and uses it to select two skilled furries he can tell you'll be incapable of resisting and sends them to cross your path at a representation dinner. They very convincingly persuade you to have them over the next weekend your wife's away, and it's WILD(!). And of course "your" robot is carefully documenting the whole thing, which I also sell to my partner, for an additional cut of the profits from the Toxico job. While you still wrestle with your conscience about whether last weekend was really a Good Thing, someone appears at your office door to propose you use your influence to switch the city energy supply over to a 90-year contract on Toxico's patented owl-burning power plant (with levels of carcinogen emissions they'll never be able to get us for!). Incidentally that someone at your office door has probably also been chosen according to your robot file to be a kind of person you'd have a harder-than-usual time to say no to for one reason or the other, but in the end you still refuse because despite some personal weaknesses you're a decent man, who values and protects your town, it's people and it's environmental surroundings. And then you're informed that someone may have videos you'd really rather not become public. Unless you agree to the Toxico proposal and do so generously, your comfy little life may meet with a sudden and radical change of fortune. So who's to "blame" in this scenario? I'm sure there's a point where ignorance should be illegal, but I think generally society looks with some leniency on getting deceived. And winding back to the early parts, what does this do for your moral rights to have me remove your data? It was right there on page 200 in crisp, clear letters that you allowed me to watch your porn surfing habits and share that data and derived works with selected partners. You agreed to this! We haven't done anything that we're not allowed to do according to the contract. ...

jobigoud 9y ago

This analogy immediately reminds me that with enough high res / low res pairs we can rebuild a high resolution image from its low res version with fairly good results. Wonder if the same could be done here.

mercer 9y ago

To take a completely different approach in terms of 'derivative works': Say we have a bunch of profile images, and then describe them in text. "Blonde, caucasian, large nose, curls, receding hairline, strong jaw, big ears", or perhaps even more specific stuff like "has a mole on the left cheek at the same height as the right earlobe" and "right nostril is larger than the left" and "dimple in chink". Based on a description like this, we could identify an individual in probably a short paragraph. Nonetheless, on the data side, this is a lot less information than is represented by the raw pixels. When it comes to the topic of CA's tools, and 'psychosocial' targeting, we can't separate the broader context and the way in which one single term can encode tons of data ("looks like George Clooney with a bigger forehead"). I'd argue the same princple applies to political views, and personality.

deleted 9y ago

[deleted]

kurthr 9y ago

Based on that description I'd say it's closer to taking a RAW format and converting it into a JPG, which contains most of the (human visual) relevant data from a DCT while removing the remaining "noise"... which is a bit different from simply resizing (eg subsampling). What I would find a bit satisfying about this description is that you probably wouldn't get away with claiming that a JPG of an image was un-copyrighted and only a BMP (of the same resolution) was. To be fair, it does depend on the amount of compression before it is not recognizable, but if you can still squint and see the Mona Lisa (when you also have her phone #)... have you not violated her privacy?