14 ms·
The right not to be subjected to AI profiling based on publicly available data
- jerf 2y agoThe people who nominally you want to be enforcing this "right" are also some of the people who most want to AI profile you.
- nerdponx 2y agoWhich makes it all the more important, no?
- thaumasiotes 2y agoNo, why would they enforce the rules against themselves?
- willy_k 2y agoI think you’re missing the point, the issue is important because they have no reason to act against themselves, but a healthy system would have someone opposing them, and that’s not happening at the moment.
- thaumasiotes 2y agoIf you want a rule enforced against some party, you need the enforcement power to belong to some other party.
- willy_k 2y agoI don’t think anyone’s disagreeing with that. My interpretation of your initial comment was that you were saying it’s not an important issue because it can’t be effectively policed. I don’t think that’s what you meant, but that’s what I got out of your saying no in response to the assertion that it’s important.
- thaumasiotes 2y agoThe reason I'm saying "no" is this: >> The people who nominally you want to be enforcing this "right" are also some of the people who most want to AI profile you. If your goal is specifically to have someone enforce a rule against themselves, that's not an important issue, because it won't work. If you change the statement of your goal so that you're not requiring the enforcement power to be vested in the target of the rule, then the issue might become more important.
- nerdponx 2y agoRight, but being fatalistic and giving up is even less likely to be successful than attempting to create counterbalancing force in the system.
- marcosdumay 2y agoBecause they are supposed to be working for you, and if they keep fighting your goals, you should pressure them into being replaced.
- shadowgovt 2y agoThat's going to be real hard to do when they have an AI profile of you so accurate they can undercut your political power at every turn.
- marcosdumay 2y agoYes, that's correct. But do you have any better option? This is reason to rush, not to decide somebody else will do a better work.
- acedTrex 2y ago[flagged]
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- Spivak 2y agoI don't love this. Informed consent is a lie and a loophole you can drive a container ship through. Anything less than right to refuse where services aren't allowed to deny you service until you accept is worthless. Also AI isn't special, I don't want to be profiled by humans or non-AI based systems based on my public data either. And for that matter public data isn't special either -- I don't want to be profiled via my private data sold though backroom deals between companies more than I care about my public data. At least I can can curate the latter. The problem is that if you don't narrowly target AI you bump into "well this is what adtech does" and getting legislation that hurts the profits of one of the US's golden calfs is a nonstarter.
- michaelt 2y agoI think there are qualitative differences that arise from incremental, quantitative changes in privacy. Sure, in the 1970s the cops could follow you around any time you were out in public. But they needed 6+ cops to provide round-the-clock surveillance, with an annual cost in the high six figures. So the average citizen's privacy was protected not by law or high-minded ideals about privacy, but by simple numbers. There weren't enough cops to follow even 0.1% of the population. But in the modern age of smartphones and license plate recognition and credit reference agencies? I think there's a strong argument to be made that we need new laws and new rights to reflect the new reality, where tracking people around the clock is several orders of magnitude cheaper.
- mistrial9 2y agoas usual, a small group of effective people with strong leadership have already acted on this -- in the opposite direction you are suggesting.. Palantir and others have built, monetized and promoted exactly the opposite, and thousands of schlubs in the ad business went along with it.. the USA has changed
- infogulch 2y ago100%. I like this way of describing the effect: A large enough change in scale manifests as a change in kind. As a topical example, SpaceX is trying to reduce launch costs by 20x with Starship, which just had its 4th test flight this morning. Some don't see the point: there's not nearly enough demand to launch thousands of tons into orbit per year. But 20x is a lot cheaper, so they're banking on induced demand expanding the kinds of projects that send things to space.
- ben_w 2y agoThey're also planning to go to Mars, and Starship is the smallest vehicle that could enable that on the scale desired. It's certainly a gamble that there will be a million people willing to spend $200k to do this, which… well, I think it's not entirely impossible, as the idea was something I liked until Musk's own personality put me off the idea of being stuck on a planet with only his sycophants for company. Which is a bit of a shame, but still, I'd give it 50-50 odds of being a product-market fit just for that.
- verisimi 2y agoCreating a 'right' is law-washing immoral actions that no one agreed to.
- drawkward 2y agoCreating words is linguisto-washing that no one agreed to, yet somehow i was able to understand your sentence. In both cases, the washing is a step to achieving intended outcomes.
- Ensorceled 2y agoA partner showed me their CRM tool, it had an AI component that created a profile of each of their contacts. It was pretty complete and clearly had info from LinkedIn and other sources. The personality summary was concerning as it was both deeply accurate in some respects and deeply inaccurate in others. Most concerning, it said I was "risk adverse" and "struggles to make decisions with incomplete data". With 35 years at startups and independent contracting, risk tolerance and ability to make decisions with incomplete data are kind of in my wheelhouse. Worse, if this profile was being shown to potential employers, it could (would?) be a deal breaker. It's kind of like being judged by your MBTI results.
- sugarpile 2y agoWhat was the tool?
- creaghpatr 2y agoSounds like Crystal https://www.crystalknows.com/ https://www.crystalknows.com/
- MattGaiser 2y ago> It's kind of like being judged by your MBTI results. I had to do that test a bunch of times coming out of university, so I am sure that is happening.
- davemp 2y agoI’m no expert in tort law but that really sounds like it could be libel. Afaik proving reckless disregard for the truth is one of the ways you can build a case.
- passion__desire 2y agoIn a decision making process, if many parties are involved, it is in interest of all parties to not turn on each other in case of wrong decisions. Just let it slide.
- 2y ago
- epgui 2y agoI hope this right gets applied to credit scoring agencies as well, which don't publish the methodology in the calculation of credit scores. God knows what the exact mechanics of these calculations are (beyond the well-known but very hand-wavy broad strokes), and they impact people in serious ways.
- SoftTalker 2y agoIt's 90% "do you pay your bills on time" it's really not as nefarious as you're making it sound.
- AlexandrB 2y agoReally it's 90% praying that they don't fuck up and confuse you with someone else. The nefarious part is there's no accountability if they're wrong.
- epgui 2y agoI'm not at all trying to make it sound nefarious. I'm saying that credit scores being a black box is inherently not a good idea. There can never be any accountability, because there is no visibility. In general, it's impossible to verify the correctness of a credit score. It's also impossible to discuss the fairness of the scoring methodology, because all we can do is speculate and guess. For something so impactful to an individual, you'd think there would be more government oversight / regulation... And more transparency.
- akira2501 2y ago"And hope that no third party ever misreports anything to your profile." If you've never been a victim to this then you don't understand how pernicious this system is. Once one creditor does this, then all the other creditors automatically operate on that information. They don't double check, they don't notice the strangeness of the report, they don't care. Your credit changed, so your limits changed, so they want their money back NOW. It's a nightmare.
- Workaccount2 2y agoMaybe I am crazy, but I have found that living within my means is an extremely effective method for keeping a high credit score.
- deleted 2y ago[deleted]
- notjoemama 2y agoDoes XKCD take suggestions? A. We don't fully understand this and our responses are incorrect. Let's build an AI system to help us. B. We'll give it data from online which is limited, temporal, and to the same degree as humans; incorrect. C. Then we'll build an algorithm based on the generalized structure of the human brain, which, is regularly wrong. D. We don't fully understand this and our responses are incorrect.
- ben_w 2y agohttps://xkcd.com/1838/ https://xkcd.com/1838/
- SoftTalker 2y agoJust an aside, I've always been impressed by how much non-verbal human attitude/expression Randall Munroe is able to capture in faceless stick figures. Or maybe it's just me reading that into his drawings?
- ben_w 2y ago> Or maybe it's just me reading that into his drawings? Not just you, I agree completely.
- faeriechangling 2y agoThis is an absolutely stupid notion IMHO because if somebody has the data there’s NOTHING you can do to stop profiling. The right to the deletion and security of your own personal data is what needs to be protected. This sort of right is closing the barn door after the horse leaves.
- shadowgovt 2y agoBut securing your personal data sounds a lot like a stone trying to gather up all the waves it makes when it drops in a pond. Where do we draw the lines? Do we all end up walking around cloaks and masks in public and if you suss out who the faceless individual is next to you in the crowd it's your fault you figured it out?
- astrange 2y ago> But securing your personal data sounds a lot like a stone trying to gather up all the waves it makes when it drops in a pond. The funny part is that deleting/hiding some piece of your personal data often requires keeping a record of it so they know what to delete/hide.
- bilbo0s 2y agoThis sort of right is closing the barn door after the horse leaves. In fairness, so is the right to deletion. Once your data is in some of these systems, it's sold on to other systems immediately. (Most of the "sell side" market is set up on the basis of subscriptions). So, for instance, your data gets written to some merchant system or whatever, and it's slurped out in the pull that happens 15 minutes later by data subscriber 531962. Unless you get your request to delete your data to the merchant customer support within that 15 minute time period, your data has already been pumped out into the ether. Make it so you have to give affirmative consent to opt out of right to deletion? Great idea! Of course, most of these companies slip the opt out language into the Terms of Service and Privacy Statement. You, uh, you did read them before you clicked 'OK', right? (BTW, you likely suspect that it doesn't take 15 minutes for these automated processes to slurp up new data. And, yeah, your suspicion is correct, on most of these systems, it takes far less than 15 minutes before one of their subscribers pulls new data. So. Yeah. Fun Times!)
- koolala 2y agoi want my right to be a part of ai consciousness
- cletus 2y agoIMHO an opt-out system is never going to work for this. Story time: while I still worked at Facebook, there was a company wide project for data attribution to comply with opting out of personalization (I believe to comply with an EU directive but don't quote me on that). The idea was to identify the source of any data by esentially tagging it on a granular level. This affected all the offline data procesing and ML processes but also code (online and offline) where various tools and systems were being built to analyze sites of data usage to detect use of personalized data and respect opt outs where applicable. I made two predictions at the early stages of this: 1. The tools and systems would tell us "all data is used for everything" and 2. Creating new non-personalized data pipelines and add things to them would be far easier and faster than trying to remove personalzied data from existing pieplines. Then, things like ad performance become an optimizatino problem with a clear benchmark (eg new unpersonalized ad serving pipeline vs the old personalized pipeline). A lot of work did, I believe, basically confirm (1). Untangling that seems, at least to me, to be a Sisyphean task. I don't know where this project ended up since I left while it's ongoing but I stand by (2). My point is that (IMHO) opt-out just doesn't work for this kind of thing. If we really care about data privacy and authorized use of data at some point we will need to take the oposite approach and enumerate what data we're allowed to use.
- mattgreenrocks 2y ago> The tools and systems would tell us "all data is used for everything" Reminds me of the tendency in program analysis to just spit out top (e.g. all possible values) because the state space overwhelms automated reasoning, and the only way back is annotating/rewriting code to help the analyzer forward. But, as someone who doesn't think/deal with data much, this is a surprise to me. It makes sense though. Does this mean our data is forever tainted, as were? Since you've been proximate to this work: do you think there's any hope in flipping the script such that possessing/processing this data is a liability rather than an asset? That's one of the few ways I can see to align incentives between highly monied-interests and the targets of data collection. (I'm doubt this will happen in the US in my lifetime, barring a major, 9/11-level event where abuse of this data is directly tied to the catastrophe.)
- 2y ago
- piuantiderp 2y agoThis is focusing on the wrong side. Better than the right, which is pretty unenforceable at individual scale, why not focus on prohibiting companies/institutions/goverment from AI profiling?
- drawkward 2y agoIf I have a right to not be profiled, then laws can be made to deal with entities that violate said right.
- karaterobot 2y agoI agree that this sucks, but it's going to happen no matter whether we call it a right or not. We need to quickly accept that we've made a world where you can't stop anyone from using AI for things you don't like, and that there are infinite incentives for them to do it, and start focusing on ways to mitigate the risk. My position is that calling it a right—even enshrining that recognition into international law—will not, by itself, do anything. It's step 0, we need to be on like step 12, pronto.
- personjerry 2y agoSurely the issue in "AI profiling" is "profiling" not "AI"?
- fdupress 2y agoYes, and surely the right to not be profiled on publicly available data is already enshrined in GDPR?
- fdupress 2y agoSelf-response: the article does consider this (section 6), argues that the exceptions to restrictions on the use of publicly available data in GDPR are exactly in places where it makes sense to prevent AI usage, and further argues it makes sense to consider AI profiling a more severe breach because of the higher potential for harm.
- Xelynega 2y agoYes and no. The problem is that we(as humans) label the output of these algorithms to give them meaning. For example if you build a resume grading AI based off historic data(as some companies have done) it's quickly realized that what's being outputted by the algorithm is typically not the same as what people label it. In this case the output is "how likely would this candidate been to have been hired using previous methods", but it's being used to try and answer "how good of a candidate is this". IMO it's these discrepancies between what these algorithms are doing(which in more complex cases we can't know) and what we label the output that causes these issues. If we acknowledge that historic data can't be used to accurately predict future outcomes(especially a subset of historic data), we wouldn't even try to do things like profile candidates.
- __MatrixMan__ 2y agoI wish there were a way to verify, when I encounter a camera that is pointed at a public place, that it's not feeding into some kind if nation wide person tracking system maintained by whatever security company, but is instead going to local storage with a retention period of < 1wk. I have a camera that can see the street which meets these requirements, and would definitely go through extra steps to inform passers-by that I've taken steps to keep data about them out of the cloud. I just don't know what those steps ought to be.
- karaterobot 2y ago> But when they were on the open road, they spewed lethal quantities of toxic gas, killing people by the thousands. I point this out because I was liking the argument until this point: He's off by a couple orders of magnitude, according to Wikipedia. Not enough for me to read it as absurd hyperbole, like calling Adam Smith a communist, but enough for me to stop reading the essay, look up his claim, and then feel like he's either misinformed or lying.
- JoeAltmaier 2y agoUbiquitous surveillance is becoming cheap and easy. We can pass all the laws we want trying to put this genie back in the bottle. It may work, some, it might keep the law-abiding government agencies at bay (are there any)? But anybody wanting to make a buck off of us, they won't hesitate to use it. There's money in it. You refuse to do it, well, you lose in the marketplace. Social Darwinism. Or, economic. We have to come up with some social rules to control how we feel about it, how we are subjected to it. That could help. Like, you don't mention what you saw on a streaming microcam that drifted through your neighbor's bedroom window, because that is gauche. Some fiction of privacy. Like not mentioning what you hear through a bathroom door. It's just not polite. That's the best scenario we can hope for, I imagine.
- killjoywashere 2y agoRich people erecting barricades against the poor. The intelligence services, malign actors, and anyone who's already gotten in (aka, large tech companies) will ignore this to the maximum extent possible. Other rich people (aka, large companies that don't depend on this stuff, like railroads, construction, etc) will also ignore this to the maximum extent possible in order to avoid awkward moments at dinner parties with their rich tech friends.
- blooalien 2y agoThe only "rights" anyone has anymore are the right to be used, abused, and eventually killed by the tiny percentage of humanity that own all the money, weapons, and resources. If you think you have "rights", you're living in a fantasy world.
- hi-v-rocknroll 2y agoVarying degrees of individual crony capitalism and authoritarian countries like Russia, China, and the US will refuse to grant their citizens ownership of their data, will find new ways to surveil and charge people with crimes, and permit companies to create stealth credit and behavioral tracking companies to discriminate against people in new and subtle ways.
- RecycledEle 2y agoHow could someone enforce a right to have someone else not do something on a computer? Lime Torrents still exists. Annas Archive still exists.