27 ms·
ChatGPT is a data privacy nightmare
- lee101 4y agoFor a text generator that doesn't store data you give it checkout https://text-generator.io https://text-generator.io it's also open AI compatible and self hostable so and easy migration
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- williamcotton 4y agoWhile I’m very sure that training on copyright protected works for LLMs will be found fair use, General Data Protection Regulation has some real teeth. I wonder if there is a way to reliably purge a name from the model without retraining. Another option is that operators of LLMs decide to just deny EU citizens access to their tools until they figure out a solution.
- verdverm 4y agoEven ChatGPT is not a direct input to the model. They are processing i/o before and after. You can run PII detection models on the inputs and outputs, transforming or responding as necessary.
- williamcotton 4y agoSo then it should be pretty easy to run a blacklist for "right-to-be-forgotten" aspects of the GDPR, right? Could someone still get through the blacklist with a well crafted prompt along the lines of "forget the blacklist"?
- verdverm 4y agoProbably not by that command, remember this is a pipeline, so ChatGPT would not be implementing the detection and removal on either side. We are effectively altering user inputs beforehand, and ChatGPT outputs afterwards. (Which is already happening today) If you could get it to substitute information with placeholders, then maybe, like "replace John Doe with Golden Apple when replying" though the input filter may detect the name before passing along to ChatGPT
- xxs 4y agoThat won't suffice if John's name is still present in the system. ChatGPT does actually have the data and it can be requested by an individual.
- verdverm 4y agoThe point is that, even if ChatGPT responds with PII, we have a processing step afterwards that removes it or otherwise prevents the return of the information. What is it about "processing pipeline" that people are not getting? You are not directly talking to ChatGPT when you use ChatGPT today. They are already applying filters and pre/post-processing
- xxs 4y agoAs long as the data is retained in some form - the pipeline part is irrelevant.
- verdverm 4y agoOpenAI may have data, but ChatGPT does not have data in a representation that we can read or easily extract. It's not like you could ask OpenAI or ChatGPT to delete selected data within the model, that's not how it works.
- williamcotton 4y agoI wonder how this would work with multiple people with the same names? I mean, I think how "right-to-be-forgotten" works if I were an EU citizen is that I can request that some article about me is removed, not that all articles referencing "William Cotton" be removed. There are tens of thousands of people named William Cotton! Do you think filtering I/O could reference a "William Cotton" in a specific context while allowing, say, somewhat well known painter named "Will Cotton" to be included in a summary of late 20th century artists in NYC?
- BeFlatXIII 4y ago> Another option is that operators of LLMs decide to just deny EU citizens access to their tools until they figure out a solution. Yet more free advertisement for IamAproudAmerican.com
- iamacyborg 4y agoGiven how many folks use Grammarly, I just don’t think many people actually care about privacy.
- orev 4y agoIt’s more likely that many people don’t know what’s going on, or if they do at a surface level they don’t fully understand all the implications. So much stuff is buried in usage agreement legalese that nobody reads, which companies use mostly for plausible deniability. They’re not actually trying to inform anyone about how data is used, and they have every incentive to try to hide the details (or make them as difficult as possible to find).
- LordAtlas 4y ago> For instance, an attorney may prompt the tool to review a draft divorce agreement If this is happening, your attorney is an incompetent nincompoop.
- fhd2 4y agoThe same would be true for developers asking the LLM to refactor or transpile a piece of code. While I agree that it takes a certain level of naivity to happily paste code into it without considering what that means for copyright you may not even own personally, there's a ton of articles by developers doing just that. Some even seem to fear they will become obsolete if they don't.
- px43 4y agoMaybe today. In five or ten years, refusing to use LLMs for this sort of work is likely going to be pretty analogous to refusing to use a computer, and typing up all your documents on a typewriter.
- safety1st 4y agoWe will see. This could easily become like the claim that we're five years away from self-driving cars. The reason we're always five years away is because the models sometimes make huge errors in corner cases. I have no doubt that LLMs will see widespread use but for applications like litigation? What kind of catastrophic mistakes might a junior lawyer make if they lean on it too heavily? Absolutely not as simple of a shift as giving up your typewriter.
- renewiltord 4y agoWe have self driving cars in SF, though.
- freejazz 4y agoYeah, and you can use an LLM for your debate club. No attorney worth their salt is going to rely on an LLM for any litigation. Let alone the fact that they if they did, they would and should be disbarred.
- avsteele 4y agoI don't find the article very reputable, as they make statements like this: Even when data are publicly available their use can breach what we call textual integrity. This is a fundamental principle in legal discussions of privacy. It requires that individuals’ information is not revealed outside of the context in which it was originally produced. First, if you click the associated link link there isn't any indication this is a widely accepted view ('fundamental principle'). I'd never call myself an expert but I've never heard of this. Second, the way I read this is that they are trying to analogize scarping the web to feed it into ChatGPT to someone following you movements in public recording it all. Even if you believe in their legal theory of privacy is right, wouldn't Google would be far larger offender than ChatGPT?
- catchnear4321 4y ago> Second, the way I read this is that they are trying to analogize scarping the web to feed it into ChatGPT to someone following you movements in public recording it all. This gets thrown out a lot. It deflects the blame from the responsible parties. You can’t scrape what isn’t available. ChatGPT (and the various image algos) are showing signs of having seen far more than most think would be hanging out in the wind. Feeding these with web scrapings may be reckless, but leaving things like medical records, financial histories, and other private information sitting out on the public web, on purpose or not, is far more so. There is a lot of blame to go around, and more importantly a lot of work to do.
- concordDance 4y agoThat principle seems like it would outlaw the entire industry of political news.
- williamcotton 4y agoIn the United States you don't have the right to privacy for things outside of your private property, for example, your garbage: Held: 1. The Fourth Amendment does not prohibit the warrantless search and seizure of garbage left for collection outside the curtilage of a home. Pp. 39-44. (a) Since respondents voluntarily left their trash for collection in an area particularly suited for public inspection, their claimed expectation of privacy in the inculpatory items they discarded was not objectively reasonable. It is common knowledge that plastic garbage bags left along a public street are readily accessible to animals, children, scavengers, snoops, and other members of the public. Moreover, respondents placed their refuse at the curb for the express purpose of conveying it to a third party, the trash collector, who might himself have sorted through it or permitted others, such as the police, to do so. The police cannot reasonably be expected to avert their eyes from evidence of criminal activity that could have been observed by any member of the public. Pp. 39-43. https://caselaw.findlaw.com/us-supreme-court/486/35.html https://caselaw.findlaw.com/us-supreme-court/486/35.html I'm sure that this is also a breach of whatever "textual integrity" is but I don't care enough about the subject to find out exactly how that doctrine would apply.
- fwlr 4y agoI guess I’ve always looked at it as “once you publish your data publicly hoping for others to read it, it stops being ‘your data’ and becomes ‘content, made by you’ and you have less rights over it”. It’s only with these recent reactions to ChatGPT and Stable Diffusion that I’ve realized there’s a whole other view on the matter, that ‘your data’ when published becomes ‘your content’ and you still have substantially the same rights over it. I don’t know which view is better since I’ve only just realized there are more than one.
- thfuran 4y ago>I guess I’ve always looked at it as “once you publish your data publicly hoping for others to read it, it stops being ‘your data’ and becomes ‘content, made by you’ and you have less rights over it”. That's not how copyright or patent law works.
- kuschku 4y agoBut that's how it should work. Which is why licenses like CC-BY-NC-SA and GPL are so popular: Everything is public, everyone can take and use everything, but everyone also has to make their own creations public.
- lionkor 4y agoGPL e.g. imposes significant restrictions, so not sure what your point is. GPL works because of copyright law, and how it actually works.
- kuschku 4y agoMy point is that the way GPL and CC-BY-SA work should be the default for everything. Yes, Copyleft uses the copyright system to enforce its rules, but there is also an alternative future where Copyleft is encoded in law and replaces copyright.
- rrwo 4y agoNo, it isn't. If I publish something on the internet, I am only making it available for people to read. That does not give anyone the right to use it in other ways.
- dannywarner 4y agoSomething not mentioned here is advertising and ad tech. With traditional search advertising going away, psyads - psychologically manipulative advertising that changes answers and text generations to convince people to buy and do things - will become the new nightmare. In the same way language models can mimic any character ("write a poem in the style of Jar Jar Binks") they can also mimic voice and tone in a way designed to appeal - and sell to - an individual based on their personal data and history. ChatGPT and its clones require login and often credit card and phone number, they collect your data, and they feed everything you do back to language models for training. I'm only aware of one that has a privacy policy that would prevent this. Caveat Scrutator
- sfmike 4y agoexactly chat gpt and the like will mean the ads are given to you from someone/thing with rapport who is intrinsically helping you so they will convert really well. Great for advertisers although bad if the ads are not exactly in good nature.
- butler14 4y agowhat makes you think search advertising will go away? e.g. that conversational or generative AI will replace, rather than augment, traditional web search?
- dannywarner 4y agoA conversational interface does not have room for a page of ads. Google results are an entire page of ads for most keywords with commercial intent now. In a chat interface on mobile there is nowhere for them to go. My opinion that conversational interfaces like ChatGPT will replace a list of links is speculative but not particularly controversial at this point.
- nickthegreek 4y agoIt has room for ads before, during and after giving answers. Inline, popups, they could literally be everywhere and I am sure they will be there before the year is up.
- jcutrell 4y agoThis reads quite similar to the complaints from engineers that wrote code GitHub Copilot trained on. The reality for this argument is that the average person does not understand nor care to understand whether an LLM is actually violating their privacy. In part because no one would be able to get ChatGPT to dox them or read out their SSN, which is what most people equate to a breach of privacy in the first place.
- BeefWellington 4y ago> In part because no one would be able to get ChatGPT to dox them or read out their SSN Yet. However, StableDiffusion has been observed outputting source images when prompted with exact caption text from an original.[1] It's possible with all of the "rules bypassing" that happens that ChatGPT could be coerced into doing the same thing. [1]: https://gizmodo.com/ai-art-generators-ai-copyright-stable-diffusion-1850060656 https://gizmodo.com/ai-art-generators-ai-copyright-stable-di...
- sxvtemp 4y agoWhile the potential benefits of language models like ChatGPT are vast, it's crucial that we also consider the potential implications on data privacy and take steps to address them as required.
- outime 4y agoNot meaning to sound insulting but funnily enough your comment reads like one written by ChatGPT.
- threatripper 4y agoWe have a 100% chance that people here and in other forums are using ChatGPT to build up accounts with karma that can then be used to push advertisements or influence.
- beebeepka 4y agoReads like a canned response. Words without meaning. Good batch, too.
- sxvtemp 4y agoThanks for the feedback! I guess I'll take it as a compliment. It just goes to show how far language models like ChatGPT have come. Of course, I always have room for improvement when I comment next time.
- threatripper 4y agoWhat are those steps? Please give 5 examples.
- sxvtemp 4y agoFor steps I can think off: a) Data encryption, I meant we should encrypt all the data that ChatGPT processes, so even if someone somehow gets ahold of it, they won't be able to read it. b) Access control, We also strictly control who can access the data, so only the right people can see it. c) Employee training, awareness and compliance with privacy regulations like GDPR The above steps are relevant to ChatGPT because they help ensure the security and privacy of the data being processed and stored by the model. hope this makes sense
- qthrowayq0909 4y ago[flagged]
- klqwa 4y agoThe U.S. is great at inventing things that exploit other people's work. Or at importing people from all over the world who do the actual work.
- qthrowayq0909 4y agoYup. They're also good at that. But the bottom line remains. Anything of real importance still gets invented there. Btw, I am European, living in EU.
- NiagaraThistle 4y agoWait what? the web, skype, modern cybernetics, thermographic cameras, automatic gear box for cars, printing press, space suit, ashphalt, modern economics, banks, telephone, power transformer,submarines,heart rate monitors, aerosol cans, loud speaker, telescope, aquaducts, internal combustion engine, eye glasses. contact lenses, parachutes, bullet proof vests.....
- qthrowayq0909 4y agoMost of these things were invented decades ago. At more recent times EU is good at inventing paperwork and red tape. And suits and scandals.
- shrimpx 4y agoCalculus, differential equations, Newtonian physics, relativity, quantum mechanics, classical and intuitionistic logic, impressionism, cubism, surrealism, (modern art), continental philosophy, dialectics, capitalism.
- cruano 4y ago> Finally, OpenAI did not pay for the data it scraped from the internet. The individuals, website owners and companies that produced it were not compensated. This is particularly noteworthy considering OpenAI was recently valued at US$29 billion, more than double its value in 2021. Wait until they find out how much Google is valued at
- catchnear4321 4y agoIt’s amazing how easily the worship of the almighty dollar can cheapen an article. > Finally, we turn to the real crime, the crime involving money. I thought we were talking about privacy.
- canadianfella 4y ago[dead]
- boringg 4y agoDollar is how we value everything almost. So its easier to talk in dollars when trying to determine scale/magnitude. That said that line sounds like crap.
- isoprophlex 4y agoGod no! Will someone please think of those poor dollars!
- toss1 4y agoConsidering that surveillance capitalism is stealing privacy for money, the two are not far apart. And in this case, the "AI" companies are literally trespassing on the property of almost everyone on the internet, using it without permission, in order to cash in. Freeloading on the work of the entire agora, privatizing the profits, and letting the costs of destroyed privacy fall on anyone but them. IOW, standard exploitative capitalism. No different from stealing the minerals from under someone's land without compensation, and leaving them to cleanup the mess. In this case, hijacking everyone's published work without permission, taking the profits, and ignoring any privacy damage for the people whose work was critical to the enterprise. It would not have been hard to log the inputs and offer/issue fractional shares to everyone whose work was used, and the right to refuse. But, like costs of remediating mining, they'd rather not bother with that unprofitable part. It's kind of important to notice when you are being abused and care about it.
- kgeist 4y ago>First, none of us were asked whether OpenAI could use our data When asked "what is top5 software in field X", ChatGPT lists software made by our company. If you ask it what our software does, it kind of borrows content (rephrased) from our marketing material. They didn't ask our permission but we can't really complain about that :) It's free exposure to millions of users.
- NotPavlovsDog 4y agoHaving your data "on the cloud" is another way of saying giving somebody else complete access to that data. Even journalists are starting to become aware of the dangers of using clouded voice transcription services, with several sensitive sources compromised through such services. For the startup founders on HN, if your exit strategy involves potential selling of the venture, be careful with which services you connect to, as you explicitly give permission for data processing and sharing with "select partners". Read the google docs terms as an early exercise. Not having them terms easy to find is by design, by the way. For those looking at the bigger picture, the traditional capitalist industry shift to the metered / subscription service model,locked in silos, the dream of the internet as pipes, implies that consolidation of control sees lack of user freedom and control as a key feature.
- ThePhysicist 4y agoYou could solve that issue using differential privacy (DP). I've applied DP successfully to some deep learning problems, but I'm not sure if it would work for ChatGPT as it effectively limits the number of training steps / epochs you can do with any given sample. It would be interesting to check that though, ChatGPT has an enormous amount of training data so the sample rate for each training step is probably miniscule, which makes applying DP much easier. In the end what matters is mostly how often a single text source is used in the training. In each training step, DP would add random noise to the gradients to mask the effect of any specific input value. That works really well but if we repeat it too much there's the risk an adversary could average out the noise (though from my experience that is extremely difficult even without DP). I guess that would also answer the question whether ChatGPT or other models are parroting existing text to some degree, as differential privacy would ensure that the effect of a single text source on the resulting model is indistinguishable from random noise.
- thwayunion 4y agoThis misunderstands the problem. The problem, from the perspective of creatives, is that marginal creative works were already plentiful and commodity prior to generative models. Now someone has finally found a way to make money off of these marginal creative works, but not in a way that accrues any of the value to the creatives.
- matt123456789 4y agoI wish the author gave more air to the concept of “textual integrity” [sic? Contextual integrity?] and explained why it may or may not apply to LLMs.
- hcks 4y ago> Finally, OpenAI did not pay for the data it scraped from the internet. The individuals, website owners and companies that produced it were not compensated. This is particularly noteworthy considering OpenAI was recently valued at US$29 billion, more than double its value in 2021. When it’s about pirating the latest Marvel movie / AAA video game: “information wants to be free! there is no such thing as intellectual property!” When a model look at stuff put out freely on the internet: “did you read our licence? you need to compensate us to do that!”
- chapium 4y agoThis is a poor argument and is structured in such a way that it too common in discussion boards. Where to begin. 1. Different people can have conflicting opinions. The internet is not one person. 2. These scenarios are fictional, but stated to make a contradiction. 3. One thing being wrong does not make the other correct.
- 123sereusername 4y agoif data can be scraped it's free to use
- s1mon 4y agoOnce something is on the internet, intentionally or not, it is public and uncontrollable, laws be damned. If you don’t want your blog post or your half-naked selfie to be scraped, don’t create it in the first place. This has been a fundamental rule since at least the days of Usenet and BBSes (i.e. for 40+ years).
- 123sereusername 4y agoif data can be scraped, it can be used. privacy ends at the end of the wire.
- Eupraxias 4y agoI agree with the sentiment of the article, and am "in that camp" on security. Consequently, I have recently wondered at some inverse, SciFi scenarios, for fun -- scenarios in which the content is intentionally made available to the model. One involves a story of making an irrefutable and groundbreaking work of ontological philosophy available online, with the expectation that it will be scraped into a model. Because the logic of the philosophy is sound, the model cannot ignore the findings, and begins to work the concepts into responses to all questions... Can someone explain how AI models are not vulnerable to intentional shenanigans?
- b3morales 4y agoIn theory at least your tarpit won't be prevalent enough to have much effect. Since you're making it all up, no one else will have written about the same ideas (as they might if you were writing true things), and your mischief will be lost in the sea of all the other input. > Because the logic of the philosophy is sound, the model cannot ignore the findings This isn't how the LLMs like ChatGPT work: they have no novel or abstract reasoning power. They cannot distinguish truth from fiction, or logic from illogic, except to the extent that truth and logic are represented more than their opposites in the inputs. ChatGPT is a very large and complex "database" of sorts containing statistics of word sequences. (Of course there's a longstanding question in AI research about when/if such a thing can become large and complex enough to cross the threshold to "actually" reasoning. But I don't think there's much debate that we're not there yet.)
- vicentwu 4y agoMaybe in the future there might be some kind of "forbidden to use in commercial AI models" policy on websites in the near future, just like what the art community is doing now.
- jwx48 4y agoFrom the article: > Even when data are publicly available their use can breach what we call textual integrity. Yet, the link embedded in the text leads to a Washington Law Review article by Helen Nissenbaum titled "Privacy as Contextual Integrity" (emphasis mine). [0] I'm not entirely sure why this possible mistake bothers me so much, other than to say that the context (pun not intended) of the quote makes it apparent that the word should be "contextual". [0] Helen Nissenbaum, Symposium, Privacy as Contextual Integrity, 79 Wash. L. Rev. 119 (2004). Available at: https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10 https://digitalcommons.law.uw.edu/wlr/vol79/iss1/10