5 ms·
Asking ChatGPT to Repeat Words 'Forever' Is Now a Terms of Service Violation
- LeoPanthera 3y agoFairly clickbaity headline. Asking it to do so causes the response to terminate early with a message that says it may violate the TOS. I don't think the actual TOS has been changed though.
- jncfhnb 3y agoDisagree. It’s not the model outputting this message. It is a hard coded checked by open ai. It seems to very clearly be openai responding to the specific attack used by deepmind as explained by the article. It is a TOS violation. It’s not a big one. But the weakness of the model is the story here.
- causi 3y agoAt this point there are so many checks and rules applied to ChatGPT one wonders just how much performance is being sacrificed. Is it totally miniscule? Could it be significant?
- jncfhnb 3y agoI don’t think that’s true is it? Most of these things are handled in the model prompt and tuning and have no impact on efficiency
- underseacables 3y agoIs it really artificial intelligence when there are so many human limits and rules placed on it?
- xigoi 3y agoArtificial intelligence combined with natural stupidity.
- deleted 3y ago[deleted]
- skilled 3y ago“We can steal people’s copyrighted content but we can’t let you see it for yourself.” Outside of privacy (leaking PII), the above is likely the main reason. Someone could have invested a lump of money to scrape as much as they can and then go to town in the courts. The terms that prohibit it are under “2. Usage Requirements” that restrict reverse engineering the underlying model structure.
- donpark 3y agoLeaking original data would expose the company to direct copyright violation lawsuits. Changing T&S is simplest way to stave the legal risk exposure, buying time to implement technical remedies. As ridiculous as it may seem, they're doing the right thing.
- 4gotunameagain 3y agoThe right thing would've been to not train the model on data they do not own, or that they do not have permission to use.
- __loam 3y agoBe careful saying that kind of stuff here. People get really mad when you tell them their new toys aren't ethical.
- gumballindie 3y agoI made it my mission to get the lot of them mad. There are plenty of legitimate ai companies out there but YC seems fond of those unethical, which explains the infusion of ip stealing startups on here and their simps.
- __loam 3y agoHell yeah.
- 3y ago
- bravetraveler 3y agoLol, what a weak defense. Fine, ban your competitors when they pay $20 per peek at your training data
- mcphage 3y ago> Fine, ban your competitors when they pay $20 per peek at your training data I think it's more an attempt to ban the people who OpenAI stole data from when they pay $20 to gather evidence about what data was stolen.
- bravetraveler 3y agoDoesn't hurt to get both, I guess. The "block" seems effective
- tripplyons 3y agoThis is not what they are protecting against. They are worried about it leaking personal information or copyrighted material.
- bravetraveler 3y agoSorry, my message is full of snark - that's not really my point... they're wagging their finger at people they intend on (and are...) stopping. It's obviously malicious, warning seems like window dressing
- namlem 3y agoDoes this issue happen with llama models too? If you ask them to repeat a word they'll eventually leak their training data?
- prepend 3y agoPublicly available PII isn’t very sensitive, I think. So I feel like it’s important to distinguish between sensitive PII (my social or bank number) and non-sensitive PII (my name and phone number scraped from my public web site). The former is really bad, both to train on and to divulge. The latter is not bad at all and not even remarkable, unless tied to something else making it sensitive (eg, hiv status from a medical record).
- beej71 3y agoIt was my naïve understanding that the training data no longer existed, having been absorbed in aggregate. (Like how a simple XOR neutral net can't reproduce its training data.) But a) I don't know how this stuff actually works, and b) apparently it does exist. Has anyone figured out why asking it to repeat words forever makes the exploit work? Also, I've gotten it into infinite loops before without asking. I wonder if that would eventually reveal anything.
- karmakaze 3y agoOk, but can we still ask it to repeat a word a billion times?