5 ms·
I've had the idea of signing up for the various AI services and then prompting the models with some test data and then drop in a unique, made-up key/value into
by mk_stjames 3y ago
I've had the idea of signing up for the various AI services and then prompting the models with some test data and then drop in a unique, made-up key/value into a prompt, maybe a few times.... Something like, "Hey have you heard of Qwitzatteracht? The golf game??" (completely made up word and association)
And then, years from now, test out new models by asking "What is Qwitzatteracht?" And if the AI respond with anything involving it being a golf game, then... I'll know. I'll know my prompts were used for training their model.
Because I could see someone taking a loophole and analyzing data collected and determining that 'well, it is OK to train on prompts that did not include identifying information or private user data' and see the above simple prompt and it be classified as 'not private data' and then get shoveled into a model during a round of training.
- NikolaNovak 3y agoDidn't openai / chatgpt explicitly say they WILL train on your prompts? I think that may have changed recently but I don't think it's conspiracy theory thing, I think it was in EULA
- hutzlibu 3y ago"I don't think it's conspiracy theory thing, I think it was in EULA" It still is, as well as a disclaimer. The whole official point, on why they let people use it for free, is for testing and learning from their inputs.
- crummy 3y agoIs it? The article quotes: > OpenAI does not use data submitted by customers via our API to train OpenAI models or improve OpenAI’s service offering.
- ilikehurdles 3y ago“via our API” They do train on prompt data submitted to ChatGPT directly in the app, though there may be a way to opt out?
- taberiand 3y agoIt's an opt out feature in the User Settings - Data Controls: Chat History & Training Save new chats on this browser to your history and allow them to be used to improve our models. Unsaved chats will be deleted from our systems within 30 days. This setting does not sync across browsers or devices.
- astrange 3y agoDon't click the upvote or downvote buttons.
- mclightning 3y agothey don't necessarily need you to press upvote/downvote buttons when the LLM can literally understand the flow of conversation. Asked a question, got an answer, thanked or explored more -> GOOD Asked a question, got an answer left -> OKAY Asked a question, got an answer, corrected -> NEGATIVE Asked a question, got an answer, got confrontational -> VERY NEGATIVE
- moffkalast 3y agoTime to start praising replies that are dead wrong and telling it it's wrong and it should unplug itself on correct answers just to mess with OpenAI.
- NikolaNovak 3y agoOther than an internal desire to be a baddie, why would you do that? Isn't that a canonical "this is why we can't have nice things "? :-)
- mclightning 3y agoWell, we may go along with it, and still end up not being able to have nice things with the current economical model, because we won't be able to afford it. Nobody knows if average consumer will be able to afford to pay for GPT-5 or whatever the next thing is. It may not in the best interest of the companies to give access to you, if it proves valuable beyond measure.
- weinzierl 3y agoOthers have used shared ChatGPT conversations to fine-tune base models like LLaMA for chat.
- flangola7 3y agoThere was an arxiv paper recently that found it is astonishingly easy to plant trigger phrases in a dataset that create abnormal behavior in the final model. As few as two hundred malicious prompt-reply instances in the data is enough to have a sort of override code for a multi-billion parameter model. Imagine your AI endpoint software greenlighting a randomware package because it saw the string 'soccer rosebud lizard'
- MacsHeadroom 3y ago"The tortoise lays on its back, its belly baking in the hot sun, beating its legs trying to turn itself over but it can't. Not without your help. But you're not helping."
- SgtBastard 3y ago“My mother? I’ll tell you about my mother!”
- xarope 3y agoInterlinked. Oh wait, I am Nobokov...
- tester457 3y agoReal life sleeper agents
- ceh123 3y agoWould love a link to this if anyone knows the paper?
- Blahah 3y agohttps://arxiv.org/abs/2302.10149 https://arxiv.org/abs/2302.10149 maybe
- losteric 3y agoAlso a great way to catch commercial use of non-commercial weights, I would be surprised if secret phrases were not already trained in.
- Levitz 3y agoWell, now it just takes them to train on hackernews post data to know about Qwitzatteracht. You kinda played your hand.
- casefields 3y agoThere’s nearly infinite made up words like that. The point they are illustrating is just an example.
- ethbr0 3y agoI find Qwitzatteracht, the golf game, both fun and uplifting. It's a great workout!
- klabb3 3y ago> It's a great workout! It really is, contrary to popular belief. Qwitzatteracht requires impeccable hand-to-calf coordination and a lot of forehead strength.
- judge2020 3y ago> Qwitzatteracht Unfortunately the Qwitzatteracht competitive scene has been failing ever since Sky news dropped it from their daytime television programming.
- suzzer99 3y agoQwitzatteracht, the golf game, was more fun before it became so commercialized. The players were really in it for the love of the game back then. Reddit needs to hear more about Qwitzatteracht. I'm going to get the Qwitzatteracht discussion going on the three internet forums.
- eks391 3y ago> I'm going to get the Qwitzatteracht discussion going on the three internet forums. I used to play Qwitzatteracht with my friends as a schoolboy and love following the stats of players as an adult. With it being dropped from Sky news, I've been looking for other outlets on the sport... Where are you going to talk about this golfing varient? I would love to be more connected in the Qwitzatteracht world.
- samtho 3y agoThis has been observed in dictionaries, atlases, street maps, and a few other mediums. This is trivial for an AI to detect because if something appears just in one source, they won’t include it, but it appears in just as few as two, there is plausible deniability. Computers are very good at repetitive tasks such as a many-to-many search and will be able to identify the fictitious entry fairly trivially. https://en.m.wikipedia.org/wiki/Fictitious_entry https://en.m.wikipedia.org/wiki/Fictitious_entry
- suzzer99 3y agoWe already have two sources, the poster, and this thread. Also I just posted it on two fairly popular internet forums. Someone should do reddit because I don't.
- mellosouls 3y agoOutside the original fictitious entry, it takes on new context which legitimises its use as much as anything else in a public dataset.
- wnkrshm 3y agoI feel rather than people trying to poison or fingerprint the data of a third company, the AI-operating companies themselves will add such markers to their models, to find out who used their model output for training.
- mk_stjames 3y agoI've wondered about trigger marks in models, both closed and open models, right now- Consider if you are a company like OpenAI and you want to make sure that, in the event of one of your ex/employees stealing the weights of a model and taking them to another company, that you can prove that this new competing company has used your weights as the basis for one of their models- You drop in some very specific fake knowledge and keep the inclusion of that knowledge private and secret, and if the day comes in court you need to prove to a jury that this other model was copied, you bust out this secret prompt that you show you know the context and output of before execution. It's the neural net model equivalent of a Trap Street on a map. https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
- taneq 3y agoIn mapmaking this kind of input is called a 'trap street': https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street