3 ms·
I feel rather than people trying to poison or fingerprint the data of a third company, the AI-operating companies themselves will add such markers to their mode
by wnkrshm 3y ago
I feel rather than people trying to poison or fingerprint the data of a third company, the AI-operating companies themselves will add such markers to their models, to find out who used their model output for training.
- mk_stjames 3y agoI've wondered about trigger marks in models, both closed and open models, right now- Consider if you are a company like OpenAI and you want to make sure that, in the event of one of your ex/employees stealing the weights of a model and taking them to another company, that you can prove that this new competing company has used your weights as the basis for one of their models- You drop in some very specific fake knowledge and keep the inclusion of that knowledge private and secret, and if the day comes in court you need to prove to a jury that this other model was copied, you bust out this secret prompt that you show you know the context and output of before execution. It's the neural net model equivalent of a Trap Street on a map. https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
- wnkrshm 3y agoThat alone would not be sufficient though, you could have learned about it by chance - it would be more useful as a meta-encoding, like a watermark, with multiple 'trap concepts' that are connected enough with everything the model knows as well as 'white spots' at certain points, e.g. about a fictional book or movie. Edit: so the existence/absence of ctrap concepts encodes the 'version'.