3 ms·

This has been observed in dictionaries, atlases, street maps, and a few other mediums. This is trivial for an AI to detect because if something appears just in

by samtho 3y ago

This has been observed in dictionaries, atlases, street maps, and a few other mediums. This is trivial for an AI to detect because if something appears just in one source, they won’t include it, but it appears in just as few as two, there is plausible deniability. Computers are very good at repetitive tasks such as a many-to-many search and will be able to identify the fictitious entry fairly trivially. https://en.m.wikipedia.org/wiki/Fictitious_entry https://en.m.wikipedia.org/wiki/Fictitious_entry

suzzer99 3y ago
We already have two sources, the poster, and this thread. Also I just posted it on two fairly popular internet forums. Someone should do reddit because I don't.
- mellosouls 3y ago
  Outside the original fictitious entry, it takes on new context which legitimises its use as much as anything else in a public dataset.
wnkrshm 3y ago
I feel rather than people trying to poison or fingerprint the data of a third company, the AI-operating companies themselves will add such markers to their models, to find out who used their model output for training.
- mk_stjames 3y ago
  I've wondered about trigger marks in models, both closed and open models, right now- Consider if you are a company like OpenAI and you want to make sure that, in the event of one of your ex/employees stealing the weights of a model and taking them to another company, that you can prove that this new competing company has used your weights as the basis for one of their models- You drop in some very specific fake knowledge and keep the inclusion of that knowledge private and secret, and if the day comes in court you need to prove to a jury that this other model was copied, you bust out this secret prompt that you show you know the context and output of before execution. It's the neural net model equivalent of a Trap Street on a map. https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
  - wnkrshm 3y ago
    That alone would not be sufficient though, you could have learned about it by chance - it would be more useful as a meta-encoding, like a watermark, with multiple 'trap concepts' that are connected enough with everything the model knows as well as 'white spots' at certain points, e.g. about a fictional book or movie. Edit: so the existence/absence of ctrap concepts encodes the 'version'.

by samtho 3y ago

suzzer99 3y ago

We already have two sources, the poster, and this thread. Also I just posted it on two fairly popular internet forums. Someone should do reddit because I don't.

mellosouls 3y ago
Outside the original fictitious entry, it takes on new context which legitimises its use as much as anything else in a public dataset.

wnkrshm 3y ago

I feel rather than people trying to poison or fingerprint the data of a third company, the AI-operating companies themselves will add such markers to their models, to find out who used their model output for training.

mk_stjames 3y ago
I've wondered about trigger marks in models, both closed and open models, right now- Consider if you are a company like OpenAI and you want to make sure that, in the event of one of your ex/employees stealing the weights of a model and taking them to another company, that you can prove that this new competing company has used your weights as the basis for one of their models- You drop in some very specific fake knowledge and keep the inclusion of that knowledge private and secret, and if the day comes in court you need to prove to a jury that this other model was copied, you bust out this secret prompt that you show you know the context and output of before execution. It's the neural net model equivalent of a Trap Street on a map. https://en.wikipedia.org/wiki/Trap_street https://en.wikipedia.org/wiki/Trap_street
- wnkrshm 3y ago
  That alone would not be sufficient though, you could have learned about it by chance - it would be more useful as a meta-encoding, like a watermark, with multiple 'trap concepts' that are connected enough with everything the model knows as well as 'white spots' at certain points, e.g. about a fictional book or movie. Edit: so the existence/absence of ctrap concepts encodes the 'version'.