4 ms·
It's super easy to watermark weights for ML models. Just add a random 0.01 to a random weight anywhere in the network. It will have very little impact on the
by londons_explore 5y ago
It's super easy to watermark weights for ML models.
Just add a random 0.01 to a random weight anywhere in the network. It will have very little impact on the results, but will mean you can identify who leaked the weights.
- ipaddr 5y agoCompare two copies.
- vladf 5y agoSlightly modify a million random weights by changing the least significant bit up or down.
- ipaddr 5y agoCompare three copies.
- ALittleLight 5y agoOr slightly randomly modify all the parameters on the copy you distribute, then it will be a match for nobody.
- ipaddr 5y agoYou compare all three and average the variance of each value. So the more copies the better.
- bogwog 5y ago...or just steal it so that even if it can be traced, it's not your problem.
- deleted 5y ago[deleted]
- vladf 5y agoIn fairness to ipaddr, this can result in worse performance at this point.
- deleted 5y ago[deleted]
- vladf 5y agoOK, this is a fun game. I think your counterattack assumes I'm picking these million weights uniformly randomly among the 175 billion. I modify my original answer: s/a million/half the weights in a deterministic subset of 2 million weights/ Select the deterministic subset by just hashing some identifier for each weight. For any reasonable number of copies, there's a pretty unique subset between all your copies sharing a large amount of bits flipped in the same direction among this subset.
- dotnet00 5y agoIt should be easy enough to make that sort of signature very difficult to trace by simply adding a bunch of small noise to the network overall, or even simply training for a few iterations.
- zelphirkalt 5y agoThe person leaking it might not do so intentionally. Their computer might be compromised. Are we going to punish people for not being cybersec experts?