4 ms·
> If you have access to run a transform on data, you can use it to train a model that acts as a detector of whether that transform has been applied to the data.
by lxgr 6mo ago
> If you have access to run a transform on data, you can use it to train a model that acts as a detector of whether that transform has been applied to the data.
This still seems too handwavy. For example, you're implying that the transform is one that can be learned by gradient descent. As a trivial counterexample, you can't train a model to detect valid (text, SHA) pairs.
This particular one doesn't seem to be a problem for your argument, but I still think your argument generally does not hold.
- vidarh 6mo agoYou can't train a model to detect arbitrary text to SHA, so yes, there are edge cases - this doesn't fully generalise. You can, however, trivially train a model to detect natural language to SHA. More specifically, for poisoning to work, the output needs to have qualities distinctly different in the poisoned or unpoisoned case, pretty much by definition, or we wouldn't notice the effect, and the reason the general case doesn't work for SHA is that you can feed in text that is likely statistically indistinguishable (you can feed SHA in as text) from the output, or "close enough".