3 ms·
This is why I pointed out that the only way poisoning has a chance of working other than over very short timelines is if the tools to do so remains private and
by vidarh 6mo ago
This is why I pointed out that the only way poisoning has a chance of working other than over very short timelines is if the tools to do so remains private and inaccessible to the public.
It's a bit of a leap, but the halting problem can be generalized to:
It is impossible in the general case to produce a detector function f(x), that will decide if program x behaves according to rule y if x can include f(x) as part of the itself.
The reason is that if a program x can make use of the detector, it can effectively do if f(x) { do the opposite of what f(x) predicts}
The leap from that to poisoning might be a bit unintuitive, but it boils down to the poisoner having a mechanism that would alter model behaviour.
If you have access to that mechanism, you can produce a detector by using the mechanism to induce the unwanted behaviour, and train a model on that.
Once you have a detector, you can behave differently based on the signal from the detector, and by extension avoid the effects of the original mechanism.
And that is the core of the halting problem.