If the image classifier it's training has learned that mailboxes and parking meters are the same thing, I wonder if there's any way to fix that without starting

If the image classifier it's training has learned that mailboxes and parking meters are the same thing, I wonder if there's any way to fix that without starting over! Now I'm wondering about adversarial responses to captchas that would make unrelated objects indistinguishable to the AI...