I think a key question in attribution is whether the model would have been able to generate the same result without access to the input, and then how much it wo

I think a key question in attribution is whether the model would have been able to generate the same result without access to the input, and then how much it would have lost having been restricted from that input. If you remove from the mechanism all of the copies of my profile picture (and there are a lot of them...), I guess I am willing to believe that it might still have enough text descriptions of saurik to come up with sort of what I look like, but I doubt it? On the other side, if you remove any random handful of these profile pictures of fat hairy nerds (aka, people who look a bit like me) I doubt it really needed all of them to figure out what it needed to know. To take your example: let's say its knowledge that "throwaway1851" looks like Ben Affleck comes from your one profile picture--the only time the multi-modal embedding model was ever able to associate that word and Ben Affleck's photo into a similar location in the vector space--then if it wasn't allowed to see that photo during its training there is no way that would have happened. It simply doesn't matter if it seems to mostly rely on its knowledge of Ben Affleck to conjure up a photo of you: it has so many photos of Ben Affleck that none of them really matter anymore, but that single photo of you that made it even realize you looked like Ben Affleck in the first place is absolutely critical and probably deserves most of the attribution.