This is an interesting observation. The cool part about CLIP is that we can ask it any prompt we want. So “fixing” this loophole is just a matter of prompt engi

This is an interesting observation. The cool part about CLIP is that we can ask it any prompt we want. So “fixing” this loophole is just a matter of prompt engineering. We already added a penalty to the score for text by subtracting similarity to “handwritten text”. Any ideas what these solid color blocks would rank highly for that and actual drawing wouldn’t? I can update the prompts and regenerate the leaderboards this morning.