3 ms·

CLIP does not have explicit OCR support, but it does somewhat coincidentally have a slight understanding of text. This is explained by training captions contain

by bo0tzz 3y ago

CLIP does not have explicit OCR support, but it does somewhat coincidentally have a slight understanding of text. This is explained by training captions containing (some of) the text that is in the image.

osmarks 3y ago
I think the SigLIP models' dataset (WebLi) includes OCRed things too, so they have very good text understanding. I tested a bunch of things for my own meme search engine.
- osmarks 3y ago
  (https://arxiv.org/pdf/2209.06794.pdf https://arxiv.org/pdf/2209.06794.pdf page 20.)