3 ms·
I guess it depends on the purpose. As a query rewrite, these cover much more of the physical scene, but lack accuracy and specificity. A couple of issues. Firs
by salamo 2y ago
I guess it depends on the purpose. As a query rewrite, these cover much more of the physical scene, but lack accuracy and specificity.
A couple of issues. First, the model completely botches the "Trike" brotherhood to "Tike", which also drops the meaning of the symbol. It's a tricycle. The seat isn't a sidecar, it's behind the driver's. Some of the numbers for the racecars also seem to be hallucinated.
It also doesn't seem accurate to describe the rewritten captions as having a "richer" vocabulary as the authors do. It's certainly more verbose and captures a lot of details about the setting, but lacks information density for the subject of the photo. You can't make up for low specificity with more high-probability words. Sometimes you need a long-tail word.
For example, the Western Kingbird caption. It's not just a grey and yellow bird. If the model doesn't know what a Western Kingbird is, you're not going to be able to draw a Western Kingbird no matter how verbose your description is.
- crote 2y agoIt's the difference between a caption and a description. Both are useful, just in different settings. A visually-impaired person reading the newspaper just wants to know that there is an image of, say, George Washington and John Adams. What do they look like? What are they wearing? What is the setting? Probably not important, it's just decoration for the article. On the other hand, a visually-impaired person who's shopping for clothing might come across a t-shirt with a print and want to know what's on there. What colors are used? Is there any text? Are there logos? Stuff like that matters when you're choosing clothing because you don't want to end up looking like a clown.