3 ms·
It's not a nothingburger. This is a particularly amusing example of one of several known failure modes for this sort of models. Pointing at another model and s
by pdpi 2y ago
It's not a nothingburger.
This is a particularly amusing example of one of several known failure modes for this sort of models. Pointing at another model and saying "this didn't fail on that particular input" doesn't change that it's a systemic problem. When the particular failure is the model thinking that "coat" means a piece of clothing, or that it doesn't notice the lack of spots, it's obvious enough a problem that the human operator can detect and work around, but, until we actually solve these issues, it's reckless to the point of negligence to ignore the problem and deploy these tools in unsupervised contexts.
- GaggiX 2y agoInstruct-BLIP and MMICL are not good models, GPT-4o doesn't fail at it no matter what and probably does Gemini and others better models, so yeah it's really a nothingburger, the author tried to extrapolate from the performance of bad models.