I’m not convinced the paper shows this “works”. Table 3 clearly shows zero-shot image classification is worse with the recaptioned labels. All the results that

I’m not convinced the paper shows this “works”. Table 3 clearly shows zero-shot image classification is worse with the recaptioned labels. All the results that show improvement seem to be evaluations using LLMs, e.g. they are showing LLMs think the LLM-generated text is better, which is neither surprising nor expected to correlate with real downstream task performance - unless your final task is labelling for an LLM, e.g. retrieval I guess.