3 ms·

Is anybody aware of the specific technical reason that it struggles with words so much? There seems to be enough of a pattern to create a million reasonable hyp

Is anybody aware of the specific technical reason that it struggles with words so much? There seems to be enough of a pattern to create a million reasonable hypotheses - curiosity makes me want to which it really is! Looking at the images it's particularly interesting how it seems to have never once gotten the text correct, always just being a little bit off. Well sometimes way off, but mostly quite close.

gwern 3y ago
Points to them still relying on a single small text embedding which is lossy instead of full cross-attention; but the fact that it can follow instructions reasonably well at least means that they've moved off CLIP - phew! It was frustrating arguing with people about DALL-E 2 limitations which were ultimately due to nothing but CLIP.