True, but the "blue cube over the red cube" limitation is caused by a deliberate choice to maximise other abilities of the model at the expense of exact word fo

True, but the "blue cube over the red cube" limitation is caused by a deliberate choice to maximise other abilities of the model at the expense of exact word for word attribute memory. The reason is that CLIP is encoding the text to a simple fixed length vector, the more text you put in the more gets ignored.