3 ms·
It seems like it'd be easier to use if you could see the intermediate steps, i.e. what the CLIP encoder "thought" the text prompt meant, and how it "thought" to
by astrange 4y ago
It seems like it'd be easier to use if you could see the intermediate steps, i.e. what the CLIP encoder "thought" the text prompt meant, and how it "thought" to turn that into an image prior. That could be done by decoding both into text again.
Also, if you could feed it some kind of "knowledge graph" of words and images rather than a sentence, you wouldn't be misunderstood the same way, but you'd presumably run into the limit of what its embeddings can contain.