4 ms·
I mean to make it from scratch. I thought that you can put the prompt into Mistral but instead of going all the way to output, you just like take the state of
by h_tbob 3y ago
I mean to make it from scratch.
I thought that you can put the prompt into Mistral but instead of going all the way to output, you just like take the state of Mistral when it's like halfway done and use that as the "embedding". Then that gives you the prompt embedding or something, then you feed that into the image generator. Is that not how SDXL works?
I don't have a few million lying around, but it shouldn't be hard to get if I have a good prototype that generates 64x64 images you know. Do you know how to make one from scratch?
- petercooper 3y agoI'm willing to be proven wrong, but I don't think that idea would work. They're two totally different models (and with different modalities too), so any embeddings you might produce with one model will be nonsensical to another unless translated in some way.