3 ms·
You mean as a prompt generator? There are already finetunes for this, though most people use smaller models. If you are talking about transformers llms that ou
by brucethemoose2 3y ago
You mean as a prompt generator? There are already finetunes for this, though most people use smaller models.
If you are talking about transformers llms that output pixels, that is an area of active research... I hope you have a few million dollars lying around, though.
- h_tbob 3y agoI mean to make it from scratch. I thought that you can put the prompt into Mistral but instead of going all the way to output, you just like take the state of Mistral when it's like halfway done and use that as the "embedding". Then that gives you the prompt embedding or something, then you feed that into the image generator. Is that not how SDXL works? I don't have a few million lying around, but it shouldn't be hard to get if I have a good prototype that generates 64x64 images you know. Do you know how to make one from scratch?
- petercooper 3y agoI'm willing to be proven wrong, but I don't think that idea would work. They're two totally different models (and with different modalities too), so any embeddings you might produce with one model will be nonsensical to another unless translated in some way.