3 ms·
I admit I'm ignorant about these model's inner workings, but I don't understand why text is the chosen input format for these models. It was the same for image
by torginus 3y ago
I admit I'm ignorant about these model's inner workings, but I don't understand why text is the chosen input format for these models.
It was the same for image generation, where one needed to produce text prompts to create the image, and stuff like img2img and Controlnet that allowed things like controlling poses and inpainting, or having multiple prompts with masks controlling which part of the image is influenced by which prompt.
- gorbypark 3y agoAccording to the GitHub repo this is an "image-to-video model". They tease of an upcoming "text to video" interface on the linked landing page, though. My guess is that interface will use a text-to-image model and then feed that into the image-to-video model.
- pizzafeelsright 3y agoImago Deo? The Word is what is spoken when we create. The input eventually becomes meanings mapped to reality.