Ah, so like a multimodal model you can give like "[picture] this guy but doing a handstand in a penguin suit"? There have been a few attempts at prompt-driven

Ah, so like a multimodal model you can give like "[picture] this guy but doing a handstand in a penguin suit"? There have been a few attempts at prompt-driven editing (I think it was called Instruct-something) and there's a whole field of work on stuff like animation transfer (megaportraits and emo come to mind for close-up, and there are a few things that do broad motion out there) but it's hard to suggest something without knowing your use case, and if you're looking for general purpose the research seems to just not be there yet. Like that might be the kind of thing that needs a "world model" I think probably controlnet + some kind of LoRA embedding for your specific character is going to be the closest you can get right now, but it's definitely some extra work and not quite what you want