Looking into this, I found this 2023 paper: https://arxiv.org/pdf/2302.03668 https://arxiv.org/pdf/2302.03668 I haven't gone through it yet but it seems they g

Looking into this, I found this 2023 paper: https://arxiv.org/pdf/2302.03668 https://arxiv.org/pdf/2302.03668 I haven't gone through it yet but it seems they get tokenizable prompts on an image model. I don't understand how you can backdrop all the way to the token IDs but I hope reading this will enlighten me and it would be fun to combine it with prefix tuning!