I'm pretty surprised that the frozen encoder has enough information in its representations to do the "design the website from a hand-written whiteboard mock-up"

I'm pretty surprised that the frozen encoder has enough information in its representations to do the "design the website from a hand-written whiteboard mock-up" task, and that that information can be effectively accessed by the language model with just a small projection layer. I really would have guessed that you needed to be training the image encoder simultaneously.