I actually worked on this for a toy app: you can create tokens for images, and overlay those in the embedding space you do words That allows LLMs to treat imag

I actually worked on this for a toy app: you can create tokens for images, and overlay those in the embedding space you do words That allows LLMs to treat images like "words" and speak on their contents. - That being said looking at the demo I'm not sure Bard isn't just using the video transcripts. I was using them to guide seeking, but you could answer all the test questions shown in the video without image understanding