3 ms·
I think for someone who hasn’t seen the material at all before it would be a lot for a semester. They don’t know what backpropagation is but by the end will und
by janalsncm 2y ago
I think for someone who hasn’t seen the material at all before it would be a lot for a semester. They don’t know what backpropagation is but by the end will understand a diffusion model? It’s ambitious, I think.
The other thing is this seems to be very CNN heavy. Four lectures on the topic seems like a lot.
Also, I don’t see embeddings explicitly mentioned as a topic. They’re a huge component of industrial research, and creating good embeddings and retrieving them quickly is a topic I feel students should also be exposed to. (Yes, they mention “representation” with autoencoders but quite frankly the code bit is generally not useful for similarity metrics.)
Finally, it would be nice to expose students to multimodal learning. Something like CLIP would be pretty neat to expose students to. It’s a great insight when you realize that you can train projections of multiple modalities into a shared high dimensional space. If they’re going to cover diffusion models certainly complexity isn’t a concern.
- diebeforei485 2y ago> I think for someone who hasn’t seen the material at all before it would be a lot for a semester. They don’t know what backpropagation is but by the end will understand a diffusion model? It’s ambitious, I think. Welcome to CMU :)
- deleted 2y ago[deleted]
- mliker 2y ago> They don’t know what backpropagation is but by the end will understand a diffusion model? That seems plausible to learn in a semester long course, especially at an institution like CMU
- berniep 2y agoCan you recommend any tutorials/resources about designing and training simple multi-modal models like CLIP? Or should I just be reading the papers and following along?
- janalsncm 2y agoWell, the core idea is to train a text encoder and an image encoder jointly with in-batch negatives. In other words, a two tower model maximizing the diagonal and minimizing everything else. They have pseudo code in the paper.