4 ms·
I guess what I find really interesting is, how come we can start to self-label data we encounter in the wild, yet the DNN needs data to constantly be labeled at
by supernova87a 4y ago
I guess what I find really interesting is, how come we can start to self-label data we encounter in the wild, yet the DNN needs data to constantly be labeled at the same intensity per image, into the billions?
- simonw 4y agoIf you look at the text labels for the data used by Stable Diffusion you'll find that they are very low quality. Take a look at some here: https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/images https://laion-aesthetic.datasette.io/laion-aesthetic-6pls/im... Clearly quality of labeling isn't nearly as important once you are training on billions of images.
- fragmede 4y agoTextual Inversion is about to flip that around (if I'm understanding the paper correctly). https://textual-inversion.github.io/ https://textual-inversion.github.io/
- andreyk 4y agoThis is only true for multi modal learning, but yeah, in that case we need text and image pairs. More than likely it's possible to pretrain image and language separately, and then use a vastly smaller number of pairs. But that's hypothetical.