"We introduce mmc4, a corpus of 585M images interleaved in 43B English tokens from the popular c4 dataset." This training corpus mmc4 is the one used by OpenFl

"We introduce mmc4, a corpus of 585M images interleaved in 43B English tokens from the popular c4 dataset." This training corpus mmc4 is the one used by OpenFlamingo (https://laion.ai/blog/open-flamingo/ https://laion.ai/blog/open-flamingo/). Something caught my eye when I read the pdf. They are using GPT-4 to name their word clusters: "Topic names are generated by GPT-4 conditioned on the top 20 words for each topic, prompted by a request for a short 1-2 word summary." I feel like LLMs will be used increasingly for this purpose; whenever someone has to make a mostly arbitrary decision like the kind that would make 'bike shedding arguments', they can delegate it to the LLM. The point isn't that the LLM is necessarily better or objective, but rather that it's a kind of 'schelling point' for agreeing on things that don't matter. For example I could imagine a super annoying peer reviewer (or even co-author) ask why you used the name 'celebrations' as a topic name associated to the set of words 'fun, wedding, beautiful, christmas, happy, card, birthday, gift, blog, perfect' like in the pdf, and why not use some other word. Instead of having some meaningless back and forth 'bike shedding' discussion with the reviewer over email, you can just say you used GPT-4 and move on to more important things.