I think the running commentary is interesting. It sounds like a person talking through their thought process out loud. This is how Molmo’s dataset, PixMo was c

I think the running commentary is interesting. It sounds like a person talking through their thought process out loud. This is how Molmo’s dataset, PixMo was created – by recording human annotators describing the image out loud. I wonder if this is how QvQ was trained as well? > we ask annotators to describe images in speech for 60 to 90 seconds rather than asking them to write descriptions. We prompt the annotators to describe everything they see in great detail and include descriptions of spatial positioning and relationships. Empirically, we found that with this modality switching "trick" annotators provide far more detailed descriptions in less time, and for each description we collect an audio receipt (i.e., the annotator's recording) proving that a VLM was not used. — https://molmo.allenai.org/blog https://molmo.allenai.org/blog