Neat demo. It seems like there may be a few things happening in tandem. I uploaded a picture of a forest and it came back with visually similar images. So the

Neat demo. It seems like there may be a few things happening in tandem. I uploaded a picture of a forest and it came back with visually similar images. So the first thing it might be doing is some kind of KNN, and if the pictures have location labels associated applying some sort of weighted average to determine GPS coordinates. This is pretty cool. I also tried flipping the image horizontally, and it came back with the same images. So their embedding isn't based off of exact matches (good) and seems to be invariant to some basic translations (good). It also seems like it's directly extracting visual features from the image. This can be done with something like Blip[0]. Then I uploaded a screenshot from Magic School Bus. It still extracted information to guess the "location" of the cartoon (San Francisco, which is wrong). So that's probably how it works. I also found the text output is similar in some ways with OpenGVLab InternViT [1]. So perhaps this or something like it is being used to extract features. And of course there may be an LLM on top of these extracted features with some sort of prompt template. But I should add that the text explanation is the least useful part of the result, since it is unreliable and less informative than the "boring" similarity metrics above. [0] https://huggingface.co/Salesforce/blip-image-captioning-large https://huggingface.co/Salesforce/blip-image-captioning-larg... [1] https://internvl.opengvlab.com/ https://internvl.opengvlab.com/