3 ms·
I notice a surprising number of duplicates. E.g. if I sort by aesthetic, there’s the same 500x500 Tuscan village painting multiple times on the first page of re
by yojo 4y ago
I notice a surprising number of duplicates. E.g. if I sort by aesthetic, there’s the same 500x500 Tuscan village painting multiple times on the first page of results.
Presumably it wouldn’t be so hard to hash the images and filter out repeats. Is the idea to keep the duplicates to preserve the description mappings?
- bambax 4y agoI noticed this too, with the same description every time. How does this work in the model, does this give repeated images a bigger weight? It's surprising that these weren't filtered out, and it would be interesting to know the number of unique images. (When it is mentioned that a model was trained on 10 billion images for example, obviously if each image is repeated 5 times then the actual number of images is 2 billions, not 10.)