2 ms·
This is, unfortunately, standard practice on generative papers. Best is to show cherry picked and random selection. But if you don't cherry pick you don't get p
by godelski 4y ago
This is, unfortunately, standard practice on generative papers. Best is to show cherry picked and random selection. But if you don't cherry pick you don't get published. Even having random samples can put you in a tough position.
To be fair, metrics for vision are limited and many don't know how to correctly inturpret them. You both want to know what's the best images the network can produce a well as the average. Reviewers, unfortunately, just want to reject (no incentive you accept). That said, it can still be a useful tool because you know the best (or near) that the model can produce. This is also why platforms like HuggingFace are so critical.
I do think we should ask more of the research community but just know that this is in fact normal and even standard. Most research judge by things like citations and just reading the works themselves. The more hype in a research area the more noisy the signal for good research. (This is good research imo fwiw)