3 ms·
How is text to image even scored? Seems like a subjective measurement..
by brcmthrowaway 6mo ago
How is text to image even scored? Seems like a subjective measurement..
- be7a 6mo agoUsers get two completions for their prompt and rank them. From this you can then use Bradley-Terry to get Elo scores per model.
- vunderba 6mo agoLM Arena is a particularly bad comparison site too. Prompts that they use are usually incredibly generic like "A digital render of a sleek, futuristic motorcycle racing through a neon-lit cityscape." I actually built GenAI Showdown a while back because I was deeply unsatisfied with LM Arena and other purported comparison tables which either (A) relied solely on visual fidelity (which is a far less interesting benchmark than adherence, IMHO) and/or (B) relied on extremely simplistic and banal prompts.