3 ms·
what is your setup for drawing pelican? Do you ask model to check generated image, find issues and iterate over it which would demonstrate models real abilities
by andriy_koval 6mo ago
what is your setup for drawing pelican? Do you ask model to check generated image, find issues and iterate over it which would demonstrate models real abilities?
- simonw 6mo agoIt's generally one-shot-only - whatever comes out the first time is what I go with. I've been contemplating a more fair version where each model gets 3-5 attempts and then can select which rendered image is "best".
- andriy_koval 6mo agoI think it will make results way better and more representative of model abilities..
- simonw 6mo agoIt would... but the test is inherently silly, so I'm still not sure if it's worth me investing that extra effort in it.
- irthomasthomas 6mo agoTry llm-consortium with --judging-method rank