3 ms·
> Is it giving an unfair advantage to Model X if we use Model X as the judge? There have been studies that showed that models tended to rate responses from the
by sebastiennight 4mo ago
> Is it giving an unfair advantage to Model X if we use Model X as the judge?
There have been studies that showed that models tended to rate responses from their own family of models better than equivalent responses from other families, eg. gpt-4 would prefer a response from gpt-3
- LiamPowell 4mo agoI suspected as much, and that brings us to the second issue where if we use a cohort of judges then the model that likes it's own code the most still wins.