3 ms·
Hi Antoine! Interesting point about backend variance. Do you think serving layer should become part of standard LLM eval reporting?
by rebekkamikkoa 5mo ago
Hi Antoine!
Interesting point about backend variance. Do you think serving layer should become part of standard LLM eval reporting?
- zambelli 5mo agoHi! Yes, I definitely think so. I've seen variance across all model families I looked at. The magnitude changes, but the presence of variance is a constant.