3 ms·
You should publish your evaluation set, that seems pretty interesting! What’s your favourite one?
by hmokiguess 8mo ago
You should publish your evaluation set, that seems pretty interesting!
What’s your favourite one?
- iainmerrick 8mo agoWhy would you ask that? The whole point of making it private is to avoid it leaking into the training data.
- hmokiguess 8mo agoI thought open benchmarks helped, sorry, guess I was being naive.
- iainmerrick 8mo agoHa, sorry, I was a bit brusque there. Open benchmarks do help, but they mostly help the vendors, not we the users!
- Espressosaurus 8mo agoKeeping tests private is the only way to keep them valid.