3 ms·

I think this should be discussed more. Models that can only be accessed via API cannot be tested without giving their owners access to the test data. You just h

I think this should be discussed more. Models that can only be accessed via API cannot be tested without giving their owners access to the test data. You just have to trust that they’ll do the right thing.

Tepix 2y ago
In particular, in cases where the model gets 16 hours to solve a task that a human can solve in a few minutes, cheating is trivial!