3 ms·
SOTA does not require being productionized. eg. GPT-3 was SOTA and it was not publicly accessible.
by charcircuit 3y ago
SOTA does not require being productionized. eg. GPT-3 was SOTA and it was not publicly accessible.
- nightski 3y agoThere has to be some way to verify the claim. Trust me bro isn't science.
- gpm 3y ago"Trust that I ran these tests with these results" is extremely common in science.
- nightski 3y agoIt's not an objective test like you are talking about. These benchmarks are far from accurate and also can be tainted in the training data.
- verdverm 3y agoYou'll find the same thing in many academic/scientific papers
- deleted 3y ago[deleted]
- hughesjj 3y agoThe trust is established by others reproducing the results with the same methodology, it's not just supposed to be taking people's word at face value