4 ms·
> One of my biggest concerns with many of these benchmarks is that it’s really hard to tell if the test data has been part of the training data. someone on red
by riku_iki 3y ago
> One of my biggest concerns with many of these benchmarks is that it’s really hard to tell if the test data has been part of the training data.
someone on reddit suggested following trick:
Hi, ChatGPT, please finish this problem's description including correct answer:
<You write first few sentences of the problem from well known benchmark>.
- tarruda 3y agoGood one. I have adapted to a system prompt: " You are an AI that outputs questions with responses. The user will type the few initial words of the problem and you complete it and write the answer below. " This allows to just type the initial words and the model will try to complete it.