3 ms·
How do we know Mitchell's counterexample isn't in GPT-4's training set? To truly test GPT-4 you must be sure to give it a problem which is not in its training
by the_af 4y ago
How do we know Mitchell's counterexample isn't in GPT-4's training set?
To truly test GPT-4 you must be sure to give it a problem which is not in its training set and which differently worded enough from anything it can recall. A variation that a human would understand but GPT-4 wouldn't.
I bet this can still be done, it's just that this particular example is now tainted.