4 ms·

Note that current GPT-4 pass@1 for HumanEval is closer to 90% than to 67% reported in GPT-4 technical report, as reported, e.g., in [1] [1] https://arxiv.org/a

Note that current GPT-4 pass@1 for HumanEval is closer to 90% than to 67% reported in GPT-4 technical report, as reported, e.g., in [1] [1] https://arxiv.org/abs/2305.01210 https://arxiv.org/abs/2305.01210

nabakin 3y ago
Good point, I guess Meta should be using that number in their chart
cosmojg 3y ago
Unfortunately, we have no idea whether GPT-4 has been further trained or finetuned on contaminated data since then.