One argument is that it can contaminate training data from output of itself or other models. We have already documented evidence of the effect of this. In the

One argument is that it can contaminate training data from output of itself or other models. We have already documented evidence of the effect of this. In the GPT-4 technical report [1], they reported contamination of humaneval data in the training data. They did measure against a "non-contaminated" training set but no idea if that can still be trusted. Why would this matter? We can have seemingly strong benchmarks for containments but measures poorly against new and quarantined information. Classic over fitting. Another argument is that data being put out there could very much be wrong and the amounts of it amplified by other models. Take a look at this sample of demonstration data for codealpaca [2]. Not only is its output wrong but bad practices like,making up a random computation without it having access to a place to run a calculation, teaches the model these type of responses are ok. { "instruction": "What would be the output of the following JavaScript snippet?", "input": "let area = 6 * 5;\nlet radius = area / 3.14;", "output": "The output of the JavaScript snippet is the radius, which is 1.91." } 1: https://cdn.openai.com/papers/gpt-4.pdf https://cdn.openai.com/papers/gpt-4.pdf 2: https://github.com/sahil280114/codealpaca/commit/0d265112c705f88848783dfdf9bdca9df64d8e35#diff-0585315a2292ca9c3186c41e52ce9427a39b5f595d4bee6aae15e9443c95051b https://github.com/sahil280114/codealpaca/commit/0d265112c70...