Will this dataset be able to produce a model that's actually better than GPT-3? One of the things mentioned in the paper is that the dataset had some buggy fil

Will this dataset be able to produce a model that's actually better than GPT-3? One of the things mentioned in the paper is that the dataset had some buggy filtering applied to it (https://arxiv.org/pdf/2005.14165.pdf https://arxiv.org/pdf/2005.14165.pdf, page 9), that minimally impacted benchmarking some, there's a whole section on how they tried to deal with it. The gist though is that a cleaner training run, even on slightly more data, may (should?) actually produce something a bit better. Does anyone know if OpenAI has retrained/updated gpt-3 yet?