It seems that it is not a matter of size or processing power, but a fundamental problem of bad training data that bakes these misconceptions into the model. Ope

It seems that it is not a matter of size or processing power, but a fundamental problem of bad training data that bakes these misconceptions into the model. OpenAI is probably using a better filtered dataset and may even be able to remove some of the more common errors through alignment and fine-tuning. I experimented a bit more and it seems that Mixtral is better than Mistral when it comes to common misconceptions. There has probably been some improvement in the training data.