I'd be surprised if they didn't train at least partially on some GPT 4 synthetic data. But it is interesting that for example Mistral 7B Instruct v0.1 would ver

I'd be surprised if they didn't train at least partially on some GPT 4 synthetic data. But it is interesting that for example Mistral 7B Instruct v0.1 would very clearly and consistently state it was made in Paris by Mistral.AI and the v0.2 version couldn't tell you what it was or where it came from to save its life. The fine tuning for that must be very finicky.