Using 3.5 seems like it would have been okay (technically but not legally given OpenAI ToS), but Mixtral is already somewhere around that level, so it's not ach

Using 3.5 seems like it would have been okay (technically but not legally given OpenAI ToS), but Mixtral is already somewhere around that level, so it's not achieving a goal of improving towards GPT-4 level responses without having to write those yourself, which seems to have been the goal OP wanted to get to. I've heard that training e.g. Mixtral on Mixtral's own outputs is a really bad idea, don't know full details.