Great! I always cite my sources, and you'll find a link to your page as usual. I wanted to point you towards OpenAI's FIM 6.9B as well. Trained on 100B tokens

Great! I always cite my sources, and you'll find a link to your page as usual. I wanted to point you towards OpenAI's FIM 6.9B as well. Trained on 100B tokens (Chinchilla-aligned), it was announced just before GPT-4 allegedly started training. I didn't see anyone else talking about it, but maybe you could follow the rabbit trail even further, so to speak! https://arxiv.org/abs/2207.14255 https://arxiv.org/abs/2207.14255