For those interested in micro/tiny/small models, a couple more links. https://arxiv.org/abs/1909.11687 https://arxiv.org/abs/1909.11687 https://arxiv.org/abs/

For those interested in micro/tiny/small models, a couple more links. https://arxiv.org/abs/1909.11687 https://arxiv.org/abs/1909.11687 https://arxiv.org/abs/2210.07904 https://arxiv.org/abs/2210.07904 I wrote an article on the subject earlier this year - https://neuml.hashnode.dev/train-a-language-model-from-scratch https://neuml.hashnode.dev/train-a-language-model-from-scrat...