4 ms·

The one you linked to is based on Karpathy's tutorial: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY, except changing

by zerd 2y ago

The one you linked to is based on Karpathy's tutorial: https://www.youtube.com/watch?v=kCc8FmEb1nY https://www.youtube.com/watch?v=kCc8FmEb1nY, except changing to train on tinystories in stead of Shakespeare. smolGPT also looks inspired by nanoGPT, also from Karpathy: https://github.com/karpathy/nanoGPT/blob/master/train.py https://github.com/karpathy/nanoGPT/blob/master/train.py