2 ms·
It would be great if someone created a tutorial around this explaining exactly how it works and how to do a test training run. I’m aware it’s not feasible to tr
by blackeyeblitzar 3y ago
It would be great if someone created a tutorial around this explaining exactly how it works and how to do a test training run. I’m aware it’s not feasible to train a “real” model on personal hardware but it would be nice to have a practical learning experience. I’m not sure if there are good alternatives for that.
- vineyardmike 3y agoThe author has a whole series where he does exactly that. YouTube videos, code examples, documentation, everything. Explains the math, explains how to code it, explains the architecture. Everything.
- karpathy 3y agoI wrote this, which might be a bit helpful: https://github.com/karpathy/llm.c/blob/master/doc/layernorm/layernorm.md https://github.com/karpathy/llm.c/blob/master/doc/layernorm/... But if you don't have the background, I'd recommend my YouTube videos, see the Zero To Hero playlist: https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxbuWI23v9cThsA9GvCAUhRvKZ https://www.youtube.com/watch?v=VMj-3S1tku0&list=PLAqhIrjkxb...
- blackeyeblitzar 3y agoThank you so much for responding. I will definitely check these out and also pass it on to others who might be interested.
- MAMAMassakali 3y agoThank you so much for the Zero To Hero playlist!