4 ms·
Anyone else read the book that the author mentions, Build a Large Language Model (from Scratch) [0]? After watching Karpathy's video [1] I've been looking for a
by rsanek 1y ago
Anyone else read the book that the author mentions, Build a Large Language Model (from Scratch) [0]? After watching Karpathy's video [1] I've been looking for a good source to do a deeper dive.
[0] https://www.manning.com/books/build-a-large-language-model-from-scratch https://www.manning.com/books/build-a-large-language-model-f...
[1] https://www.youtube.com/watch?v=7xTGNNLPyMI https://www.youtube.com/watch?v=7xTGNNLPyMI
- kamranjon 1y agoIt’s good - I’m working through it right now
- ForceBru 1y agoYes, it's really good
- tra3 1y agoIs [1] worth a watch if I want to get a high level/basic understanding of how LLMs work?
- rsanek 1y agoYeah, it's very well done
- malshe 1y agoHere is the code used in the book - https://github.com/rasbt/LLMs-from-scratch https://github.com/rasbt/LLMs-from-scratch
- horizion2025 1y agoIs there a non-video equivalent. I always prefer reading/digesting at my own pace compared to following a video.
- gpjt 1y agoCheck the first link in the parent comment, it's a link to the book.
- tanelpoder 1y agoYes, can confirm, the book is great. I was also happy to see that the author correctly (in my mind) used the term “embedding vectors” vs. “vector embeddings” that most others seem to use… Some more context about my pet peeve: https://tanelpoder.com/posts/embedding-vectors-vs-vector-embeddings/ https://tanelpoder.com/posts/embedding-vectors-vs-vector-emb...
- gchadwick 1y agoI thought it was a great book, dives into all the details and lays it out step by step with some nice examples. Obviously it's a pretty basic architecture and very simplistic training but I found it gave me the grounding to then understand more complex architectures.