5 ms·
Does anyone have a good recommendation for a book that would cover the underlying ideas behind LLMs? Google ends up giving me a lot of ads, and ChatGPT is vagu
by Zetice 4y ago
Does anyone have a good recommendation for a book that would cover the underlying ideas behind LLMs? Google ends up giving me a lot of ads, and ChatGPT is vague about specifics as per usual.
- danenania 4y agoNot a book, but here's a really good explanation in blog post form from Stephen Wolfram: https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
- Zetice 4y agoI do not trust that man one iota.
- seizethecheese 4y agoThe blog post is very good.
- cjblack 4y agoWhy?
- Zetice 4y agoHe's got a habit of self aggrandizing, antagonism, and deception in an effort to promote himself and his brand, I worry that his explanations are designed to maximally benefit him, rather than to maximally explain the topic. He's a brilliant man, I just don't trust him.
- bulkprotocol 4y agoIs that the case with this specific article?
- Zetice 4y agoThat’s not how trust works.
- defgeneric 4y agoI agree generally but read the post and it only mentions cellular automata briefly and promotes Wolfram Alpha once. Overall it's very good at moving from Markov chains to neural nets with decent examples and graphics.
- olooney 4y agoThe 2nd edition of Deep Learning with Python[1] (by François Chollet of Keras fame) has a pretty good chapter on the transformer architecture[2]. In fact, the explanations in the book are clear and lucid across the board. [1]: https://www.amazon.com/Deep-Learning-Python-Francois-Chollet/dp/1617294438 https://www.amazon.com/Deep-Learning-Python-Francois-Chollet... [2]: https://en.wikipedia.org/wiki/Transformer_(machine_learning_model) https://en.wikipedia.org/wiki/Transformer_(machine_learning_...
- ww520 4y agoML these days have moved forward fast that books are outdated after couple years. Look at articles, podcasts, youtube videos, and research papers for the latest changes. ChatGPT utilizes existing algorithms. Its insight is scaling the weight parameters up to billions, and human feedback & guidance help a lot in finetuning the model to be user friendly to general users. To understand LLM from ground up, the following topics would help. - Machine Learning basics. e.g. weight parameters being trained. - Neural Net basics. - Nature Language Processing basics. - Word vectorization, word embedding. e.g. Word2Vec. - Recurrent Neural Net basics. - LSTM model. - Attention and Transformer model. - Generative model like GAN. - Generative Pre-trained Transformer. I might miss a few topics. Actually ask ChatGPT to explain each topic. See how far it goes.
- giardini 4y agoOr you could save some time by clicking on: "What Is ChatGPT Doing … and Why Does It Work?" https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-... graciously provided above in this discussion by danenania. As seizethecheese asserts, also above, "The blog post is very good."