4 ms·
What is your goal ? 1) Know more about how they work on the academic side ? 2) Be able to work in a company that work on LLM ? 3) Be able to work with LLMs 4) a
by cdavid 3y ago
What is your goal ? 1) Know more about how they work on the academic side ? 2) Be able to work in a company that work on LLM ? 3) Be able to work with LLMs 4) agents ? Each of those goal may require different learning "streams"
The book "NLP with transformers", or fast.ai is good for 3). For 1), assuming you do know how they work, I recommend you start reading papers.
I find the discussions around "prompt engineering" to be rather pointless and they are quickly obsolete anyway (newer, more powerful LLMs makes it more and more obvious)
- bdamirch 3y agoIdeally my goal is a mix of 2 and 3. I think agents are quite interesting, but maybe something I’d look at later.
- FezzikTheGiant 2y agoI'm in the exact same situation as you rn. Freshman year of college and really interested in LLMs. Did you decide how you're gonna go about it? Maybe we could share resources
- bdamirch 2y agoI'm focused on finals at the moment but I'd be more than happy to share what I've been looking at so far (I plan to look into this a lot more in the summer). When it comes to the theory, 3blue1brown released some really nice videos that solidified a lot of my current understanding, especially with the attention mechanism. I think I am also going to do a mixture of reading papers/watching youtube videos on things that are interesting (ex. Qlora for fine-tuning or diffusion models for image generation) and trying to build out a simple implementation myself and see where that takes me. Maybe I'll start out with the Karpathy nano-gpt videos but try and do it with a different data set. But for people like myself who lack the math background, data, and compute to be able to train very strong LLMs, I think it is also a good idea to try and build some projects/apps that use a fine-tuned LLM, or just call the OpenAI API. I'm still a bit lost myself, but in 2 weeks when I'm done with exams, I'm more than happy to keep exchanging resources with you.
- FezzikTheGiant 2y agoPerfect. I have my own finals in a couple weeks haha, planning to look more into this and possibly build a project over the summer. 3B1B's videos sound like a good resource, and the nano-GPT videos look really useful too. I'll share what I come across as well!