3 ms·
My YouTube videos fall into two tracks: 1. technical track (all the GPT repro series) 2. general audience track For (2), I had a 1hr video from 1 year ago, b
by karpathy 2y ago
My YouTube videos fall into two tracks:
1. technical track (all the GPT repro series)
2. general audience track
For (2), I had a 1hr video from 1 year ago, but I didn't actually expect that video to be some kind of authoritative introduction to LLMs. The history is that I was invited to give an LLM talk (to general audience), prepared some random slides for a day, gave the talk, and then re-recorded the talk in my hotel room later in a single take, and that become the video. It was quite random and haphazard. So I wanted to loop back around more formally and do a more comprehensive intro to LLMs for general audience; Something I could for example give to my parents, or a friend who uses ChatGPT all the time and is interested in it, but doesn't have the technical background to go through my videos in (1). That's this video.
- gdjdkslslp 2y agoI haven’t watched this video yet, but do you plan to create any technical videos in the (1) series on RL in LLMs?
- amelius 2y agoThis would be very welcome as it brings us closer to understanding the secret sauce behind training a real, practical LLM.
- MasterScrat 2y agoHis intro to RL (not for LLM) blog post is a great read FYI https://karpathy.github.io/2016/05/31/rl/ https://karpathy.github.io/2016/05/31/rl/
- deleted 2y ago[deleted]
- gardenhedge 2y agoI watched that one after finding it randomly! Will give the new one a watch too
- gnunez 2y agoGreat work! I love your videos; they've taught me so much. Any plans for a Mixture of Experts (MoE) video? My understanding is that starting from GPT4 most advance models use MoE to some extent. For example, can I take the model from your GPT2 video and just change the feed forward layer to an MoE layer like the one found here (1)? I guess I can just try it myself but I enjoy the expert guidance you provide in your videos. Please don't stop! great content! 1. https://github.com/mistralai/mistral-inference/blob/main/src/mistral_inference/moe.py https://github.com/mistralai/mistral-inference/blob/main/src...
- yingliu4203 2y agoBoth are the best videos in NN and LLM. Your kindness is highly appreciated.
- pj_mukh 2y ago"Something I could for example give to my parents," Not for nothing, Andrej's parents maybe different from mine, but I definitely can't send this to my parents.
- deleted 2y ago[deleted]
- carbine 2y agothank you SO much for all your hard work, we sincerely appreciate it