4 ms·

Maybe it's just me, but I'm not doing anything that calls itself 'zero to hero'. Would love some good resources (preferably textbook, or at least written) on LL

by OJFord 3y ago

Maybe it's just me, but I'm not doing anything that calls itself 'zero to hero'. Would love some good resources (preferably textbook, or at least written) on LLMs though. I don't even understand the link to 'generative' image/video AI, which seems to have exploded at roughly the same time and surely isn't a coincidence. I studied a little (literally 'intro to') ML at university, about enough to grok it as an application of stats, tie into things seen elsewhere, but not really more than that. Every supposéd tutorial or explainer I've seen posted here or been able to find has been a weird (IMO) mix if simultaneously assuming a decent (at least greater than mine) ML background, but also really dumbed down clone this repo download that model switch between them like this, fine-tune them by cd'ing to this directory and ... Ok but what's actually going on?

fenomas 3y ago
The zero to hero video series is what you're looking for - look past the name and watch it. It's excellent.
frontier 3y ago
Karpathy's series is many many hours long and really does take you from zero to GPT. It's excellent! You sound triggered by the title - that may not even be the official title - but it definitely deserves it. Go look it up.
- OJFord 3y ago
  The title suggests I wouldn't like it, yes. But as a video series it's not 'a textbook or at least written' is i - not really the format I'm looking for personally.
  - thatcherthorn 3y ago
    Truly.. one of the greatest minds in our ML era. Don't get caught up on the format :)
    - OJFord 3y ago
      I just don't find it an effective way of learning personally. I didn't expect this to be so controversial - different people learn differently.
      - quickthrower2 3y ago
        I recommend this for groundwork to get you near LLM, and cover the journey deeply. I used some of this as a helper course for Karpathy. I learned things here he didn’t cover and vice versa. https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/ https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/ I haven’t done tonnes of courses so there might be better. But this is good as a free one.
        OJFord 3y ago
        Thanks!
  - frontier 3y ago
    Fair enough if you prefer to slog through an entire textbook. But for anyone else.. I can't recommend this series more highly, it was just amazing, no filler, pure step after step, explained methodically to the end goal.
    - quickthrower2 3y ago
      Lol I had to slog through some traditional material to keep up with Karpathy. A lot is covered in those videos.
antonvs 3y ago
> I'm not doing anything that calls itself 'zero to hero'. Sounds like you have a case of the Mondays. You just need to turn that frown upside down!
- OJFord 3y ago
  Must have left my flair at home.
pavs 3y ago
Since you are "judging a book by its cover", or this case a name. This might interest you, that karpathy was co-founding developer of OpenAI, left to work at Tesla to head their AI development for 5-ish years and now back at OpenAI. I can understand that you might be interested in book form only, I was lile this for the longest time, until I bumped into some really high quality video series that changed my mind to be a bit more flexible. Also Important to note that LLM development is moving at a very fast pace recently so a book form might not be ideal. The basic ideas might be same, most of it might be out of date 6-12 months from now. I dont see how anyone could write a quality book that covers everything on this.
- OJFord 3y ago
  I realise that; also that it doesn't help that afaiui it's been more industry-led than academia. But I truly am starting from pretty much 'zero', and maybe I wasn't clear but I'm not looking to be 'hero' in the sense of up to date with the cutting edge, or even necessarily putting anything in to practice at all, I'm more interested in the background theory, and fine with that missing the absolute latest extra technique, just want to understand the meat of it better. A refresher on SVMs & PCA (which I barely remember - I think I could convincingly explain SVMs to someone numerate but non-tech/mathematician, but not otherwise) and then a catch up to roughly what's going on with LLMs & image/video as mentioned would be great. > I can understand that you might be interested in book form only, I was lile this for the longest time, until I bumped into some really high quality video series that changed my mind to be a bit more flexible. I enjoy videos for many things, but mostly entertainment, I don't personally find I can learn that well from them, especially more technical/theoretical stuff, sure to some combination of screen fatigue, it being harder to skip around and reference something, and distraction - something seems obvious briefly so my mind wonders, check something in another tab 'just quickly', and before you know it ten minutes have passed, I've been hearing the speaking but suddenly realise I haven't been listening, have no idea what's going on any more. Obviously they work for some people, that's fine.
- quickthrower2 3y ago
  Karpathy has flash attention of Pytorch in his repo. I understand that is fairly recent (in human weeks, maybe not AI dog weeks)

by OJFord 3y ago

fenomas 3y ago

The zero to hero video series is what you're looking for - look past the name and watch it. It's excellent.

frontier 3y ago

Karpathy's series is many many hours long and really does take you from zero to GPT. It's excellent! You sound triggered by the title - that may not even be the official title - but it definitely deserves it. Go look it up.

OJFord 3y ago
The title suggests I wouldn't like it, yes. But as a video series it's not 'a textbook or at least written' is i - not really the format I'm looking for personally.
- thatcherthorn 3y ago
  Truly.. one of the greatest minds in our ML era. Don't get caught up on the format :)
  - OJFord 3y ago
    I just don't find it an effective way of learning personally. I didn't expect this to be so controversial - different people learn differently.
    - quickthrower2 3y ago
      I recommend this for groundwork to get you near LLM, and cover the journey deeply. I used some of this as a helper course for Karpathy. I learned things here he didn’t cover and vice versa. https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/ https://www.cs.toronto.edu/~rgrosse/courses/csc321_2018/ I haven’t done tonnes of courses so there might be better. But this is good as a free one.
      - OJFord 3y ago
        Thanks!
- frontier 3y ago
  Fair enough if you prefer to slog through an entire textbook. But for anyone else.. I can't recommend this series more highly, it was just amazing, no filler, pure step after step, explained methodically to the end goal.
  - quickthrower2 3y ago
    Lol I had to slog through some traditional material to keep up with Karpathy. A lot is covered in those videos.

antonvs 3y ago

> I'm not doing anything that calls itself 'zero to hero'. Sounds like you have a case of the Mondays. You just need to turn that frown upside down!

OJFord 3y ago
Must have left my flair at home.

pavs 3y ago

Since you are "judging a book by its cover", or this case a name. This might interest you, that karpathy was co-founding developer of OpenAI, left to work at Tesla to head their AI development for 5-ish years and now back at OpenAI. I can understand that you might be interested in book form only, I was lile this for the longest time, until I bumped into some really high quality video series that changed my mind to be a bit more flexible. Also Important to note that LLM development is moving at a very fast pace recently so a book form might not be ideal. The basic ideas might be same, most of it might be out of date 6-12 months from now. I dont see how anyone could write a quality book that covers everything on this.

OJFord 3y ago
I realise that; also that it doesn't help that afaiui it's been more industry-led than academia. But I truly am starting from pretty much 'zero', and maybe I wasn't clear but I'm not looking to be 'hero' in the sense of up to date with the cutting edge, or even necessarily putting anything in to practice at all, I'm more interested in the background theory, and fine with that missing the absolute latest extra technique, just want to understand the meat of it better. A refresher on SVMs & PCA (which I barely remember - I think I could convincingly explain SVMs to someone numerate but non-tech/mathematician, but not otherwise) and then a catch up to roughly what's going on with LLMs & image/video as mentioned would be great. > I can understand that you might be interested in book form only, I was lile this for the longest time, until I bumped into some really high quality video series that changed my mind to be a bit more flexible. I enjoy videos for many things, but mostly entertainment, I don't personally find I can learn that well from them, especially more technical/theoretical stuff, sure to some combination of screen fatigue, it being harder to skip around and reference something, and distraction - something seems obvious briefly so my mind wonders, check something in another tab 'just quickly', and before you know it ten minutes have passed, I've been hearing the speaking but suddenly realise I haven't been listening, have no idea what's going on any more. Obviously they work for some people, that's fine.
quickthrower2 3y ago
Karpathy has flash attention of Pytorch in his repo. I understand that is fairly recent (in human weeks, maybe not AI dog weeks)