4 ms·
I still don’t understand the process of learning ML, like sure we build micrograd but is it only didactic exercise or can we use it to train it to do something
by coolThingsFirst 3y ago
I still don’t understand the process of learning ML, like sure we build micrograd but is it only didactic exercise or can we use it to train it to do something serious on our own hardware?
- nextos 3y agoThis is like building a tiny C compiler. At some program scale, optimizations become important.
- pmelendez 3y ago>do something serious I guess it depends on what you mean by serious. Pre-training a competitive LLM with current methods and consumer hardware is prohibitive for sure. Solving a classification problem could be totally doable depending on the domain.
- edgyquant 3y agoI don’t understand this comment, for one we’re engineers/hackers and should be curious how this stuff works. It’s exciting. Practically speaking this is like asking why learn how to write a simple forum or blog when we can’t host Facebook on our on hardware: it’s going to be hard to work on the latest models if you don’t first understand the basics.
- coolThingsFirst 3y agoYou learn what Db is you can use a db for your own custom task. What so i do with micrograd? Hang the code on my wall?
- edgyquant 3y agoYes exactly. Learn this so you can then apply it to your own software
- sva_ 3y agoYou'll probably need some hardware acceleration. There's a good course that builds something like micrograd in the beginning and extends on it: https://dlsyscourse.org/lectures/ https://dlsyscourse.org/lectures/
- coolThingsFirst 3y agoI have a 3060 Ti, that enough?
- sva_ 3y agoIt should be worth to do these kind of hardware accelerations with any kind of hardware that has more TFlop/s (and memory bandwidth) than your CPU basically. I mean, enough for what?
- genman 3y ago8 GB of memory is a little restrictive but still usable for many problems like visual classification problems. Training would take a little more time though as you can't do as much batch processing as it would be possible with more memory.
- genman 3y agoYou can totally do some visual classification problems (like object detection) on current consumer hardware. Even more. You can also take some smaller existing language models and fine tune them for some special task - also completely feasible.