2 ms·
I'm with you but I'm yet to find a solution that isn't both rocket science and requiring rocket hardware.
by keyle 3y ago
I'm with you but I'm yet to find a solution that isn't both rocket science and requiring rocket hardware.
- satvikpendem 3y agoWhat do you mean?
- selfhoster11 3y agoNot in my experience. For an ML hobbyist, I don't have particularly strong hardware - around 40 GB of system RAM and a very small GPU with 8GB of RAM. I can run all sizes of models, up to and including 70B (albeit slowly). 13 is definitely the most usable on this particular hardware, given the trade-off between model capabilities and generation speed.