4 ms·
The real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.
by crooked-v 3y ago
The real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.
- brucethemoose2 3y agoNot anymore. Llama.cpp and kobold.cpp run well with low vram and non Nvidia GPUs. Huggingface is full of finetunes now, and I believe a 33b model can be finetuned on a single 3090. Llama.cpp is developing some kind of training, but I have no idea what the requirements will be.
- causality0 3y agoTo me, GPT's real "secret sauce" is how it primes the model for interactive prompting instead of just text completion.
- ynniv 3y agoThere are "chat" and "instruct" variants of most models now.