2 ms·
You mean for finetuning an LLM? PyTorch + QLORA is very affordable. Otherwise JAX on TPUs is a reasonable way to finetune on a large dataset. I'm not sure abo
by brucethemoose2 3y ago
You mean for finetuning an LLM?
PyTorch + QLORA is very affordable.
Otherwise JAX on TPUs is a reasonable way to finetune on a large dataset. I'm not sure about the exact cost these days.
- gorenb 3y agoFor making an LLM, not fine-tuning an existing one.
- brucethemoose2 3y agoOh, yeah, thats a different can of worms. I know nothing of this, but I hear that you probably want H100s (instead of A100s) and a good fault tolerant platform, kinda like: https://www.mosaicml.com/blog/mpt-30b https://www.mosaicml.com/blog/mpt-30b But you are either talking about a toy LLM or spending hundreds of thousands of dollars.