3 ms·
Ask HN: PyTorch or Jax for Making an LLM?
- brucethemoose2 3y agoYou mean for finetuning an LLM? PyTorch + QLORA is very affordable. Otherwise JAX on TPUs is a reasonable way to finetune on a large dataset. I'm not sure about the exact cost these days.
- gorenb 3y agoFor making an LLM, not fine-tuning an existing one.
- brucethemoose2 3y agoOh, yeah, thats a different can of worms. I know nothing of this, but I hear that you probably want H100s (instead of A100s) and a good fault tolerant platform, kinda like: https://www.mosaicml.com/blog/mpt-30b https://www.mosaicml.com/blog/mpt-30b But you are either talking about a toy LLM or spending hundreds of thousands of dollars.