2 ms·

Can you calculate that for me on a napkin? Every calculation I make for training, but certainly for inference, makes me break even after well under a year and t

Can you calculate that for me on a napkin? Every calculation I make for training, but certainly for inference, makes me break even after well under a year and then it’s vastly less if I buy the hardware myself.

whimsicalism 4y ago
You can infer on your local machine. I doubt it will take you a year to train a model. Vicuna-13B cost $300 to train/fine-tune [0]. They trained on an A100 which costs $10k [1]. [0]: https://vicuna.lmsys.org/ https://vicuna.lmsys.org/ [1]: https://github.com/lm-sys/FastChat/blob/main/scripts/train-vicuna.yaml#L2 https://github.com/lm-sys/FastChat/blob/main/scripts/train-v...