4 ms·
At $DAY_JOB nowadays we run 128x H100 runs without thinking twice nowadays. Only takes a few days to train a small-ish LLM with that to test out some ideas.
by ein0p 3y ago
At $DAY_JOB nowadays we run 128x H100 runs without thinking twice nowadays. Only takes a few days to train a small-ish LLM with that to test out some ideas.
- uptownfunk 3y agoWhere are they hosted?
- ein0p 3y agoAWS and GCP both.
- turnsout 3y agoOut of curiosity, what leads you to train models from the ground up rather than fine tuning existing models?
- ein0p 3y agoWe do both. You can’t just fine tune if you’re trying a different model architecture, or even change some of the hyperparameters on an existing one. Every now and again you might be able to reuse some of the weights, but that’s about it. That’s part of the reason research is so incredibly expensive and time consuming in this field. I bet that $80k is only a fraction of the overall cost for the model described in the article, too.
- moffkalast 3y agoecho $DAY_JOB
- agravier 3y agoAlso `echo $NIGHT_JOB`, I'm curious like that. To preempt other queries, maybe just paste the `set` output.
- Havoc 3y agoI don’t get why people do this at all. It adds no clarity only awkwardness over “At my job”
- evilduck 3y ago"At my job" is annoying and awkward too if you're not going to specify where when making grand claims. It's not quite as annoying as "In my country we..." without specifying where, but it's close.
- moffkalast 3y agoAh yes, like people saying "back in my day" and not even giving a precise date, smh.
- coolspot 3y ago> echo $DAY_JOB FAANG AI Lab