2 ms·
> I’m sure someone will eventually pony up for the compute time to do 65B. How much would it cost? Can you give a breakdown of hardware/paas requirements and t
by nico 4y ago
> I’m sure someone will eventually pony up for the compute time to do 65B.
How much would it cost? Can you give a breakdown of hardware/paas requirements and the costs?
- nullstyle 4y agoI was able to finetune the pretrained 13B model (using the same alpaca dataset, just running on my dime) in about 14 hours on an A6000. I think it will work out to between $50 and $60 when the bill hits me. Be warned everyone is slapping code together so fast that, if your experience is like mine, you'll spend most of your time working around assumptions made by prior authors or hand merging patches between forks to get your setup running well.
- nico 4y agoThank you! Very insightful. Crazy pace.
- gorbypark 4y agoAnother datapoint, first hand report from a random internet person that 7B took 2.5-3hrs on a 8xA100 80Gb setup. If it ends up being a straight up linear thing, that means about ~27 hours for the 65B model. Depending on the host and if it's a preemptable instance or not, that could be about $12-30 per hour.
- nullstyle 4y agoHave a link? I haven't seen any finetuning scripts in the wild that train a PEFT model on a multrigpu setup yet and would love to play around with one.