2 ms·
Why would it? Do you know how much it costs to finetune one of these models for such a niche language? I'm not just talking about the cost of training, but also
by behnamoh 3y ago
Why would it? Do you know how much it costs to finetune one of these models for such a niche language? I'm not just talking about the cost of training, but also the cost of acquiring data because there's much less data about niche languages.
- sitkack 3y ago> Do you know how much it costs to finetune Between 30-3000$, often in the 300$ range.
- kirill5pol 3y agoFrom their numbers 3 hours with 32 A100 80GBs. From lambda cloud: 3 hours * 4 ~22$/hr for 8x A100 ~= $265 So yeah not too expensive even for a native fine tune (obviously this ignores all other costs other than the GPUs)
- wtarreau 3y agoIt certainly costs much less to the society to train for Pascal once than to make everyone burn CPU cycles running Python!
- PeterStuer 3y ago96 x A100 hours for a finetune according to the article. The cost of the dataset curation for a given language is hard to quantify as there are many unknowns. However, it seems perfectly crowdsourcable to volunteers.