4 ms·

Nice! I suppose that Pascal is not on your planned list of supported languages, right?

• behnamoh 3y ago
  Why would it? Do you know how much it costs to finetune one of these models for such a niche language? I'm not just talking about the cost of training, but also the cost of acquiring data because there's much less data about niche languages.
  • sitkack 3y ago
    > Do you know how much it costs to finetune Between 30-3000$, often in the 300$ range.
    • kirill5pol 3y ago
      From their numbers 3 hours with 32 A100 80GBs. From lambda cloud: 3 hours * 4 ~22$/hr for 8x A100 ~= $265 So yeah not too expensive even for a native fine tune (obviously this ignores all other costs other than the GPUs)
      • IanCal 3y ago
        It's wild to me that the raw compute cost is so low.
      • cbozeman 3y ago
        Sweet Jesus, that is amazing.
  • wtarreau 3y ago
    It certainly costs much less to the society to train for Pascal once than to make everyone burn CPU cycles running Python!
  • PeterStuer 3y ago
    96 x A100 hours for a finetune according to the article. The cost of the dataset curation for a given language is hard to quantify as there are many unknowns. However, it seems perfectly crowdsourcable to volunteers.
    • albert_e 3y ago
      A project like SETI@home should help these efforts I believe?
      • Sabinus 3y ago
        Maybe the stable horde could work it into the project. https://stablehorde.net/ https://stablehorde.net/