3 ms·
LLMs come with a higher cost per transaction compared to dedicated models. imo the industry will consolidate around LLMs for early prototypes and as part of th
by losteric 3y ago
LLMs come with a higher cost per transaction compared to dedicated models.
imo the industry will consolidate around LLMs for early prototypes and as part of the workflow for building a corpus to train domain-specific models.
Source: team went through this process. LLM cost ~1% unskilled human labelers, then we finetuned BERT to bring the cost down to < 0.001% (savings $120k/y compared to just using the LLM)
- ldjkfkdsjnv 3y agoThe cost will drop like a rock in the coming years. Its always the same pattern with technology. Theres probably 5-20 Billion in capital working on solving this problem right now.
- losteric 3y agoWouldn't cost savings also translate to simpler models? I anticipate the process of distilling knowledge to simpler models will also become much cheaper and non-technical
- two_in_one 3y agoTo be fair you need to add the cost of new development and estimate when it pays bask. Without that it's not obvious that custom solution is better. For small tasks it's likely not.