3 ms·
Not true if there’s enough competition that you need more resources for a bigger model in 2-3 years. Anecdotally, it seems like SOTA model training costs are ri
by kippinitreal 6y ago
Not true if there’s enough competition that you need more resources for a bigger model in 2-3 years. Anecdotally, it seems like SOTA model training costs are rising much faster than computer cost is falling.
- Jack000 6y agomaybe, but model quality doesn't scale linearly with model size. The performance/dollar metric is more important, and that definitely will decline over time. Personally, I have a boots trapped business that uses a transformer model. I'm not worried about supermassive models like gpt3 because it would be way too expensive to deploy for my use case, with marginal additional quality.