3 ms·
> people who inference their models at scale What does "inferencing models at scale" mean?
by esperent 4y ago
> people who inference their models at scale
What does "inferencing models at scale" mean?
- andai 4y agoOptimizing it to be cheaper to run (e.g. OpenAI saved a lot of money with the turbo variant that ChatGPT uses, relative to the original GPT-3 models).
- cypress66 4y agoActually using the model, instead of doing inference a few times when writing your paper and then that's it.
- Closi 4y agoRunning server farms to quickly give answers to users, like bing or ChatGPT. The models produced have to be fast and efficient to support capacity/cost, which has some detriment to quality/accuracy.