3 ms·
Are you saying they reduced the quality of the model in order to save compute? Would it make sense for them to offer a premium version of the model at at a very
by edg5000 2y ago
Are you saying they reduced the quality of the model in order to save compute? Would it make sense for them to offer a premium version of the model at at a very high price? At least offer it to those willing to pay?
It would not make sense to reduce output quality only to save on compute at inference, why not offer a premium (and perhaps perhaps slower) tier?
Unless the cost is at training time, maybe it would not be cost-effective for them to keep a model like that up to date.
As you can tell I am a bit uninformed on the topic.
- bt1a 2y agoYeah, as someone who had access to gpt-4 early in 2023, the endpoint used to take over a minute to respond and the quality of the responses was mindblowing. Simply too expensive to serve at scale, not to mention the silicon constraints that are even more prohibitive when the organization needs to lock up a lot of their compute for training The Next Big Model. Thats a lot of compute that cant be on standby for serving inference