3 ms·
Oh, they undoubtedly were. We have decent volume, but worked very hard in our prompting to minimize input and output tokens, so we don't have a big bill. I just
by phillipcarter 3y ago
Oh, they undoubtedly were. We have decent volume, but worked very hard in our prompting to minimize input and output tokens, so we don't have a big bill. I just think that in the past months they've added significantly more capacity and done a lot of work on their inference servers to make things run better.
- waffletower 3y agoGiven Meta's obvious, and most welcome, open source play with LLaMA 2, it behooves OpenAI to be performant and accessible to everyone. For my application, I am finding that 70b LLaMA 2 models are very close to the inference quality of gpt-3.5-turbo.