4 ms·
OpenAI API Batch Requests
- dosinga 2y agoIs running things in batch mode also cheaper? I don't see it, but it would be a nice way for OpenAI to smooth out their demand
- davidbarker 2y agoAccording to their tweet, the discount on API costs will be 50%, so pretty significant. https://x.com/openaidevs/status/1779922566091522492 https://x.com/openaidevs/status/1779922566091522492
- serjester 2y agoThis seems like a fantastic approach to smoothing their GPU utilization. It’s so difficult to compete with OpenAI - they ship so many “no brainer” features that are constantly setting the bar for what an LLM provider needs to provide - chat API, multi modality, function calling, json output, it’s endless.