3 ms·
From the Realtime API blog post: https://openai.com/index/introducing-the-realtime-api/ https://openai.com/index/introducing-the-realtime-api/ > Audio in the C
by minimaxir 2y ago
From the Realtime API blog post: https://openai.com/index/introducing-the-realtime-api/ https://openai.com/index/introducing-the-realtime-api/
> Audio in the Chat Completions API will be released in the coming weeks, as a new model `gpt-4o-audio-preview`. With `gpt-4o-audio-preview`, developers can input text or audio into GPT-4o and receive responses in text, audio, or both.
> The Realtime API uses both text tokens and audio tokens. Text input tokens are priced at $5 per 1M and $20 per 1M output tokens. Audio input is priced at $100 per 1M tokens and output is $200 per 1M tokens. This equates to approximately $0.06 per minute of audio input and $0.24 per minute of audio output. Audio in the Chat Completions API will be the same price.
As usual, OpenAI failed to emphasize the real-game changer feature at their Dev Day: audio output from the standard generation API.
This has severe implications for text-to-speech apps, particularly if the audio output style is as steerable as the gpt-4o voice demos.
- OutOfHere 2y ago> and $0.24 per minute of audio output That is substantially more expensive than TTS (text-to-speech) which already is quite expensive.
- qwertox 2y agoI agree. I'm wondering if it is possible to disable output streaming of audio and just get the text response event.
- colaco 2y agoIt seems so. The configuration of the session accepts a parameter (modalities) that could restrict the response only to text. See it in https://platform.openai.com/docs/api-reference/realtime-client-events https://platform.openai.com/docs/api-reference/realtime-clie....
- bcherry 2y agocorrect - you should also be able to save a lot by skipping their built-in VAD and doing turn detection (if you need it) locally to avoid paying for silent inputs.
- minimaxir 2y agoFair, it wouldn't work well for on-demand generation in an app, but for ad-hoc cases like a voice-over it's not a huge expense. If OpenAI decides to fully ignore ethics and dive deep into voice cloning, then all bets are off.