3 ms·
People are going to want to use GPT-4o style real time conversational voice AI a lot. I can see people having hour-long conversations on a regular basis within
by modeless 2y ago
People are going to want to use GPT-4o style real time conversational voice AI a lot. I can see people having hour-long conversations on a regular basis within a few years. That application alone is going to need mountains of GPUs, more than are needed for training. And it will be extended to real time video input and output within a few years with another increase in GPU requirements.
- nmfisher 2y agoIf it does play out like this, I feel that inferencing will be taken over by lower-power ASICs (e.g. Groq) or even commodity CPU hardware. GPUs are an expensive solution (both power consumption and capex) to a straightforward problem. There are already real-time TTS models running on ARM, and while the fidelity isn't on par with large GPU models yet, a year or two of hardware improvements and software optimizations will probably close that gap. The same may not apply to audio2audio models, though, and NVIDIA will probably keep a firm grip on training, too.