The only thing I've seen run on raw audio are WaveNet models, and those are way too expensive to get into an embedded chip, no public real-time implementations

The only thing I've seen run on raw audio are WaveNet models, and those are way too expensive to get into an embedded chip, no public real-time implementations exist, though Baidu had a paper which claimed real-time execution speed on some server class Intel CPU last week. They do mention that their CPU implementation could be parallelized too.