3 ms·
These "pure speech" models could really benefit from being coupled to a large language model like ChatGPT. YouTube live transcriptions are terrible, because th
by jiggawatts 4y ago
These "pure speech" models could really benefit from being coupled to a large language model like ChatGPT.
YouTube live transcriptions are terrible, because they get confused by homonyms and can't follow the context in a sentence.
In the same manner that Dall-E joined an LLM to an image generator, they ought to train a combined speech model + LLM so that the uncertainties in the speech model output is disambiguated by the LLM.
- andy_ppp 4y agoSame with spell check, with context and phonetics you could make autocorrection better than any human.
- spullara 4y agoUse whisper. It doesn't get confused in that way.
- IshKebab 4y agoAre you sure about that? I've seen YouTube captions understand homophones, and Google Assistant definitely can (though that may be linked to some other system). Surely the speech recognition model itself learns some basic language statistics in order to recognise homophones?
- reliableturing 4y agoThis is exactly part of this Google USM approach, although these pretrained models are significantly smaller than ChatGPT. They reference this paper [1] which contains more details on the pretrained text-only alignment with the speech model. [1] https://arxiv.org/abs/2209.15329 https://arxiv.org/abs/2209.15329
- kleiba 4y agoThere are no "pure speech" models. All ASR uses language models.