text to WAV: You predict mel spectrums with a transformer architecture (so word embedding + attention decoder) and then convert them into audio signals with Pa

text to WAV: You predict mel spectrums with a transformer architecture (so word embedding + attention decoder) and then convert them into audio signals with Parallel WaveGAN or Hifi-GAN. FastSpeech2 (Microsoft) is extremely good. WAV to text: You detect wave shapes with convolutions to generate an embedding, then attention layers to turn it into an encoding, then convert that to logits. Logits go into language model and that predicts the final sentence with a beam search decoder. wav2vec 2.0 (Facebook) is amazing.