That particular shortcoming is more about training data than fundamental limitations of the architecture (of which there are plenty too). To do well on tasks r

That particular shortcoming is more about training data than fundamental limitations of the architecture (of which there are plenty too). To do well on tasks related to pronunciation the model needs either to have been trained on audio data (just starting to be done - e.g. Reka.ai), and/or trained extensively on dictionary/etc pronunciation data and tasks related to its use.