The secret of RNN-T is that it is __extremely__ hard to train, it is very unstable. On a single GPU you'll spend years to train reasonable quality model, moreov

The secret of RNN-T is that it is __extremely__ hard to train, it is very unstable. On a single GPU you'll spend years to train reasonable quality model, moreover a streaming one. Streaming training requires teacher-student setup. Thats why there are dozen RNNT projects around, some of them more reasonable, some less, but most of them struggle to demonstrate even a good librispeech WER. You have a long way to go.