I hope so, but I'm not convinced. The experimental results don't persuade me that deep transformers are better than the existing extremely wide transformers wi

I hope so, but I'm not convinced. The experimental results don't persuade me that deep transformers are better than the existing extremely wide transformers with tons of parameters. Yes, they show a much better result on the multilingual benchmark (M2M-100), but that's not a super well studied problem, so I don't know that the baseline they're beating is very well optimized. And on OPUS-100 they only beat the baseline by building models with vastly more parameters, which shouldn't really surprise anybody.