Thank you. That's consistent with my (very quick, superficial) read too. Technically, modern DNNs (transformers, CNNs, RNNs, etc.) don't have convergence guara

Thank you. That's consistent with my (very quick, superficial) read too. Technically, modern DNNs (transformers, CNNs, RNNs, etc.) don't have convergence guarantees either. We've just gotten used to SGD somehow always working! :-P