> certain very large deep neural networks don’t seem to need a drawn-out compression phase in order to generalize well. Instead, researchers program in somethin

> certain very large deep neural networks don’t seem to need a drawn-out compression phase in order to generalize well. Instead, researchers program in something called early stopping, which cuts training short to prevent the network from encoding too many correlations in the first place. This would explain the success of attention mechanisms quite well. If you are good in selecting the right input you don't need to discard a lot later.