For Machine Learning (esp. computer vision) it is really important. First, as others have pointed out, it lets you use bigger batches of data in a single forwa

For Machine Learning (esp. computer vision) it is really important. First, as others have pointed out, it lets you use bigger batches of data in a single forward pass. This has not only performance aspects, but also certain regularization aspects. Second, if you want/need to use higher resolution inputs then you really need a lot of memory. For example, MaskRCNN - a popular instance segmentation network - can only do batch_size=1 at 960x960 pixel images (with some tweaks). 960x960 is not a particularity high resolution by contemporary standards and also a batch_size=1 can cause gradient fluctuations (this is usually not desired). There is also some trend in NLP (Natural Language) to work with lots of data/bigger models that are memory hungry (e.g. GPT-2).