Research [0] suggests that pretraining to uncover sparse subnetworks is faster when using "easy" subsets of the training data. Very small part of the overall pi

Research [0] suggests that pretraining to uncover sparse subnetworks is faster when using "easy" subsets of the training data. Very small part of the overall picture but I do expect to see more research on data hashing/subsetting/ordering for training optimization in the near future. [0] https://arxiv.org/pdf/2206.01278.pdf https://arxiv.org/pdf/2206.01278.pdf