3 ms·Pretraining Under Infinite Compute3 points by jedharris 1y agojedharris 1y ago"Our results show that simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future."