Ok this is way too long to read in one sitting, but it looks incredible? I've been looking for resources like this for """real""" model training at scale. The a

Ok this is way too long to read in one sitting, but it looks incredible? I've been looking for resources like this for """real""" model training at scale. The authors have worked on BLOOM (the first >100B parameter model), starcoder and fineweb so they actually know this space. I wish it were not a single giant webpage, but I guess I can deal with that.