jfrankle

7 ms·

1. 1.▲
   by jfrankle 1y ago

   whyyy
2. 2.▲
   by jfrankle 3y ago

   Honestly, just a matter of having the time to clean everything up and get it out. The ancillary code, model cards, etc. take a surprising amount of time.
3. 3.▲
   by jfrankle 4y ago

   We've come to accept that it's an impossible problem at this point. Instead, we're getting good at automatically detecting hardware failures and rapidly restarting runs on fewer nodes. We're also exploring batch sizes th
4. 4.▲
   by jfrankle 4y ago

   It's an indictment of the A100 node that died on us yesterday, leaving us with 248 GPUs in the particular cluster where we were running the experiments :( It turns out that, in these kinds of large-scale experiments, hardware failures
5. 5.▲
   Pruning Neural Networks at Initialization: Why Are We Missing the Mark?(arxiv.org)

   3 points by jfrankle 6y ago|0 comments
6. 6.▲
   by jfrankle 8y ago

   This is an illuminating (and notably rigorous) read for anyone interested in neural network sparsity and compression. But - equally importantly - it's a valuable read for anyone interested in the replicability of neural network researc
7. 7.▲
   by jfrankle 8y ago

   (Author of paper here) This is approximately my growing suspicion.