Biggest productivity boosters for me: - devdocs.io for pytorch - conda for packaging w CUDA - einops - tensorboard - huggingface datasets Interesting mode

Biggest productivity boosters for me: - devdocs.io for pytorch - conda for packaging w CUDA - einops - tensorboard - huggingface datasets Interesting models/structures: - resnet - unet - transformers - convnext - vision transformers - ddpm - novel optimizers - generative flow nets - “your classifier is secretly an energy-based model and yiu should treat it like one” paper - self-supervision - distance metric learning Places where you can read implementations: - lucidrains’ github - timm computer vision models library - fastai - labml (annotated quite nicely) Biggest foreseeable headaches: - not suuper easy to do test-driven development - data normalization (floating point error, not using eg batchnorm) - sensitivity of model performance to (hyper)params (layer sizes, learning rates, optimizer, etc) - impatience - lack of data I’d also recommend watching Mark Saroufim live code in PyTorch, on YouTube. My 2 cents, you can only get really fast as well as good at this with a lot of experience. A lot of rules-of-thumb have to come together just right for the whole system to work.