3 ms·
I wonder if future versions of PyTorch will automatically apply mixed precision if not specified, the article makes it seem like a no brainer to use them by def
by jerpint 3y ago
I wonder if future versions of PyTorch will automatically apply mixed precision if not specified, the article makes it seem like a no brainer to use them by default
- hatthew 3y agoIt can still be complicated by many things. Some parts of a module may not work well in lower precision and need to be in higher precision. If you ever create new tensors within the forward pass, you need to manually adjust your code to automatically use the right datatype. You still need to figure out which low-precision dtype you want to use in your model (and perhaps use different ones in different parts of your model). Etc.