They do though. A ton of the neurons end up not activating on any particular layer, leading to a huge waste as you're passing a zero down the layers. I spoke w

They do though. A ton of the neurons end up not activating on any particular layer, leading to a huge waste as you're passing a zero down the layers. I spoke with a PM at TT and he told me an important idea is that we're spending a lot of electricity multiplying things with zeros.