Back-propagating a batch you go from one loss to the sum of individual losses and then down to each example separately. Each example has different activations t

Back-propagating a batch you go from one loss to the sum of individual losses and then down to each example separately. Each example has different activations that multiply with the upstream gradients, so first we need to compute per example weight gradients. The computation is a perfect mirror of the forward pass.