Oh ok, I see. He was talking about only the gradients of weights all going to the same, single neuron. Then yes, the gradient of the loss in respect to the outp

Oh ok, I see. He was talking about only the gradients of weights all going to the same, single neuron. Then yes, the gradient of the loss in respect to the output of that neuron is a single value, which will be multiplied by x vector. So w_grad vector of weights going to that neuron will have a sign of dL/df, which again, is a single value. Hopefully this is clear.