Maybe I'm thinking about things wrong, but can't you have a scalar quantity to represent error for backprop at the end of a neural network (in a supervised regr

Maybe I'm thinking about things wrong, but can't you have a scalar quantity to represent error for backprop at the end of a neural network (in a supervised regression problem for example)? That scalar error becomes a vector as it is assigned to various weights. I'm not trying to be obtuse, but it seems like the scalar reward in RL is also modifying countless variables in the Q functions of the state-action pairs that led to the final outcome/reward... Are those, by definition, smaller in number than the parameters of a neural network?