It really is that simple. Yes, there's many different approaches to this (which can become quite clever and complex, which is true of training in general), but

It really is that simple. Yes, there's many different approaches to this (which can become quite clever and complex, which is true of training in general), but it all really boils down to adding or averaging the gradients in most cases.