The power rails have a non zero impedance (as in not DC), and large currents flow through the rails. These currents will create voltages that will seem to be n

The power rails have a non zero impedance (as in not DC), and large currents flow through the rails. These currents will create voltages that will seem to be noise to anything connected to them. The caps provide a low Z path to ground so the currents so your rails are stable. Circuits really need schematics though, so its kinda hard to explain w/out. The fundamental idea is easy enough, but dealing with this properly is very hard.