Capacitors do store energy, in their electric field. Potential energy in general, as far as I'm aware, is stored in fields of some sort -- water turning a wheel

Capacitors do store energy, in their electric field. Potential energy in general, as far as I'm aware, is stored in fields of some sort -- water turning a wheel (gravitational), tension in a spring (electromagnetic), capacitors (electromagnetic), chemical energy in gasoline (electromagnetic), nuclear energy (weak/strong fields I believe, not as well versed here) and so forth.