The insulator in common aluminum electrolytic capacitors is also a ceramic -- aluminum oxide. The positive plate is aluminum, and the oxide grows on it because

The insulator in common aluminum electrolytic capacitors is also a ceramic -- aluminum oxide. The positive plate is aluminum, and the oxide grows on it because when the capacitor is charged, the positive plate attracts free oxygen atoms from the electrolyte (and hydrogen atoms are released at the cathode). While it is brittle, it's also self-healing. When it cracks and exposes the aluminum underneath, a new layer of oxide is plated immediately, as long as the capacitor is charged. Equipment that's been left powered off for a long time and then turned on can sometimes have caps explode because there's a lot of leakage current from holes in the ceramic until it heals (minutes to hours). Lithium batteries are usually completely ruined if they ever discharge down to zero.