Technically speaking yes it will draw power, all wiring draws power (the wires develop capacitance, but there's also electrostatic leakages around the bare conn

Technically speaking yes it will draw power, all wiring draws power (the wires develop capacitance, but there's also electrostatic leakages around the bare connectors, but at 110/240V it's negligible, this is a concern around 10,000V+, with DC it likely has no electrostatic leakage, and very little drain due to capacitance. However, what I believe you're thinking about is the transformer required to turn 110V->5V and yes you're correct. An old school transformer would drain a few watts when inactive, however the current types of transformers used in Wall->USB devices provide virtually no drain, perhaps less than a watt, so will have no noticeable effect on your electricity bill. Simply put, changing say 20 wall sockets in your house to one of these USB devices will likely have less effect on your electricity bill than plugging in a 7W night light and leaving it in.