You can still imagine a gravity well that even a nuclear powered rocket would be unable to climb out of. Nuclear thermal rocket has greater specific impulse tha

You can still imagine a gravity well that even a nuclear powered rocket would be unable to climb out of. Nuclear thermal rocket has greater specific impulse than chemical propellant rocket, but not by that much. But there are other ways to climb out of any gravity well (except black hole) than a usual mass-reaction rocket, not benefiting buoyancy. If the atmosphere is transparent enough, a plane or a ballon could carry a small craft high enough, and then it could be propelled by pressure of a strong laser below. Of course, in alien economics and sufficiently deep gravity well, this could be prohibitively expensive.