To expand on and put numbers to this: to escape the solar system from Earth, you need to add about 17 km/s on top of the energy you start with by working from E

To expand on and put numbers to this: to escape the solar system from Earth, you need to add about 17 km/s on top of the energy you start with by working from Earth's orbital energy. Earth is going about 30 km/s around the Sun, which helps quite a lot! However, this also means that to hit the Sun, you need to remove more or less all of the energy that Earth starts you with. This is the key insight - if you do a bit too much or a bit too little, it slingshots around the Sun (getting close to it but not going into it) and stays in orbit rather than landing in the Sun.