The way I understand it is that it's not impossible to cover the distance, but that by the time you do, the distance between your starting point and your destin

The way I understand it is that it's not impossible to cover the distance, but that by the time you do, the distance between your starting point and your destination will have (at least) doubled due to the expansion of space. So you're still as far away from your destination as in the beginning (or even further), but now you're the same distance from your starting position as well and are effectively stranded.