It'd speed up and miss, but say it passes through a cloud of gas which acts as drag and slows it down a little, now it misses the body slightly less, keep this

It'd speed up and miss, but say it passes through a cloud of gas which acts as drag and slows it down a little, now it misses the body slightly less, keep this going for a few thousand years and its lost enough energy relative to the first body that it can't keep 'missing' anymore and collides/falls in. They're essentially counteracting that relative speed over time by bleeding energy to all sorts of drag (one exotic example that comes to mind is with things like neutron stars and black holes bleeding out vast amounts of energy in gravitational waves - literal waves in space-time we can detect from billions of light years away - in the moments before they crash into each other).