The point is that if you put the experimentally measured distance, masses, and force into Newton's inverse square law in a regime where it does not actually hol

The point is that if you put the experimentally measured distance, masses, and force into Newton's inverse square law in a regime where it does not actually hold, you will get a different G and this is how you will know that modifications are needed. G will be variable in that regime.