It certainly sounds like they would be sending it to the address provided by the scammer. The issue is their system assumes the first person to interact with it

It certainly sounds like they would be sending it to the address provided by the scammer. The issue is their system assumes the first person to interact with it is trustworthy: gives a real phone number and address. If that first contact with Google was MITM'd, they seem to have no way to develop an un-compromised relationship with the real entity.