It matters because the dollars stay in the US and something must happen with them. So it's not like we are just sending them money and getting nothing in retur

It matters because the dollars stay in the US and something must happen with them. So it's not like we are just sending them money and getting nothing in return. As soon as they are exchanged for pesos, that means someone else bought dollars and is going to do something with them. Some industries are priced in dollars but the only major country where people actually spend dollars (without exchanging them) is the US. A trade deficit means overall the other country is buying more investments instead of goods. But those investments are in the US. To actually spend the money somewhere else they would need to exchange it again. And those investments in the US aren't inherently a bad thing. If it's just buying US government bonds and the government not doing anything productive with that money it might be a bad thing though.