If you chose a server in your basement, then it’s communicating with federated servers. Those servers that your friends use have the same data that you’ve sent

If you chose a server in your basement, then it’s communicating with federated servers. Those servers that your friends use have the same data that you’ve sent to them. So they don’t even need to find your main server, just collect enough. An out of the US host is easier for them actually. Say hello to NSA exploits without a warrant! Have you looked to see what metadata something like Matrix collects? How federation and contact lists work on that server?