So long as you trust your server’s admin, your data should be safe from them and non-federated servers. However once your server is linked to another, the whole

So long as you trust your server’s admin, your data should be safe from them and non-federated servers. However once your server is linked to another, the whole map of connected servers can see your outward-facing info. If you are DMing users outside your network, you can also consider those messages non-private, as your server, the destination server, and possibly intermediatry servers will have seen it. Services like this would do well to integrate the Signal protocol, although I don’t know how well it scales with federated servers, I can’t recall any actual tests of that functionality yet.