As there are simply no providers yet who offers ready-to-install packages of all these services (at least I don't know one), you go with that service you trust

As there are simply no providers yet who offers ready-to-install packages of all these services (at least I don't know one), you go with that service you trust and that suits you best. Some examples for relatively cheap VPS are Digitalocean, Vultr, Linode or, if you want to host somewhere "safe": Exoscale, which has datacenter in Switzerland and Germany.