Use the stack that makes the project as easy, robust and low maintenance as possible taking into account risks involved in anything new you have to learn along

Use the stack that makes the project as easy, robust and low maintenance as possible taking into account risks involved in anything new you have to learn along the way. Sticking to only what you know can make life harder for yourself as well and means you're not expanding your skills. If you're highly skilled in Java and C# you're not going to find it challenging to pick up Python or Rails. Heroku is a great choice for a freelancer as well as it's so low maintenance by a large margin in my opinion compared to something like AWS. You certainly want to avoid things as low level as VPSs as well. Personally I think you should take a couple of days to experiment with Heroku so you can see there isn't really that much to it.