It's funny you mention PAAS; in my last two jobs I've been respectively devving for a large on-prem datacenter, and an entirely PAAS/IAAS team. Both had their

It's funny you mention PAAS; in my last two jobs I've been respectively devving for a large on-prem datacenter, and an entirely PAAS/IAAS team. Both had their own pitfalls, and both can be supplemented with automation in some areas, it really is to some extent an apples to oranges comparison, and automation is a boon in both environments. re: the arch assumptions, man you've touched on a can of worms there (things should be forseen but aren't due to time.) I have so many things to say about this but in brief; the longer you do this "sort of stuff" the faster you are at seeing common patterns/pitfalls/avoiding that, and to that end is why I recommend SO STRONGLY working with someone who is a seasoned expert at this since a lot of that intuition is very domain specific/voodoo-esque. Now, this will help you in the future, but it won't help you now; in the absence of proper expertise/time/resources to determine the arch, I take two approaches: "thinking about it really hard", alongside my team. If I'm not an expert, and my peer isn't an expert, maybe however both of us together may manage to catch each other's mistakes in logic better than alone. Pair architecting can really facilitate working through a crunch, additionally if you do so with the philosophy of, "assuming any of these decisions are CRAP, do we have a path to remediating it without pain"; such that even if/when you make the wrong choices, you can see a direction to rectify it. I realize this is nebulous but I hope it conveys at least the shape of an approach. Re: the success of PaaS, absolutely, however, it incurs pitfalls commensurate with its benefits. Take Azure SQL for instance. 1 TB DB limit. It's a FANTASTIC managed service, but for many truly big data scenarios that's an unworkable cap. Additionally, if someone really deep breaks, you may have a whole other support chain to go through rather than in-house expertise. (I do not mean to direct that latter statement explicitly at azure, I've seen it across PAAS/IAAS providers) WRT Lock In, I've taken to aggressively avoiding *AAS systems that I don't have a good migration story for upfront (paramount to my statement re: if this is a terrible choice have a way out); use these tools and a knowledge of the costs/benefits to chose the components that solve the problems you want, we have a toolchest and can often pick and chose the most appropriate ones. Re: ML: AML does this to some extent; endpoint scalability and custom ML modules internally that you can paramaterize to dynamically tune some functionality. It certainly is not a perfect solution but it does some neat things, I've found some success using it but I by no means intend to paint it as a silver bullet; however I share your sentiment that in the long run these capabilities are very useful and more plug-and-play/flexible hosted ML seems a natural next step with the direction we've been going in to more "commoditize" some aspects of data analytics. Disclaimer since I mention some MS products; I am an MS eng but all of this is just my own ramblings as a dev. Any advocation/caveats I say are only my own experiences, and I am not even a true expert on all of our in-house platforms so much as I am a consumer.