There's two major types of monitoring you should be concerned with, log aggregation and metrics collection. For log aggregation the most mature open-source sol

There's two major types of monitoring you should be concerned with, log aggregation and metrics collection. For log aggregation the most mature open-source solution I know of is the ELK stack (Elasticsearch for storage, Logstash for aggregation, Kibana for visualization). Set that up (bonus points if you use Puppet/Chef/Ansible/Whatever to set it up for you), and make sure you can detect anomalies from your visualization interface. For metrics collection, Prometheus has a special place in my heart, I love its architecture. That said there are quite a few options, InfluxDB is another (arguably simpler) option. You can then use something like Grafana to visualize the metrics. For configuration management tools, Chef is complicated and you better know Ruby and have a few months to dig into it. Puppet is simpler, but has a lot of weird quirks. Ansible is simple, but not as powerful (or used); so pick your poison, they all are pretty similar conceptually. Basically at the end of your journey you should be able to kill any server (or network connection, or application running on a server) and be alerted to it in near realtime. You should be able to diagnose what's wrong quickly using Kibana and Grafana, and finally you should be able to destroy and recreate the server using a few well understood commands and have it be completely reconfigured and ready to accept connections. Of course this is pie in the sky (like having perfect unit/integration testing for a very large application), but that should be the goal.