I agree that a big part of the problem is SNMP and logs/screen scraping. NETCONF is a good starting point for this but there should be an equivalent for getting

I agree that a big part of the problem is SNMP and logs/screen scraping. NETCONF is a good starting point for this but there should be an equivalent for getting XML of monitoring and metrics in a streaming manner. There are an infinite number of potential issues and the standard high level SNMP driven overviews of networks always fall short. It would be really cool if there was an open source network measurement and instrumentation movement. Facebook actually posts things along these lines such as their NANOG talks on automation.