So interesting... flow (first NetFlow) has been around since the mid 90s but it's always been a volume problem relative to the (single) computers of the day.

So interesting... flow (first NetFlow) has been around since the mid 90s but it's always been a volume problem relative to the (single) computers of the day. The big issue we kept hearing from folks from the notes community is that the current 'cheat' is totake flow (now sFlow, IPFIX as well) and just instantly aggregate it to turn it into tsdb-like summaries. The challenge is that you don't actually know in advance what you want to look at details for - whether you're investigating perf, availability, security, or just efficient ops. So that's the current big problem - how to get and store and analyze thousands to tens of thousands to millions (for big networks) of flow records/second. The 'old' problem from 1996 to 2006 of devices not being able to export has mostly gone away. There are bugs that happen every so often (like Juniper / sampled that ras called out and got resolved last year), but newer Juniper and Ciscos can do line-rate NetFlow/IPFIX to ~ 100-200k flows/sec per devices or sometimes per card. Which feeds back into point #1. Not that useful to have extra resolution if your systems are just doing top N on 10 fixed aggregation dimensions... What I'd really like to see the silicon do is tcpstats, app decode and latency data. Both have applications for performance and security (even perf data can hint at botnets vs humans, for example, and of course URLs let you look for sql injection and app vs network problems). The load balancer folks and some of the "Network Packet Brokers" have roadmap or the beginnings of this but it'd be better to have on Broadcom... Or, an even more basic thing would be to have not per-packet sampling+sFlow like Broadcom and other chips do, but also a primitive for "flow sampling" to get all packets for 1/Nth of flows (say, 1/1000th) to get sampled such decodes up on a daemon running on say Cumulus or Arista.