3 ms·

What I'd love to see is "how did network log processing happen in 1985, 1990, 1995, 2000, 2005, 2010, 2015, and how will it look in 2015". The ratios between n

by rdl 11y ago

What I'd love to see is "how did network log processing happen in 1985, 1990, 1995, 2000, 2005, 2010, 2015, and how will it look in 2015". The ratios between network traffic, processing, etc. are all changing, and DDoS, etc. are evolving. My perception is it's getting harder to monitor large systems at the same time as it is getting easier to just push raw packets (in a distributed, CDN-type sense). The complexity of running Enterprise networks in the 1990s used to be lots of non-IP legacy protocols, router bugs, etc.; and the limitations of silicon pushing a lot of things into Layer 2 vs. Layer 3 processing. Weird discontinuities around Cisco slacking off around 2000, too.

avifreedman 11y ago
So interesting... flow (first NetFlow) has been around since the mid 90s but it's always been a volume problem relative to the (single) computers of the day. The big issue we kept hearing from folks from the notes community is that the current 'cheat' is totake flow (now sFlow, IPFIX as well) and just instantly aggregate it to turn it into tsdb-like summaries. The challenge is that you don't actually know in advance what you want to look at details for - whether you're investigating perf, availability, security, or just efficient ops. So that's the current big problem - how to get and store and analyze thousands to tens of thousands to millions (for big networks) of flow records/second. The 'old' problem from 1996 to 2006 of devices not being able to export has mostly gone away. There are bugs that happen every so often (like Juniper / sampled that ras called out and got resolved last year), but newer Juniper and Ciscos can do line-rate NetFlow/IPFIX to ~ 100-200k flows/sec per devices or sometimes per card. Which feeds back into point #1. Not that useful to have extra resolution if your systems are just doing top N on 10 fixed aggregation dimensions... What I'd really like to see the silicon do is tcpstats, app decode and latency data. Both have applications for performance and security (even perf data can hint at botnets vs humans, for example, and of course URLs let you look for sql injection and app vs network problems). The load balancer folks and some of the "Network Packet Brokers" have roadmap or the beginnings of this but it'd be better to have on Broadcom... Or, an even more basic thing would be to have not per-packet sampling+sFlow like Broadcom and other chips do, but also a primitive for "flow sampling" to get all packets for 1/Nth of flows (say, 1/1000th) to get sampled such decodes up on a daemon running on say Cumulus or Arista.

by rdl 11y ago

avifreedman 11y ago

So interesting... flow (first NetFlow) has been around since the mid 90s but it's always been a volume problem relative to the (single) computers of the day. The big issue we kept hearing from folks from the notes community is that the current 'cheat' is totake flow (now sFlow, IPFIX as well) and just instantly aggregate it to turn it into tsdb-like summaries. The challenge is that you don't actually know in advance what you want to look at details for - whether you're investigating perf, availability, security, or just efficient ops. So that's the current big problem - how to get and store and analyze thousands to tens of thousands to millions (for big networks) of flow records/second. The 'old' problem from 1996 to 2006 of devices not being able to export has mostly gone away. There are bugs that happen every so often (like Juniper / sampled that ras called out and got resolved last year), but newer Juniper and Ciscos can do line-rate NetFlow/IPFIX to ~ 100-200k flows/sec per devices or sometimes per card. Which feeds back into point #1. Not that useful to have extra resolution if your systems are just doing top N on 10 fixed aggregation dimensions... What I'd really like to see the silicon do is tcpstats, app decode and latency data. Both have applications for performance and security (even perf data can hint at botnets vs humans, for example, and of course URLs let you look for sql injection and app vs network problems). The load balancer folks and some of the "Network Packet Brokers" have roadmap or the beginnings of this but it'd be better to have on Broadcom... Or, an even more basic thing would be to have not per-packet sampling+sFlow like Broadcom and other chips do, but also a primitive for "flow sampling" to get all packets for 1/Nth of flows (say, 1/1000th) to get sampled such decodes up on a daemon running on say Cumulus or Arista.