I worked with a cyber ML team one time that explained that management wanted some kind of threat detection model to run in-line with only enough cases left at t

I worked with a cyber ML team one time that explained that management wanted some kind of threat detection model to run in-line with only enough cases left at the end for a small team of human analysts. The math was bleak (I don't remember all the details, but it's something like this): - The humans could handle about 1000 events per day across the team. - The system received around 100 million events per day. - This meant that something like a 99.999% (5-nines!) true positive rate was the target goal. But management poured millions into the team over the years because somebody had come to believe that the machines could handle 100% of the challenge and had gone to some AI conferences and become a true believer. The correct approach was team the analysts and the machines together in more effective ways (triage, enrichment, context providers, etc), and the correct goal was to focus on improving the results of the joint system over just the human team, not reach for absolute perfection. Triage, and accepting that sometimes shit gets through and use good practices like layered defense, n-factor auth, and zero-trust systems would carry the day. I remember the presentation where the new head of the cyber team explained that it's acceptable for trojans that target Windows 98 to get through as they had no vulnerable systems the trojan could exploit -- the entire company was MacOS and Linux. You would have thought heads would have fallen off of the executives.