4 ms·
Yes, monitoring, anomaly detection, stopping computer viruses, intrusion detection, etc. are old problems. And rules have long been the relatively sophisticate
by graycat 4y ago
Yes, monitoring, anomaly detection, stopping computer viruses, intrusion detection, etc. are old problems. And rules have long been the relatively sophisticated solution technique, i.e., more than just thresholds! And false alarm rate and detection rate have not played much of a role in the discussions.
And attacking the problems assuming knowing some relatively fine details about the system, e.g., some Web server, that is the target of the work and the problems want to detect and correct has been common. E.g., when have an instance of some bad stuff that didn't detect soon enough but want to, analyze that problem or class of similar problems and build a detector just for that. And, yup, for that particular problem, might have a nicely low false alarm rate and relatively high detection rate -- on this class of problems seen before, analyzed, and understood. So, here maybe should say that have built a model.
But I was not impressed with rules, didn't want to need a lot of fine expertise in the target system, and
wanted to detect also problems never seen before, with some good progress on rates of false alarms and miss detections.
Yes, what I did is not easily called "model-based" and was supposed to be "new, correct, and significant". Maybe it was at least partly new; maybe it still is.
I started with the data being random variables -- that's not assuming much! Then would like to have the assumptions of independent and identically distributed (i.i.d.) random variables, but for practice that is asking a bit too much! And for some positive integer n, having i.i.d. random n-tuples seemed a bit much!
Well, I used something weaker, but still strong enough, with some math, to get some results, called exchangability.
So, right, my techniques are distribution free, that is, assume nothing about probability distributions and make no attempt to estimate such. Gee, I certainly didn't assume a Gaussian distribution!
I was able to get false alarm rate adjustable and known exactly in advance. Looked like progress to me!
I had some real data, fed it into my software, and, presto, bingo, the false alarm rate was as adjusted and desired!
And using a classic result of S. Ulam, I was able to show that for detection rate the math was not trivial, that is, was better than merely trivial. More progress?
To use the work, don't have to be an expert in the system being monitored, don't have anything like a model, and get to detect problems never seen before, sometimes called zero day problems.
So, just have some random variables, exchangability, and the Ulam result.
But, you are correct: Rules, models, thresholds, etc. are what is popular; the Ulam result is not very popular!
My paper has been in the literature for many years now, and I suspect that so far the number of readers is 0.00. Not really popular! Not likely to compete with the new Tom Cruise movie!
Ah, I forgot: I also worked up a relatively fast algorithm for the core calculations!
I learned my lesson and am doing things quite different now! I started with a problem some hundreds of millions of people would like solved!
At Easter, I talked to a guy who works in computer and network security. I asked him what would be the security risks if a computer with an up to date instance of Windows Server as a Web server just openly faced the Internet? His reaction was that the results would be multiple disasters from attackers around the world. He mentioned "SQL injection" -- gads, I thought I was quite careful when I took the data from the user and constructed my SQL queries!
So, near the top of my TODO list is just to write a little .NET code, listen on some IP ports, scribble what comes in to a file, and see what I get!
Anyone want to predict what I'm likely to get?