4 ms·

Off-topic, but really, what is "IBM's leading artificial intelligence language"?

by nnq 13y ago

PeterisP 13y ago
Probably Prolog, as IBM's Watson tech seems to use it - but yeah, Prolog skills are a very specialized niche that's used very, very rarely in industry.
graycat 13y ago
At its time, KnowledgeTool. It's a forward chaining expert system language using the Forgy RETE algorithm and is a pre-processor to PL/I. It was marketed as for artificial intelligence but, really, as it was created by our group at Yorktown Heights, was mostly intended to automate monitoring and system management for server farms and networks. One of my contributions was architecture, algorithms, and code for rule-subroutines. The core of the work was to understand the PL/I stack of dynamic descendancy and to exploit it to keep the state of the user's rule subroutine calls that had yet to return while calling a run-time package with the RETE algorithm and then return to the user's code with it's state intact. It was a fast idea one afternoon and then a long night typing. When I got some sleep and back to the office, our programmer trying to finish the implementation of rule subroutines was done that day instead of some weeks, and the result was much cleaner. In addition, I cooked up some new ideas for anomaly detection in data from the envisioned server farms and networks. It's an ambitious approach to behavioral monitoring (try to say what is normal and declare anything else as an anomaly and potential problem to be investigated -- as in the old Herve Debar post at SANS, such monitoring has had false alarm rate too high, and I solved that problem) where a user gets a knob to turn to select false alarm rate exactly in advance. The work was ambitious when I did it because it can use quite a lot of data; now the cost of so much data is so low that it is from trivial down to no concern. As applied probability, it's a statistical hypothesis test that is both multivariate (use more information and, thus, get higher detection rate of real problems) and distribution-free (make no assumptions about probability distributions) -- maybe still the only example of a test with these two properties. The paper was published in Information Sciences. I had some data from a cluster at Allstate and showed with that data that the false alarm rate calculation was correct. And with some additional simulated data, showed that the detector was darned sensitive (intuitively) to anything different. Basically could have a checkerboard with normal performance on the white squares and anomalies on the black squares, and the detector will figure this out, including if the checkerboard is in 20 dimensions. Due to such checkerboard examples, data in fewer dimensions would be useless; so, in this case, the multidimensional aspect was crucial. While such a checkerboard is unlikely in practice, the practical point is that the detector is adaptive to quite bizarre normal performance. Indeed, we should expect that the multidimensional geometric region of normal performance will, often in practice, be essentially a fractal -- my work does well approximating fractals. Now that there is so much data on real-time performance from high end server farms and networks, my work should be deployed, but I doubt that it has been.

Off-topic, but really, what is "IBM's leading artificial intelligence language"?

by nnq 13y ago

Off-topic, but really, what is "IBM's leading artificial intelligence language"?

Probably Prolog, as IBM's Watson tech seems to use it - but yeah, Prolog skills are a very specialized niche that's used very, very rarely in industry.

graycat 13y ago

At its time, KnowledgeTool. It's a forward chaining expert system language using the Forgy RETE algorithm and is a pre-processor to PL/I. It was marketed as for artificial intelligence but, really, as it was created by our group at Yorktown Heights, was mostly intended to automate monitoring and system management for server farms and networks. One of my contributions was architecture, algorithms, and code for rule-subroutines. The core of the work was to understand the PL/I stack of dynamic descendancy and to exploit it to keep the state of the user's rule subroutine calls that had yet to return while calling a run-time package with the RETE algorithm and then return to the user's code with it's state intact. It was a fast idea one afternoon and then a long night typing. When I got some sleep and back to the office, our programmer trying to finish the implementation of rule subroutines was done that day instead of some weeks, and the result was much cleaner. In addition, I cooked up some new ideas for anomaly detection in data from the envisioned server farms and networks. It's an ambitious approach to behavioral monitoring (try to say what is normal and declare anything else as an anomaly and potential problem to be investigated -- as in the old Herve Debar post at SANS, such monitoring has had false alarm rate too high, and I solved that problem) where a user gets a knob to turn to select false alarm rate exactly in advance. The work was ambitious when I did it because it can use quite a lot of data; now the cost of so much data is so low that it is from trivial down to no concern. As applied probability, it's a statistical hypothesis test that is both multivariate (use more information and, thus, get higher detection rate of real problems) and distribution-free (make no assumptions about probability distributions) -- maybe still the only example of a test with these two properties. The paper was published in Information Sciences. I had some data from a cluster at Allstate and showed with that data that the false alarm rate calculation was correct. And with some additional simulated data, showed that the detector was darned sensitive (intuitively) to anything different. Basically could have a checkerboard with normal performance on the white squares and anomalies on the black squares, and the detector will figure this out, including if the checkerboard is in 20 dimensions. Due to such checkerboard examples, data in fewer dimensions would be useless; so, in this case, the multidimensional aspect was crucial. While such a checkerboard is unlikely in practice, the practical point is that the detector is adaptive to quite bizarre normal performance. Indeed, we should expect that the multidimensional geometric region of normal performance will, often in practice, be essentially a fractal -- my work does well approximating fractals. Now that there is so much data on real-time performance from high end server farms and networks, my work should be deployed, but I doubt that it has been.