8 ms·
On your last paragraph, we agree. Moreover, your a), b) say some of what I think better than I said it. Otherwise you have a good critique of traditional stat
by graycat 13y ago
On your last paragraph, we agree.
Moreover, your a), b) say some of what I think better than I said it.
Otherwise you have a good critique of traditional
statistics.
On traditional statistics, I voted with my research
and did work that was distribution-free, also
multidimensional, unusual for distribution-free. I did wonder
if there could be another way, and what you wrote
about Vapnik suggests that there can be.
This thread now is much deeper into the analysis of
machine learning algorithms than I noticed in
what I saw of Stanford professor Ng's course and
several books and papers in PDF I downloaded and
scanned. I would guess the level
of Vapnik's work would be that of a graduate seminar
and not a ugrad computer science course and that the
mathematical prerequisites would filter out nearly
everyone with a ugrad CS degree and nearly all
their professors. Why? From some of my experience getting
a paper of mine reviewed, measure preserving totally
blew away several editors in chief of famous
CS journals and chaired professors of CS at
famous research universities; one such person wrote
me "Neither I nor anyone on my board of editors
has the mathematical prerequisites to review your
paper.", and a second such person wrote me
a similar remark. The math prerequisites were not so
advanced, say, just at the level a graduate course
in probability via, say, Loeve or Neveu and fast
and easy reading for, say, E. Cinlar, P. Billingsley,
L. Breiman, J. Doob, A. Karr, E. Dynkin, etc. But the paper was 'CS' because it
was on a CS problem. For AI, I cooked up the paper
to improve on some of our work with 'expert systems'
for the problem. I have to suspect that work
in machine learning that needs some of Loeve
or functional analysis from, say, Rudin will
not be popular in CS departments.
This thread has become a good start on a grad seminar.
Just now I can't follow up on the leads here;
instead I have to finish off the code for
two Web pages and get my new Web site live.
- srean 13y agoMachine learning is really one part probability theory, one part functional analysis one part optimization and another part algorithms. I think with your background you will enjoy it. The particular two papers by Vapnik and Chervonenkis are very readable and does not use anything more technical than Chernoff bounds, its more about how they are used rather than what are used. CS students typically do not have probability in their curriculum, at least no measure theory, so you will see a lot of Physicist turned machine learners in this field. You might have something interesting going on with your own research finding, if you have not published it I would encourage you to do so. A possible venue is JMLR (journal of machine learning research, there editors are not scared of a little measure theory). BTW a surprising fact is that Vapnik's bounds are dimensionality independent, provided you are working with what are known as function classes with finite VC (Vapnki-Chervonenkis) dimension. Dimensionality is a big deal in m/c learning because the data they are interested in is typically quite high dimensional.
- graycat 13y ago> You might have something interesting going on with your own research finding, if you have not published it I would encourage you to do so. 'Information Sciences', 1999, "... Anomaly Detector". So look for 'zero day' problems in server farms and networks. Get a multidimensional, distribution-free hypothesis test where know false alarm rate in advance and can adjust it exactly in small steps over a wide range. What is novel is the 'applied probability' that permits the results with false alarm rate. The assumptions about the input data are realistic and, thus, do not permit doing as well as the Neyman-Pearson result, but, still, intuitively, the detection rate should be relatively high. There is a weaker sense, not in the paper, where the detection rate is as high as possible. I.e., essentially, for the selected false alarm rate, the 'critical region' where null hypothesis (a healthy system) is accepted has least possible Lebesgue measure. It appears that the input data can have dimensions 10, 100, 1000, etc. I have an algorithm that makes the computations fast, but it's not in the paper. The computing needed was a bit much when I published the paper, but that computing is quite reasonable now. For a system where really care a lot about reliability, performance, security, etc., it could be worthwhile to deploy what I cooked up. My current project is quite different but does need a server farm, and I might deploy my detector there. Then with that demo, if people like my detector, I might let people buy it.