5 ms·

You've got the podium here- tell us specifically what we're missing by focusing on AI/ML. I am curious.

graycat 9y ago
The math, enough to make it mathematical, complete with theorems and proofs. Examples are in mathematical and applied statistics, optimization, applied probability, stochastic processes, control theory, and more. There is a lot in just statistical hypothesis test -- Kolmogorov-Smirnov with a really cute stochastic process foundation. Then there is a small ocean of non-parametric tests. Or, suppose we want a statistical hypothesis test that is both multi-dimensional and distribution-free and with Ulam's 'tightness' is not trivial. Well, we'd like the most powerful test -- how to do that? There's essentially nothing in current AI/ML that is new and would work for such statistics. For a 50,000 foot view, pure and applied math are huge, old, deep fields also awash in applications. What's in AL/ML so far now is tiny in comparison.
- sja 9y ago
  While I agree that it is vital for a practitioner know the mathematical basis behind what they're doing in order to maximize their chances of success, I find your current argument lacks substance. You argue that there is a vast amount of solved problems in tomes of "pure" mathematics. But this doesn't mean that any of those solutions are relevant and/or necessary to the problems people are working on. Your "statistical test" argument is misleading. Machine learning techniques aren't designed to form such statistics, and judging them by their ability to do so willfully disregards the real world success many companies have found by using ML/AI to solve their problems. Certainly, there are challenges that are waiting to have their solutions implemented from a rediscovered manuscript, but listing impressive-sounding statistical terminology doesn't demonstrate lack of value for machine learning/AI techniques. Furthermore, it doesn't even demonstrate value for the "pure" techniques you list. To clarify, I'm talking about real-world value, not just theoretical value (and again, I'm not arguing that such techniques don't have value, just that your argument isn't doing a good job communicating how such techniques have orders of magnitude more value than ML/AI).
  - graycat 9y ago
    AI/ML now is too often just heuristic curve fitting. Okay, with good training data and then testing data, can get some utility that way. So, if that's the best can do in the context, okay. A secret in old applications of regression model building, that idea of cut the data in half, fit with one half and test with the other, was common and appropriate. But we'd like to do better. Some of what is in statistics and applied math more generally will let us do better. But the idea if dividing the data into training and testing is old. There's much more in the literature on how to build and evaluate models. E.g., there's analysis of variance and categorical data analysis (e.g., log linear). E.g., for my startup, a guy gave a talk and said that can't do that, encounter four problems. He was right about the four problems. And if just look in elementary texts, yup, do see the four problems. But I'd already seen all four and worked up some theorems and proofs to get around all four, in the context of my real problem. The foundation of the theorems and proofs was mostly some advanced pure math. Without new theorems and proofs, I'd been stuck like he was. In an important sense, the pure math guys are right: They are looking for the big, important structure and powerful properties, where all the furniture is in the room, turning on the lights so that can see all the furniture (A. Wiles), and that can be darned powerful stuff in practice. > You argue that there is a vast amount of solved problems in tomes of "pure" mathematics. But this doesn't mean that any of those solutions are relevant and/or necessary to the problems people are working on. Flour is a raw material. Some people can use it to make a fantastic Sacher Torte. Pure math can be regarded as a raw material. Some people can use it to get new, powerful, valuable results for real applications in practice. > orders of magnitude more value than ML/AI Those techniques are narrow and weak. The stuff long on the shelves of the libraries is much more broadly applicable and usually much more powerful. I've made a lot of applications of applied math, and not one of them would the AL/ML discussed these days be better and nearly never would be competitive.
    - srean 9y ago
      What 4 problems ?
- rjdagost 9y ago
  You’ve told us vague things about wide areas of mathematics, but you haven’t told us what problem your start-up is solving using these branches of mathematics or how it is a better allocation of investor dollars than focusing on ML. I’ve worked on a wide range of applied mathematics problems in machine learning, computer vision, signal / image processing, control theory, and applied statistics. Very few people outside of a mathematics department care at all about theorems, proofs, and "correctness" in the formal mathematical sense. I can count on one hand how many times I’ve needed to actually do a proof, and the number of people who have cared is smaller still. All anyone cares about is how well you can solve their problems. Sometimes advanced mathematics is the right tool for the job, but often it is not.
  - graycat 9y ago
    > You’ve told us vague things about wide areas of mathematics, but you haven’t told us what problem your start-up is solving using these branches of mathematics or how it is a better allocation of investor dollars than focusing on ML. Easy: ML is quite narrow. More options yield the same, usually better, investment results. Compared with the applied math on the shelves of the libraries, ML is nearly pinpoint narrow. In addition, too much of ML is heuristic curve fitting short on math guarantees. Also, where some real problems have some features that justify some math assumptions, ML is slow to exploit those. In the workshop, kitchen, or lab, use the best tools you can. So don't limit yourself to just what saw on a cooking show on TV. My view is that mathematical proof is a fantastically powerful part of applying math. The core reason is that it lets do some additional, maybe significantly original, derivations to get better solutions for real problems. E.g., once I was in an AI group working on monitoring and managing server farms and networks. We got rivers of data on many variables at high data rates. There we wanted to detect problems ASAP in near real time. Okay, necessarily, inescapably, we have two ways to be wrong -- false positives (claiming the system is sick when it's healthy) and false negatives (claiming the systems is healthy when it's sick). Okay, then, right away we are forced, like it or not, into the framework of some statistical hypothesis tests continually applied. There, as is usual in hypothesis testing, we want to be able to select a false alarm rate and get that rate exactly. We'd like the most powerful test as in the lemma of Neyman-Pearson but likely do not have enough data. Still we want test power in some good sense; if we are willing to tolerate a higher false alarm rate, then we'd like a higher detection rate; when we get a detection, we'd like to know, as a measure of seriousness, the lowest false alarm rate at which the data would still be a detection; at least we don't want a trivial test. For this data, no way can we know the probability distribution of the data under the null hypothesis (the system is healthy). This is because even the univariate data is commonly a total mess, far worse than was long common in statistics (U. Grenander explained this to me in his office at Brown). And, of course, with so much data on so many variables, we should be doing multi-variate testing. So, net we need tests that are both multi-variate and distribution-free. Go look for those in E. Lehmann, etc. and won't find any. So, I invented some. Then, how the heck to adjust false alarm rate? That was a problem in applied probability that needed new theorems and proofs. So, I did those. For those I used measure theory, measure preserving from ergodic theory, group theory from abstract algebra, and more. Yup, theorems and proofs. Otherwise I would not have known what the heck I had, what the properties were, what the false alarm rates were, etc. Then do my tests have any power? I didn't have data enough to apply Neyman-Pearson, but I found a way. Are my tests trivial? I used a result of S. Ulam and stated and proved a theorem -- not trivial. The work was intended to be practical. Likely and apparently the work remains the cat's meow, best in the world, for detection of problems never seen before -- behavioral monitoring, zero day monitoring. Gee, just read that the last security attack on Target cost them $300 million. They needed better monitoring! New theorems and proofs were crucial, and the work was intended to be practical. A guy had a fast opportunity to do some marketing but, with the short time interval, had limited resources. He had a 0-1 linear programming formulation, 40,000 constraints and 600,000 variables for a small, test case. Sure, no linear programming or integer linear programming package would have a chance. So, I got his formulation, saw some special structure to exploit, did some non-linear duality derivations for some Lagrangian relaxation, found how to get some bounds, wrote some code, and in 905 seconds on a slow computer found feasible solutions guaranteed to be within 0.025% of optimality. Well, the derivations were essentially theorems and proofs. He'd tried a computer science approach, simulated annealing. He ran for days and then stopped with the best he had and with no idea how close he was to optimality. My work, from theorems and proofs, was much, MUCH better. The crucial core applied math of my startup is some theorems and proofs; I wouldn't believe that the mathematical operations would yield what I want otherwise. Sure, usually a working applied mathematician doesn't always have to stir up new theorems and proofs, but at times that's darned helpful and valuable.

The math, enough to make it mathematical, complete with theorems and proofs. Examples are in mathematical and applied statistics, optimization, applied probability, stochastic processes, control theory, and more. There is a lot in just statistical hypothesis test -- Kolmogorov-Smirnov with a really cute stochastic process foundation. Then there is a small ocean of non-parametric tests. Or, suppose we want a statistical hypothesis test that is both multi-dimensional and distribution-free and with Ulam's 'tightness' is not trivial. Well, we'd like the most powerful test -- how to do that? There's essentially nothing in current AI/ML that is new and would work for such statistics. For a 50,000 foot view, pure and applied math are huge, old, deep fields also awash in applications. What's in AL/ML so far now is tiny in comparison.

sja 9y ago
While I agree that it is vital for a practitioner know the mathematical basis behind what they're doing in order to maximize their chances of success, I find your current argument lacks substance. You argue that there is a vast amount of solved problems in tomes of "pure" mathematics. But this doesn't mean that any of those solutions are relevant and/or necessary to the problems people are working on. Your "statistical test" argument is misleading. Machine learning techniques aren't designed to form such statistics, and judging them by their ability to do so willfully disregards the real world success many companies have found by using ML/AI to solve their problems. Certainly, there are challenges that are waiting to have their solutions implemented from a rediscovered manuscript, but listing impressive-sounding statistical terminology doesn't demonstrate lack of value for machine learning/AI techniques. Furthermore, it doesn't even demonstrate value for the "pure" techniques you list. To clarify, I'm talking about real-world value, not just theoretical value (and again, I'm not arguing that such techniques don't have value, just that your argument isn't doing a good job communicating how such techniques have orders of magnitude more value than ML/AI).
- graycat 9y ago
  AI/ML now is too often just heuristic curve fitting. Okay, with good training data and then testing data, can get some utility that way. So, if that's the best can do in the context, okay. A secret in old applications of regression model building, that idea of cut the data in half, fit with one half and test with the other, was common and appropriate. But we'd like to do better. Some of what is in statistics and applied math more generally will let us do better. But the idea if dividing the data into training and testing is old. There's much more in the literature on how to build and evaluate models. E.g., there's analysis of variance and categorical data analysis (e.g., log linear). E.g., for my startup, a guy gave a talk and said that can't do that, encounter four problems. He was right about the four problems. And if just look in elementary texts, yup, do see the four problems. But I'd already seen all four and worked up some theorems and proofs to get around all four, in the context of my real problem. The foundation of the theorems and proofs was mostly some advanced pure math. Without new theorems and proofs, I'd been stuck like he was. In an important sense, the pure math guys are right: They are looking for the big, important structure and powerful properties, where all the furniture is in the room, turning on the lights so that can see all the furniture (A. Wiles), and that can be darned powerful stuff in practice. > You argue that there is a vast amount of solved problems in tomes of "pure" mathematics. But this doesn't mean that any of those solutions are relevant and/or necessary to the problems people are working on. Flour is a raw material. Some people can use it to make a fantastic Sacher Torte. Pure math can be regarded as a raw material. Some people can use it to get new, powerful, valuable results for real applications in practice. > orders of magnitude more value than ML/AI Those techniques are narrow and weak. The stuff long on the shelves of the libraries is much more broadly applicable and usually much more powerful. I've made a lot of applications of applied math, and not one of them would the AL/ML discussed these days be better and nearly never would be competitive.
  - srean 9y ago
    What 4 problems ?
rjdagost 9y ago
You’ve told us vague things about wide areas of mathematics, but you haven’t told us what problem your start-up is solving using these branches of mathematics or how it is a better allocation of investor dollars than focusing on ML. I’ve worked on a wide range of applied mathematics problems in machine learning, computer vision, signal / image processing, control theory, and applied statistics. Very few people outside of a mathematics department care at all about theorems, proofs, and "correctness" in the formal mathematical sense. I can count on one hand how many times I’ve needed to actually do a proof, and the number of people who have cared is smaller still. All anyone cares about is how well you can solve their problems. Sometimes advanced mathematics is the right tool for the job, but often it is not.
- graycat 9y ago
  > You’ve told us vague things about wide areas of mathematics, but you haven’t told us what problem your start-up is solving using these branches of mathematics or how it is a better allocation of investor dollars than focusing on ML. Easy: ML is quite narrow. More options yield the same, usually better, investment results. Compared with the applied math on the shelves of the libraries, ML is nearly pinpoint narrow. In addition, too much of ML is heuristic curve fitting short on math guarantees. Also, where some real problems have some features that justify some math assumptions, ML is slow to exploit those. In the workshop, kitchen, or lab, use the best tools you can. So don't limit yourself to just what saw on a cooking show on TV. My view is that mathematical proof is a fantastically powerful part of applying math. The core reason is that it lets do some additional, maybe significantly original, derivations to get better solutions for real problems. E.g., once I was in an AI group working on monitoring and managing server farms and networks. We got rivers of data on many variables at high data rates. There we wanted to detect problems ASAP in near real time. Okay, necessarily, inescapably, we have two ways to be wrong -- false positives (claiming the system is sick when it's healthy) and false negatives (claiming the systems is healthy when it's sick). Okay, then, right away we are forced, like it or not, into the framework of some statistical hypothesis tests continually applied. There, as is usual in hypothesis testing, we want to be able to select a false alarm rate and get that rate exactly. We'd like the most powerful test as in the lemma of Neyman-Pearson but likely do not have enough data. Still we want test power in some good sense; if we are willing to tolerate a higher false alarm rate, then we'd like a higher detection rate; when we get a detection, we'd like to know, as a measure of seriousness, the lowest false alarm rate at which the data would still be a detection; at least we don't want a trivial test. For this data, no way can we know the probability distribution of the data under the null hypothesis (the system is healthy). This is because even the univariate data is commonly a total mess, far worse than was long common in statistics (U. Grenander explained this to me in his office at Brown). And, of course, with so much data on so many variables, we should be doing multi-variate testing. So, net we need tests that are both multi-variate and distribution-free. Go look for those in E. Lehmann, etc. and won't find any. So, I invented some. Then, how the heck to adjust false alarm rate? That was a problem in applied probability that needed new theorems and proofs. So, I did those. For those I used measure theory, measure preserving from ergodic theory, group theory from abstract algebra, and more. Yup, theorems and proofs. Otherwise I would not have known what the heck I had, what the properties were, what the false alarm rates were, etc. Then do my tests have any power? I didn't have data enough to apply Neyman-Pearson, but I found a way. Are my tests trivial? I used a result of S. Ulam and stated and proved a theorem -- not trivial. The work was intended to be practical. Likely and apparently the work remains the cat's meow, best in the world, for detection of problems never seen before -- behavioral monitoring, zero day monitoring. Gee, just read that the last security attack on Target cost them $300 million. They needed better monitoring! New theorems and proofs were crucial, and the work was intended to be practical. A guy had a fast opportunity to do some marketing but, with the short time interval, had limited resources. He had a 0-1 linear programming formulation, 40,000 constraints and 600,000 variables for a small, test case. Sure, no linear programming or integer linear programming package would have a chance. So, I got his formulation, saw some special structure to exploit, did some non-linear duality derivations for some Lagrangian relaxation, found how to get some bounds, wrote some code, and in 905 seconds on a slow computer found feasible solutions guaranteed to be within 0.025% of optimality. Well, the derivations were essentially theorems and proofs. He'd tried a computer science approach, simulated annealing. He ran for days and then stopped with the best he had and with no idea how close he was to optimality. My work, from theorems and proofs, was much, MUCH better. The crucial core applied math of my startup is some theorems and proofs; I wouldn't believe that the mathematical operations would yield what I want otherwise. Sure, usually a working applied mathematician doesn't always have to stir up new theorems and proofs, but at times that's darned helpful and valuable.