11 ms·

Uh, guys with the big bucks: AL/ML are at most a tiny fraction of the material, tools, power, and value of the pure/applied math on the shelves of the research

Uh, guys with the big bucks: AL/ML are at most a tiny fraction of the material, tools, power, and value of the pure/applied math on the shelves of the research libraries. If you have a problem you want solved with AI/ML, 99 44/100% of the time you are better off going for what's on the shelves of the libraries, as taught in various high end grad schools, than with anything currently specifically AL/ML. So, broadly, go for work in statistics, optimization, stochastic processes, and optimal control. For a specific application, may want some work that stands on that existing material and is also at least somewhat original. E.g., the crucial technical core of my startup is some original applied math I derived based on pure/applied math that's long been on the shelves of the research libraries. For the valuable work of my startup, what's in AI/ML now is in comparison at best weak, nearly silly early grade school baby talk. Really, guys, 99 44/100% of the good stuff is still where it's long been -- on the shelves of the best research libraries. And for the education for that work, it's definitely NOT in departments of computer science. Instead look at selected programs in pure/applied math in some of the best research universities.

aub3bhat 9y ago
This is a spectacularly bad advice, which "off-the-shelves" research libraries that you mention? OpenCV, LibSVM, Weka or Matlab/Octave?? Most examples/implementations in OpenCV are outpaced by Deep Learning methods. TensorFlow, Caffe, PyTorch/Torch which today implement most of the state of the art methods were all written on average a year or two ago. >> E.g., the crucial technical core of my startup is some original applied math I derived based on pure/applied math that's long been on the shelves of the research libraries. For the valuable work of my startup, what's in AI/ML now is in comparison at best weak, nearly silly early grade school baby talk. Rather than vaguely mentioning libraries and startup, I suggest you offer concrete evidence behind your claims. >> And for the education for that work, it's definitely NOT in departments of computer science. Instead look at selected programs in pure/applied math in some of the best research universities. This is just truthiness (Math feels more "hardcore" than CS so Math department must be the better source.). Having known several people at Google/Apple/Facebook-FAIR, CS departments are typically major source of AI/ML researchers.
- guiambros 9y ago
  > ... which "off-the-shelves" research libraries that you mention? OpenCV, LibSVM, Weka or Matlab/Octave?? I think they meant "library" in the literal sense - the one with books -, and not "software libraries". But yes, while common sense, it's empty of any meaning. Like "it's better to invest in the core math and statistical fundamentals than the specific applications". Right, but so what? Folks like DeepMind, Numenta, etc are developing new core research material, and applying in practice.
- graycat 9y ago
  > This is a spectacularly bad advice, which "off-the-shelves" research libraries that you mention? Harvard, Princeton, MIT, Stanford, Berkeley, U. Chicago, Johns Hopkins, Cal Tech, need I go on? It's very good advice; sorry you don't understand it. In short, the computer science people know how to write software, but for exploiting pure/applied math they don't know what software to write; that needs a mathematician, and the computer science profs rarely have enough background in math to do well with the subject. The math is based on pure math and is heavily theorems and proofs, and about the only way to be good with that material is to be a BS, MS, and hopefully Ph.D. in pure/applied math. The computer science AI/ML work has obtained some good results; just why the techniques work is too often still a mystery. But the emphasis on gigantic quantities of data make the work a niche and real applications rare.
  - nl 9y ago
    > The computer science AI/ML work has obtained some good results; just why the techniques work is too often still a mystery. But the emphasis on gigantic quantities of data make the work a niche and real applications rare. From an industrial point of view, that 'niche' includes a large proportion of the consumer internet and the 'rare' applications include online advertising, which powers two of the four most valuable companies on earth.
- raverbashing 9y ago
  > Most examples/implementations in OpenCV are outpaced by Deep Learning methods. Right. Except that: - In some cases there are very good (and fast) techniques for finding out what you need quickly (like identifying faces) - You don't want to retrain a network for your specific case (which might not have an existing model already) - Traditional solutions might be good enough - OpenCV is much more than just identifying images, it has several APIs like geometrical transformation, color processing, filtering, etc http://docs.opencv.org/2.4/modules/refman.html http://docs.opencv.org/2.4/modules/refman.html
rjdagost 9y ago
You've got the podium here- tell us specifically what we're missing by focusing on AI/ML. I am curious.
- graycat 9y ago
  The math, enough to make it mathematical, complete with theorems and proofs. Examples are in mathematical and applied statistics, optimization, applied probability, stochastic processes, control theory, and more. There is a lot in just statistical hypothesis test -- Kolmogorov-Smirnov with a really cute stochastic process foundation. Then there is a small ocean of non-parametric tests. Or, suppose we want a statistical hypothesis test that is both multi-dimensional and distribution-free and with Ulam's 'tightness' is not trivial. Well, we'd like the most powerful test -- how to do that? There's essentially nothing in current AI/ML that is new and would work for such statistics. For a 50,000 foot view, pure and applied math are huge, old, deep fields also awash in applications. What's in AL/ML so far now is tiny in comparison.
  - sja 9y ago
    While I agree that it is vital for a practitioner know the mathematical basis behind what they're doing in order to maximize their chances of success, I find your current argument lacks substance. You argue that there is a vast amount of solved problems in tomes of "pure" mathematics. But this doesn't mean that any of those solutions are relevant and/or necessary to the problems people are working on. Your "statistical test" argument is misleading. Machine learning techniques aren't designed to form such statistics, and judging them by their ability to do so willfully disregards the real world success many companies have found by using ML/AI to solve their problems. Certainly, there are challenges that are waiting to have their solutions implemented from a rediscovered manuscript, but listing impressive-sounding statistical terminology doesn't demonstrate lack of value for machine learning/AI techniques. Furthermore, it doesn't even demonstrate value for the "pure" techniques you list. To clarify, I'm talking about real-world value, not just theoretical value (and again, I'm not arguing that such techniques don't have value, just that your argument isn't doing a good job communicating how such techniques have orders of magnitude more value than ML/AI).
blazespin 9y ago
These are aquihires, teams that have gelled and accomplished things. Generally don't find them in universities. And why do you assume they're not filled with a lot of stats phds?
- graycat 9y ago
  If only because there aren't many Ph.D.s in statistics with good pure/applied math backgrounds. Likely the biggest supply is for bio-statistics.
TeMPOraL 9y ago
For those who, like me, see it for the first time - "99 44/100%" is read as "99 44⁄100%", and it was a slogan for a soap. https://en.wikipedia.org/wiki/Ivory_(soap) https://en.wikipedia.org/wiki/Ivory_(soap)
- annnnd 9y ago
  And for those who, like me, still didn't get it - it was a slogan for a soap and reads "(99 + 44/100)%", in other words: 99.44% pure. Thanks TeMPOraL!
joshu 9y ago
Every time industry starts figuring stuff out, bitter academics always show up to explain things to them.
sbov 9y ago
It's just in its gold rush phase. It happens with everything. It happened with the PC, the web, mobile, even Facebook apps, and it is happening to ML and VR now. Everyone is piling on to dominate before the winners are solidified and it's too late.
fizixer 9y ago
> it's definitely NOT in departments of computer science. Instead look at selected programs in pure/applied math in some of the best research universities. Slight qualification: It's in pure/applied math in some of the best research universities, BUT, it needs a CS treatment, i.e., it needs to be utilized as numerical programming, not pen-and-paper math. I would also claim that the pure/applied math guys (MS, PhDs) don't have much edge here unless they shed their pen-and-paper habits and embrace computer-science. (likewise, CS MS,PhDs don't have an edge as long as they keep calling their discrete-algorithms class, free of numerical analysis and scientific computing, Algorithms! the exceptions are computer-graphics researchers, DSP, numerical-analysists etc). And those who are good at both pure/applied-math as well as programming/software-engineering, (cough humblebrag cough) boy...!
- graycat 9y ago
  Numerical analysis is an old field. Some of it was condition number of matrices and the resulting error analysis. Some of it was, say, double precision inner product accumulation and, for solving linear systems, iterative improvement. There was a lot more for solving ordinary (e.g., stiff) and partial differential equations. Courant was good at that. Too soon, e.g., trying to solve the Navier Stokes equations, numerical analysis gets quite deep into pure/applied math -- there's no royal road there through the CS departments and no royal road at all. E.g., a good prerequisite for work in numerical analysis is a high end course in linear algebra, e.g., matrix norms, Gershgorin approximations; not many people teach or take such a course. E.g., polynomial curve fitting can quickly lead people to solving a systems of statistics normal equations with the notoriously ill-conditioned Hilbert matrix. But, with some orthogonal polynomial approaches, can get around the numerical problems right away. I did that once for some profs at Georgetown U. I'd learned how to do that consulting in applied math and numerical analysis. For more, the days of ill-conditioned matrices led M. Newman to multiply the equations by large powers of 10 so that he had all whole numbers. Then for each prime number in a list of prime numbers, he solved the equations in the field of the integers modulo that prime. Right, for the multiplicative inverse, use the Euclidean greatest common divisor algorithm. Cute. Then use the Chinese remainder theorem to glue together the answers as quotients of multiple precision whole numbers. Ah, numerical analysis! Guys, that's not computer science, no matter how many nVidia GPUs they keep busy.
TulliusCicero 9y ago
What uselessly vague and implausible advice: "Hey people who've made incredible progress in a variety of fields the last several years using Approach X; Approach X is actually worthless, using Approach Y will blow it away! I, random anonymous internet poster, guarantee it! Exactly how would you use Approach Y, you ask? Well, just read and figure it out yourselves, duh!"
- graycat 9y ago
  What I wrote is fine. You have a rewrite that is not fine. For a better rewrite, there is a toolbox. Somewhat new X is in the toolbox. But the box also has a huge collection of other tools. If you have a real problem and some data and need a tool, then the odds that X is your best tool are low and, instead, one of the old tools is likely better. What old tools? The shelves of the research libraries are awash in them. There are books and journals, rows and rows of them. How to know which tools to apply? Need an education in the associated pure/applied math. Uh, the old tools are mostly not from and not taught in departments of computer science.
nl 9y ago
In my experience there is a huge gap between research at universities, the applied R&D done at industrial labs and the applications of research done at companies. All are valuable in their own way, but research maths has a big gap before it is useful for a company. (Source: I run a multi-university R&D project, including an applied math department.)
- graycat 9y ago
  > All are valuable in their own way, but research maths has a big gap before it is useful for a company. Counterexample: Consider the simple, first order, linear, ordinary differential equation initial value problem y'(t) = k y(t) (b - y(t)) I was feasting on picnic pork shoulder BBQ in Memphis when I saw that and with freshman calculus (never took it, taught it to myself, started on sophomore calculus) got the closed form solution. Don't even need a course in differential equations, variation of parameters or any of that! Well, one Saturday morning my solution kept the two representatives of FedEx BoD Member General Dynamics from walking out; they cancelled their plane reservations back to Texas and stayed; FedEx was saved from going out of business. So a little bit of math work helped a major business! Yup, gaps, that's the usual situation. But, early in my education, I didn't know those facts of life about gaps, etc. and believed that the stuff I was studying in math and physics would be useful, to make money, for a house, wife, kids, etc. But, naive or not, in the end I got a good education in pure/applied math. Now I'm less naive. But, now I'm an entrepreneur: I can pick a good real problem -- as we know, good problem selection is important for success. And there's no law against my drawing on all I learned in pure/applied math to derive some new math to get the first good solution to an important problem. I did that for zero day monitoring of server farms and networks, but for success, the $ kind, I was in effect depending on others to make the practical application. I had some high quality flour but was not starting Domino's Pizza. So, being too naive too often in the past, I decided to go to where the money was, to have a Web site that would get a lot of traffic, run ads from ad networks, get lots of clicks, and get paid by the click -- be 100% owner. How to do that? First, pick a problem. Really, pick a pair of problem and a solution, but, to keep it simple, first pick a problem. Right, if in the second step can't find a good solution, then loop back and pick another problem. So, there's the Internet and computing, both not nearly fully exploited. And there's ballpark 5 billion people on the Internet. Maybe in the more developed countries that can support good ad rates, there are maybe 400 million good Internet users. So, then, find a problem these 400 miliion or 5 billion people would very much like to have solved, want a new solution on average once a week, and will be willing on average to look at screens from my Web site for 30 minutes a week and, there, see maybe 50 ads with decent ad targeting. Multiply that out, and it's a successful business. Get optimistic and excited multiplying that out and get an estimate that it's the first $1 T business. Heck, get on average 24 x 7 one user a second, and it's a nice life style business -- plenty for house, wife, kids, etc. (uh, i've never had money enough to buy a house -- I'd like to be able to buy a house; I'm not joking about that, not even a little bit; if I'd had money enough to buy a house my wife might still be alive; I want money enough to buy a house; I know with 100% certainty I will never have money enough to buy a house unless I start, own, and run my own successful business -- period.). So, I found such a problem. It's not solved worth a darn -- the best solutions totally suck. The solutions are so bad no one even suspects that there could be a solution and, thus, fails even to notice the problem. Well, a solution is not trivial: Routine applications programming won't work. That AL/ML would work is a LOL joke. The applied math for a solution is not on the shelves of the libraries. So, I found some new math for a new solution. The work was not too difficult -- I've done high quality, new math before. I never had any trouble; get good at understanding the theorems and proofs and working the exercises in books by Halmos, Rudin, Fleming, Coddington, Royden, Loeve, Neveu, Breiman, Bertsekas, Hadley, Herstein, Hildebrand, Cinlar, Nemhauser, Zangwill, Luenberger, Simmons, Kelley, Suppes, Tukey, etc. and then maybe will be able to do publishable work in math. Then I wrote the software. It appears to run fine. I wrote the software in Microsoft's Visual Basic .NET -- looked, still looks, like a fine approach to me. So, no LISP, no functional programming, no particular concern about object oriented architecture, ignored JavaScript, nearly ignored CSS, never used an HTML div element, no attention to languages intended to make high levels of parallelism easy and automatic, ignored the model, view, controller Web software architecture, just typed into my favorite text editor and never used an interactive development environment, had no problems with debugging (it was enough just to write little trace messages to the Web site log file), just kept it simple. Seems fine to me. I'm on the way to going live and getting revenue. I'm a sole, solo founder, entrepreneur. I can do math and write code; there's no law against it; I don't need permission; and maybe my work will make money -- I believe there is a huge chance. If people like my work, then I'll be rich. Yes, on the gaps, one of my Ph.D. dissertation advisers told me with high concern that I should consider how long it takes to get original research into practice. Immediately I told him "I'll put it into practice right away.". That's what I'm trying to do. Uh, he assumed that I wanted to be a college prof; nope, I wanted to make money, the green kind, in business, the green money making kind. Gap between academics and applications? Not for me, now. Sure, I suffered from that gap, but now I look at the flip side and see a big opportunity.
turel 9y ago
I like your style. I am really glad that you have the resources to go it alone, against the zeitgeist. I don't think any big VC will take you up on your advice though. VCs don't work in vacuum, they need to exit at one point, and they need something that is sexy to sell at that point. AI/ML is sexy, and will be for at least the next 5-10 years. BTW, my advice to you is to be careful that the zeitgeist does not get you. Even when you want to hire a dev-ops guy, you need to sell the guy a vision. And if the vision is "we don't to sexy ML, we do dull statistics", you will not be getting the best guys. Just noticed you are being down voted. That is the zeitgeist getting to you my friend. Think how many good employees you are ditching :)
- graycat 9y ago
  We just left the AI Spring of Hope. Now we are in the Summer of Heat. Soon will be the Fall of Failure and, then, once again, maybe this time a real ice age, an AI Winter. There is old history here: (A) Sure, if count self-driving cars as AI, then that's new software, laser range finders, etc. Okay. (B) Otherwise what business wants from AL/ML is solving essentially just existing problems of their existing business. So, they have some business operations and want to do better. Well, that situation goes way back to the field of operations research. From about 1950-1970, that was hot stuff, especially for the US DoD. Well, there is some value there. But there was also a lot of hype. To get the value, usually have to select problems carefully and then do good work. Also, then, there were big bottlenecks on (i) understanding the relevant applied math, (ii) finding appropriate applications, (iii) gathering the input data, (iv) getting the software written and running, (v) being able to afford the computer time. So, yes, there were some successes. Later, I had some. But with the hype which led to low quality efforts, there was also a lot of failure. Net, soon the phrase went "operations research is dead". Similarly for applied statistics. Similarly quite broadly for applications of math to business operations. There were, and are, e.g., RSA encryption, some valuable, narrow applications, but broadly the applied math flopped. Statistics? Well, it's continued to get used where it's really important -- e.g., industrial quality control, bio-medical challenges, and experimental design, e.g., in research in agriculture. Okay, now AI/ML are being hyped for applications to business. Uh, as above, we've been there and done that. Valuable? It can be. Easy? Much easier now than decades ago but still, usually not really easy. Problems with AI/ML now, remembering that mostly we're still talking about solving business problems, e.g., much as in the 100 year old Taylor time and motion studies in assembly lines, much as in logistics, transportation, inventory, facility location, ad targeting, marketing: (i) The successful, new tools of AI/ML have darned narrow successes and are the 100 - 99 44/100% of the total of the promising tools available. (ii) Since the 99 44/100% tools still are not doing well enough to get all excited (right, there is some use of Linpack, C-PLEX, SPSS, SAS, R, ODE solvers, statistical hypothesis tests, linear multivariate statistics, time series analysis, etc.), getting all excited by the AI/ML in the 100 - 99 44/100% is a wild overshoot. The overshoot is also a waste. Here I'm just telling business guys tempted to spend money on what are essentially applied math/computing projects to save/make money in their businesses that they would do well, actually much better, just to review their courses in linear programming, production and operations, applied math of marketing, and statistics they got in their BS/MBA B-school studies. That material was rock solid, nicely polished, well justified, nicely balanced, prudent, not awash in hype, and from a long track record of successful real projects that were quite valuable. There's a lot more than what was covered in the BS/MBA B-school programs, but for a consumer introduction that material was quite good -- it was intended to be by the degree accrediting groups. > I don't think any big VC will take you up on your advice though. My view of VCs is that they are puppets on the ends of strings held by the limited partners (LPs) who still want to think like commercial bankers advised by CPAs. So, they want to invest capital in assets. They believe that they have bent over backwards far enough to regard traction significant and growing rapidly as a necessary and sufficient asset, but anything before such traction and a dime won't cover a 10 cent cup of coffee. I doesn't matter what I say, explain, reference, etc.: The LPs won't let the VCs pay attention. > VCs don't work in vacuum, they need to exit at one point, and they need something that is sexy to sell at that point. AI/ML is sexy, and will be for at least the next 5-10 years. I believe your "5-10" is too long (is that the expected time to exit or the jail term for fraud -- sarcasm here!). Generaly, once projects start to fail, the bad vibes spread very quickly. For real business uses, in general (not just very narrow niches), AI/ML is in hype mode. The time to the big flop is based on the hype, not the technology of AI/ML. But we've seen hype rise and fall before. So, to estimate the time to flop, just look, first cut, at the history of hype. The usual exit time for a VC investment and time to full liquidation of a VC fund is ballpark 8-15 years. Well, the hype will flop long before that. So, except for a fast flip, M&A, to someone with more money than brains, AL/ML, aging like butterflies or flowers in spring, will die too fast for the VCs. For my startup, there's a crucial, technical core, based on my original applied math derivations (I've published in statistics, but the math for my startup is not statistics), but the rest just looks like a promising startup. The tech core provides crucial enabling of delivering the good results -- I can't think of any way to get the good results otherwise, and current, hot AI/ML is really just weak and/or the wrong stuff. E.g., what I derived just ain't convolutional neural networks trained with terabytes of data or anything like that. And the input data I'm using is nowhere near terabytes. If there are some good, practical, valuable, doable, real business applications for convolutional neural networks trained with terabytes of data, terrific. Then that's (heuristic, curve fitting) applied math application n + 1 for some huge n already on the shelves of the research libraries. Or, look, the computer science departments can teach how to build at least the software side of computer systems, from the ROM BIOS for booting the thing up to complicated applications -- running a bank, a lot in medical records, maybe air traffic control. Fine. But with AI/ML, the CS departments are reaching to solve the problems the applied math people have been working on for decades, maybe centuries. There the main issue is not how to program but WHAT to program. And to answer the WHAT for an applied math application, darned better well be well trained in applied math. Well, nearly none of the CS profs are so well trained, and that's a problem. Yes, the applied math appllications will likely need a lot of computing, but it does not follow that the people to do the applied math are the CS people. Sorry 'bout that. I'd like to hear the opinions of E. Cinlar at Princeton. For my startup, I wrote the code. It's 24,000 programming language statements in Visual Basic .NET Framework 4 in 100,000 lines of typing. It all is like I originally envisioned and all runs, apparently correctly (although I want to do some more testing). The code is nicely documented and plenty efficient enough for production. Only a little of the code has the core applied math, and the rest is on the simple side of routine for a Web site that interacts with users. So, really, so far, I don't need to recruit and hire software developers. And if they would need to know about the crucial technical core, then it's some original applied math complete with theorems and proofs with some advanced pure/applied math prerequisites and not AI/ML, or statistics. For judging if people will be good for my startup, I believe I can do that; if they are highly impressed with AL/ML, then that would be a letter grade down! For the status, last night I wrote two, simple scripts to help me understand where my disk space is being used; my old scripts got sick on file sizes > 2^32. Then with the output from those two scripts, I'll move around some data and do an incremental backup. Then I'll check about 10 times to be careful and then restore my main boot partition from an old copy made by NTBACKUP. I need to restore because I had a hardware problem that corrupted some of the code on my boot partition. That's the nature of the work at present -- e.g., system administration mud wrestling. Then back to testing my software on the way to going live, ASAP. Then some more routine steps, and then going live. It's been a long time: The parts of the work unique to me have all been fast, fun, and easy. But there were independent, exogenous interruptions -- maybe now I've swatted down nearly all of those. I don't anticipate any equity funding. Since I'm a solo founder, by the time the VCs would write me a check, I'll have so much traction, and, thus, revenue, that I would have no good reason to accept their check -- no, 1/3rd of my business is not for sale yet! Net, for my startup, the LPs want their VCs to wait too long.

aub3bhat 9y ago

This is a spectacularly bad advice, which "off-the-shelves" research libraries that you mention? OpenCV, LibSVM, Weka or Matlab/Octave?? Most examples/implementations in OpenCV are outpaced by Deep Learning methods. TensorFlow, Caffe, PyTorch/Torch which today implement most of the state of the art methods were all written on average a year or two ago. >> E.g., the crucial technical core of my startup is some original applied math I derived based on pure/applied math that's long been on the shelves of the research libraries. For the valuable work of my startup, what's in AI/ML now is in comparison at best weak, nearly silly early grade school baby talk. Rather than vaguely mentioning libraries and startup, I suggest you offer concrete evidence behind your claims. >> And for the education for that work, it's definitely NOT in departments of computer science. Instead look at selected programs in pure/applied math in some of the best research universities. This is just truthiness (Math feels more "hardcore" than CS so Math department must be the better source.). Having known several people at Google/Apple/Facebook-FAIR, CS departments are typically major source of AI/ML researchers.

guiambros 9y ago
> ... which "off-the-shelves" research libraries that you mention? OpenCV, LibSVM, Weka or Matlab/Octave?? I think they meant "library" in the literal sense - the one with books -, and not "software libraries". But yes, while common sense, it's empty of any meaning. Like "it's better to invest in the core math and statistical fundamentals than the specific applications". Right, but so what? Folks like DeepMind, Numenta, etc are developing new core research material, and applying in practice.
graycat 9y ago
> This is a spectacularly bad advice, which "off-the-shelves" research libraries that you mention? Harvard, Princeton, MIT, Stanford, Berkeley, U. Chicago, Johns Hopkins, Cal Tech, need I go on? It's very good advice; sorry you don't understand it. In short, the computer science people know how to write software, but for exploiting pure/applied math they don't know what software to write; that needs a mathematician, and the computer science profs rarely have enough background in math to do well with the subject. The math is based on pure math and is heavily theorems and proofs, and about the only way to be good with that material is to be a BS, MS, and hopefully Ph.D. in pure/applied math. The computer science AI/ML work has obtained some good results; just why the techniques work is too often still a mystery. But the emphasis on gigantic quantities of data make the work a niche and real applications rare.
- nl 9y ago
  > The computer science AI/ML work has obtained some good results; just why the techniques work is too often still a mystery. But the emphasis on gigantic quantities of data make the work a niche and real applications rare. From an industrial point of view, that 'niche' includes a large proportion of the consumer internet and the 'rare' applications include online advertising, which powers two of the four most valuable companies on earth.
raverbashing 9y ago
> Most examples/implementations in OpenCV are outpaced by Deep Learning methods. Right. Except that: - In some cases there are very good (and fast) techniques for finding out what you need quickly (like identifying faces) - You don't want to retrain a network for your specific case (which might not have an existing model already) - Traditional solutions might be good enough - OpenCV is much more than just identifying images, it has several APIs like geometrical transformation, color processing, filtering, etc http://docs.opencv.org/2.4/modules/refman.html http://docs.opencv.org/2.4/modules/refman.html

rjdagost 9y ago

You've got the podium here- tell us specifically what we're missing by focusing on AI/ML. I am curious.

graycat 9y ago
The math, enough to make it mathematical, complete with theorems and proofs. Examples are in mathematical and applied statistics, optimization, applied probability, stochastic processes, control theory, and more. There is a lot in just statistical hypothesis test -- Kolmogorov-Smirnov with a really cute stochastic process foundation. Then there is a small ocean of non-parametric tests. Or, suppose we want a statistical hypothesis test that is both multi-dimensional and distribution-free and with Ulam's 'tightness' is not trivial. Well, we'd like the most powerful test -- how to do that? There's essentially nothing in current AI/ML that is new and would work for such statistics. For a 50,000 foot view, pure and applied math are huge, old, deep fields also awash in applications. What's in AL/ML so far now is tiny in comparison.
- sja 9y ago
  While I agree that it is vital for a practitioner know the mathematical basis behind what they're doing in order to maximize their chances of success, I find your current argument lacks substance. You argue that there is a vast amount of solved problems in tomes of "pure" mathematics. But this doesn't mean that any of those solutions are relevant and/or necessary to the problems people are working on. Your "statistical test" argument is misleading. Machine learning techniques aren't designed to form such statistics, and judging them by their ability to do so willfully disregards the real world success many companies have found by using ML/AI to solve their problems. Certainly, there are challenges that are waiting to have their solutions implemented from a rediscovered manuscript, but listing impressive-sounding statistical terminology doesn't demonstrate lack of value for machine learning/AI techniques. Furthermore, it doesn't even demonstrate value for the "pure" techniques you list. To clarify, I'm talking about real-world value, not just theoretical value (and again, I'm not arguing that such techniques don't have value, just that your argument isn't doing a good job communicating how such techniques have orders of magnitude more value than ML/AI).

blazespin 9y ago

These are aquihires, teams that have gelled and accomplished things. Generally don't find them in universities. And why do you assume they're not filled with a lot of stats phds?

graycat 9y ago
If only because there aren't many Ph.D.s in statistics with good pure/applied math backgrounds. Likely the biggest supply is for bio-statistics.

TeMPOraL 9y ago

For those who, like me, see it for the first time - "99 44/100%" is read as "99 44⁄100%", and it was a slogan for a soap. https://en.wikipedia.org/wiki/Ivory_(soap) https://en.wikipedia.org/wiki/Ivory_(soap)

annnnd 9y ago
And for those who, like me, still didn't get it - it was a slogan for a soap and reads "(99 + 44/100)%", in other words: 99.44% pure. Thanks TeMPOraL!

joshu 9y ago

Every time industry starts figuring stuff out, bitter academics always show up to explain things to them.

sbov 9y ago

It's just in its gold rush phase. It happens with everything. It happened with the PC, the web, mobile, even Facebook apps, and it is happening to ML and VR now. Everyone is piling on to dominate before the winners are solidified and it's too late.

fizixer 9y ago

> it's definitely NOT in departments of computer science. Instead look at selected programs in pure/applied math in some of the best research universities. Slight qualification: It's in pure/applied math in some of the best research universities, BUT, it needs a CS treatment, i.e., it needs to be utilized as numerical programming, not pen-and-paper math. I would also claim that the pure/applied math guys (MS, PhDs) don't have much edge here unless they shed their pen-and-paper habits and embrace computer-science. (likewise, CS MS,PhDs don't have an edge as long as they keep calling their discrete-algorithms class, free of numerical analysis and scientific computing, Algorithms! the exceptions are computer-graphics researchers, DSP, numerical-analysists etc). And those who are good at both pure/applied-math as well as programming/software-engineering, (cough humblebrag cough) boy...!

graycat 9y ago
Numerical analysis is an old field. Some of it was condition number of matrices and the resulting error analysis. Some of it was, say, double precision inner product accumulation and, for solving linear systems, iterative improvement. There was a lot more for solving ordinary (e.g., stiff) and partial differential equations. Courant was good at that. Too soon, e.g., trying to solve the Navier Stokes equations, numerical analysis gets quite deep into pure/applied math -- there's no royal road there through the CS departments and no royal road at all. E.g., a good prerequisite for work in numerical analysis is a high end course in linear algebra, e.g., matrix norms, Gershgorin approximations; not many people teach or take such a course. E.g., polynomial curve fitting can quickly lead people to solving a systems of statistics normal equations with the notoriously ill-conditioned Hilbert matrix. But, with some orthogonal polynomial approaches, can get around the numerical problems right away. I did that once for some profs at Georgetown U. I'd learned how to do that consulting in applied math and numerical analysis. For more, the days of ill-conditioned matrices led M. Newman to multiply the equations by large powers of 10 so that he had all whole numbers. Then for each prime number in a list of prime numbers, he solved the equations in the field of the integers modulo that prime. Right, for the multiplicative inverse, use the Euclidean greatest common divisor algorithm. Cute. Then use the Chinese remainder theorem to glue together the answers as quotients of multiple precision whole numbers. Ah, numerical analysis! Guys, that's not computer science, no matter how many nVidia GPUs they keep busy.

TulliusCicero 9y ago

What uselessly vague and implausible advice: "Hey people who've made incredible progress in a variety of fields the last several years using Approach X; Approach X is actually worthless, using Approach Y will blow it away! I, random anonymous internet poster, guarantee it! Exactly how would you use Approach Y, you ask? Well, just read and figure it out yourselves, duh!"

graycat 9y ago
What I wrote is fine. You have a rewrite that is not fine. For a better rewrite, there is a toolbox. Somewhat new X is in the toolbox. But the box also has a huge collection of other tools. If you have a real problem and some data and need a tool, then the odds that X is your best tool are low and, instead, one of the old tools is likely better. What old tools? The shelves of the research libraries are awash in them. There are books and journals, rows and rows of them. How to know which tools to apply? Need an education in the associated pure/applied math. Uh, the old tools are mostly not from and not taught in departments of computer science.

nl 9y ago

In my experience there is a huge gap between research at universities, the applied R&D done at industrial labs and the applications of research done at companies. All are valuable in their own way, but research maths has a big gap before it is useful for a company. (Source: I run a multi-university R&D project, including an applied math department.)

graycat 9y ago
> All are valuable in their own way, but research maths has a big gap before it is useful for a company. Counterexample: Consider the simple, first order, linear, ordinary differential equation initial value problem y'(t) = k y(t) (b - y(t)) I was feasting on picnic pork shoulder BBQ in Memphis when I saw that and with freshman calculus (never took it, taught it to myself, started on sophomore calculus) got the closed form solution. Don't even need a course in differential equations, variation of parameters or any of that! Well, one Saturday morning my solution kept the two representatives of FedEx BoD Member General Dynamics from walking out; they cancelled their plane reservations back to Texas and stayed; FedEx was saved from going out of business. So a little bit of math work helped a major business! Yup, gaps, that's the usual situation. But, early in my education, I didn't know those facts of life about gaps, etc. and believed that the stuff I was studying in math and physics would be useful, to make money, for a house, wife, kids, etc. But, naive or not, in the end I got a good education in pure/applied math. Now I'm less naive. But, now I'm an entrepreneur: I can pick a good real problem -- as we know, good problem selection is important for success. And there's no law against my drawing on all I learned in pure/applied math to derive some new math to get the first good solution to an important problem. I did that for zero day monitoring of server farms and networks, but for success, the $ kind, I was in effect depending on others to make the practical application. I had some high quality flour but was not starting Domino's Pizza. So, being too naive too often in the past, I decided to go to where the money was, to have a Web site that would get a lot of traffic, run ads from ad networks, get lots of clicks, and get paid by the click -- be 100% owner. How to do that? First, pick a problem. Really, pick a pair of problem and a solution, but, to keep it simple, first pick a problem. Right, if in the second step can't find a good solution, then loop back and pick another problem. So, there's the Internet and computing, both not nearly fully exploited. And there's ballpark 5 billion people on the Internet. Maybe in the more developed countries that can support good ad rates, there are maybe 400 million good Internet users. So, then, find a problem these 400 miliion or 5 billion people would very much like to have solved, want a new solution on average once a week, and will be willing on average to look at screens from my Web site for 30 minutes a week and, there, see maybe 50 ads with decent ad targeting. Multiply that out, and it's a successful business. Get optimistic and excited multiplying that out and get an estimate that it's the first $1 T business. Heck, get on average 24 x 7 one user a second, and it's a nice life style business -- plenty for house, wife, kids, etc. (uh, i've never had money enough to buy a house -- I'd like to be able to buy a house; I'm not joking about that, not even a little bit; if I'd had money enough to buy a house my wife might still be alive; I want money enough to buy a house; I know with 100% certainty I will never have money enough to buy a house unless I start, own, and run my own successful business -- period.). So, I found such a problem. It's not solved worth a darn -- the best solutions totally suck. The solutions are so bad no one even suspects that there could be a solution and, thus, fails even to notice the problem. Well, a solution is not trivial: Routine applications programming won't work. That AL/ML would work is a LOL joke. The applied math for a solution is not on the shelves of the libraries. So, I found some new math for a new solution. The work was not too difficult -- I've done high quality, new math before. I never had any trouble; get good at understanding the theorems and proofs and working the exercises in books by Halmos, Rudin, Fleming, Coddington, Royden, Loeve, Neveu, Breiman, Bertsekas, Hadley, Herstein, Hildebrand, Cinlar, Nemhauser, Zangwill, Luenberger, Simmons, Kelley, Suppes, Tukey, etc. and then maybe will be able to do publishable work in math. Then I wrote the software. It appears to run fine. I wrote the software in Microsoft's Visual Basic .NET -- looked, still looks, like a fine approach to me. So, no LISP, no functional programming, no particular concern about object oriented architecture, ignored JavaScript, nearly ignored CSS, never used an HTML div element, no attention to languages intended to make high levels of parallelism easy and automatic, ignored the model, view, controller Web software architecture, just typed into my favorite text editor and never used an interactive development environment, had no problems with debugging (it was enough just to write little trace messages to the Web site log file), just kept it simple. Seems fine to me. I'm on the way to going live and getting revenue. I'm a sole, solo founder, entrepreneur. I can do math and write code; there's no law against it; I don't need permission; and maybe my work will make money -- I believe there is a huge chance. If people like my work, then I'll be rich. Yes, on the gaps, one of my Ph.D. dissertation advisers told me with high concern that I should consider how long it takes to get original research into practice. Immediately I told him "I'll put it into practice right away.". That's what I'm trying to do. Uh, he assumed that I wanted to be a college prof; nope, I wanted to make money, the green kind, in business, the green money making kind. Gap between academics and applications? Not for me, now. Sure, I suffered from that gap, but now I look at the flip side and see a big opportunity.

turel 9y ago

I like your style. I am really glad that you have the resources to go it alone, against the zeitgeist. I don't think any big VC will take you up on your advice though. VCs don't work in vacuum, they need to exit at one point, and they need something that is sexy to sell at that point. AI/ML is sexy, and will be for at least the next 5-10 years. BTW, my advice to you is to be careful that the zeitgeist does not get you. Even when you want to hire a dev-ops guy, you need to sell the guy a vision. And if the vision is "we don't to sexy ML, we do dull statistics", you will not be getting the best guys. Just noticed you are being down voted. That is the zeitgeist getting to you my friend. Think how many good employees you are ditching :)

graycat 9y ago
We just left the AI Spring of Hope. Now we are in the Summer of Heat. Soon will be the Fall of Failure and, then, once again, maybe this time a real ice age, an AI Winter. There is old history here: (A) Sure, if count self-driving cars as AI, then that's new software, laser range finders, etc. Okay. (B) Otherwise what business wants from AL/ML is solving essentially just existing problems of their existing business. So, they have some business operations and want to do better. Well, that situation goes way back to the field of operations research. From about 1950-1970, that was hot stuff, especially for the US DoD. Well, there is some value there. But there was also a lot of hype. To get the value, usually have to select problems carefully and then do good work. Also, then, there were big bottlenecks on (i) understanding the relevant applied math, (ii) finding appropriate applications, (iii) gathering the input data, (iv) getting the software written and running, (v) being able to afford the computer time. So, yes, there were some successes. Later, I had some. But with the hype which led to low quality efforts, there was also a lot of failure. Net, soon the phrase went "operations research is dead". Similarly for applied statistics. Similarly quite broadly for applications of math to business operations. There were, and are, e.g., RSA encryption, some valuable, narrow applications, but broadly the applied math flopped. Statistics? Well, it's continued to get used where it's really important -- e.g., industrial quality control, bio-medical challenges, and experimental design, e.g., in research in agriculture. Okay, now AI/ML are being hyped for applications to business. Uh, as above, we've been there and done that. Valuable? It can be. Easy? Much easier now than decades ago but still, usually not really easy. Problems with AI/ML now, remembering that mostly we're still talking about solving business problems, e.g., much as in the 100 year old Taylor time and motion studies in assembly lines, much as in logistics, transportation, inventory, facility location, ad targeting, marketing: (i) The successful, new tools of AI/ML have darned narrow successes and are the 100 - 99 44/100% of the total of the promising tools available. (ii) Since the 99 44/100% tools still are not doing well enough to get all excited (right, there is some use of Linpack, C-PLEX, SPSS, SAS, R, ODE solvers, statistical hypothesis tests, linear multivariate statistics, time series analysis, etc.), getting all excited by the AI/ML in the 100 - 99 44/100% is a wild overshoot. The overshoot is also a waste. Here I'm just telling business guys tempted to spend money on what are essentially applied math/computing projects to save/make money in their businesses that they would do well, actually much better, just to review their courses in linear programming, production and operations, applied math of marketing, and statistics they got in their BS/MBA B-school studies. That material was rock solid, nicely polished, well justified, nicely balanced, prudent, not awash in hype, and from a long track record of successful real projects that were quite valuable. There's a lot more than what was covered in the BS/MBA B-school programs, but for a consumer introduction that material was quite good -- it was intended to be by the degree accrediting groups. > I don't think any big VC will take you up on your advice though. My view of VCs is that they are puppets on the ends of strings held by the limited partners (LPs) who still want to think like commercial bankers advised by CPAs. So, they want to invest capital in assets. They believe that they have bent over backwards far enough to regard traction significant and growing rapidly as a necessary and sufficient asset, but anything before such traction and a dime won't cover a 10 cent cup of coffee. I doesn't matter what I say, explain, reference, etc.: The LPs won't let the VCs pay attention. > VCs don't work in vacuum, they need to exit at one point, and they need something that is sexy to sell at that point. AI/ML is sexy, and will be for at least the next 5-10 years. I believe your "5-10" is too long (is that the expected time to exit or the jail term for fraud -- sarcasm here!). Generaly, once projects start to fail, the bad vibes spread very quickly. For real business uses, in general (not just very narrow niches), AI/ML is in hype mode. The time to the big flop is based on the hype, not the technology of AI/ML. But we've seen hype rise and fall before. So, to estimate the time to flop, just look, first cut, at the history of hype. The usual exit time for a VC investment and time to full liquidation of a VC fund is ballpark 8-15 years. Well, the hype will flop long before that. So, except for a fast flip, M&A, to someone with more money than brains, AL/ML, aging like butterflies or flowers in spring, will die too fast for the VCs. For my startup, there's a crucial, technical core, based on my original applied math derivations (I've published in statistics, but the math for my startup is not statistics), but the rest just looks like a promising startup. The tech core provides crucial enabling of delivering the good results -- I can't think of any way to get the good results otherwise, and current, hot AI/ML is really just weak and/or the wrong stuff. E.g., what I derived just ain't convolutional neural networks trained with terabytes of data or anything like that. And the input data I'm using is nowhere near terabytes. If there are some good, practical, valuable, doable, real business applications for convolutional neural networks trained with terabytes of data, terrific. Then that's (heuristic, curve fitting) applied math application n + 1 for some huge n already on the shelves of the research libraries. Or, look, the computer science departments can teach how to build at least the software side of computer systems, from the ROM BIOS for booting the thing up to complicated applications -- running a bank, a lot in medical records, maybe air traffic control. Fine. But with AI/ML, the CS departments are reaching to solve the problems the applied math people have been working on for decades, maybe centuries. There the main issue is not how to program but WHAT to program. And to answer the WHAT for an applied math application, darned better well be well trained in applied math. Well, nearly none of the CS profs are so well trained, and that's a problem. Yes, the applied math appllications will likely need a lot of computing, but it does not follow that the people to do the applied math are the CS people. Sorry 'bout that. I'd like to hear the opinions of E. Cinlar at Princeton. For my startup, I wrote the code. It's 24,000 programming language statements in Visual Basic .NET Framework 4 in 100,000 lines of typing. It all is like I originally envisioned and all runs, apparently correctly (although I want to do some more testing). The code is nicely documented and plenty efficient enough for production. Only a little of the code has the core applied math, and the rest is on the simple side of routine for a Web site that interacts with users. So, really, so far, I don't need to recruit and hire software developers. And if they would need to know about the crucial technical core, then it's some original applied math complete with theorems and proofs with some advanced pure/applied math prerequisites and not AI/ML, or statistics. For judging if people will be good for my startup, I believe I can do that; if they are highly impressed with AL/ML, then that would be a letter grade down! For the status, last night I wrote two, simple scripts to help me understand where my disk space is being used; my old scripts got sick on file sizes > 2^32. Then with the output from those two scripts, I'll move around some data and do an incremental backup. Then I'll check about 10 times to be careful and then restore my main boot partition from an old copy made by NTBACKUP. I need to restore because I had a hardware problem that corrupted some of the code on my boot partition. That's the nature of the work at present -- e.g., system administration mud wrestling. Then back to testing my software on the way to going live, ASAP. Then some more routine steps, and then going live. It's been a long time: The parts of the work unique to me have all been fast, fun, and easy. But there were independent, exogenous interruptions -- maybe now I've swatted down nearly all of those. I don't anticipate any equity funding. Since I'm a solo founder, by the time the VCs would write me a check, I'll have so much traction, and, thus, revenue, that I would have no good reason to accept their check -- no, 1/3rd of my business is not for sale yet! Net, for my startup, the LPs want their VCs to wait too long.