4 ms·

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-

Now that there's a bunch of AI/ML-related links in the front page, probably now is the best time to ask: As I learn deep learning, from the practical point-of-view, I found that the idea is simply to feed some "black box" with labeled data so next time it can give you correct label given unlabeled data. In essence, it's pattern recognition. What do you think? And then, as I try to find use cases for ML (you know, finding problem for the solution), I found that actually, many problems that can be solved with ML can actually be solved with rules. For example, detecting transaction fraud. You just need to find the right rules/formula. Forget ML, if you can't hardcode if-else, just use rules engine. What do you think? So, I'm starting to think that ML is good for solving problems where (1) we're too lazy to formulate the rules, or (2) the data is too complex/big to analyze by rules (as in, understanding image or voice). What do you think?

mmanfrin 10y ago
As I understand it, ML is about the finding of those rules. The big example I can think of is stocks -- what signals are important in determining the fiscal health of a company or stock? Every trader or investor has their own ruleset they believe in, ML (and specifically Quant) is meant to dig through as much data as possible to find those signals that affect the as-best-understood quality of a pick. For example, a human can think 'oh I will look at numbers like P/E and growth and...' whereas with ML you can feed it all those, and things like number of times the CEO has tweeted in the past week or the number of PR articles or even the language in PRs released to see if there is some strong correlative signal in the bunch, or if it is all just noise.
pouta 10y ago
I can't wait to read the answer to this!
tnecniv 10y ago
Figuring out the rules is the hardest part of programming (after choosing good names, haha).
cttet 10y ago
You are absolute right about most field of applications of ML. But in some fields like computer vision, we humans fail(at least till now) to make rules better than black box neural networks. Also, there are another group of ML people deal with rules+data, I quite agree with this article about the two types of ML: http://projecteuclid.org/download/pdf_1/euclid.ss/1009213726%20 http://projecteuclid.org/download/pdf_1/euclid.ss/1009213726...
micro_cam 10y ago
ML is about creating a process that produces models from data in a way that is likely to generalize to new data. For example if you just start trying lots of rules by hand on your fraud data set there is a good chance you'll come up with a rule that looks good on your data but doesn't generalize to new data. The number of models (or rules or formulas) you try by hand increases this chance (this related to multiple testing) and worse the process that generates them isn't repeatable since one you know a feature worked on the data set you're biased towards finding it again. So in ML you try to come up with a model generating process that is entirely automated and repeatable. This means you can do it repeatedly in cross validation, over bootstraps, out of time etc on the same data set and be fairly sure that it will generalize. The goal can still be a simple rule or formula but you achieve this simplicity for simplicity by penalizing complexity (as in lasso) or doing explicit simplification (as in pruning). The reason that complex "black box" models are so popular is that they often have really nice statistical properties in terms of generalization. It's fairly intuitive that averaging over a bunch of slightelly perterbed simple models will give you a nice combined model as in a random forest, gbm or other ensemble. Deep neural networks are less intuitive but it's been hypothesized that the depth makes them less prone to overfitting than a shallow network. A shallow network will have one global optimal set of weights that's simple and really good but it also has lots of locally optimal states that aren't as good...and the chances are high your training procedure (or manual search for a simple model that works) will get caught in one of these. It's been shown that the for deeper networks these local optima tend to be much closer in performance to the global optima so in effect making the model more complex makes it less likely you'll end up with a bad model.
- deleted 10y ago
  [deleted]
deleted 10y ago
[deleted]
visarga 10y ago
> Forget ML, if you can't hardcode if-else, just use rules engine. What do you think? Load data in X, model.fit(X, y), predictions = model.predict(new_X). It's not more complicated to apply ML.
akg_67 10y ago
What you are describing is the basis for decision tree and random forest.
signa11 10y ago
> (1) we're too lazy to formulate the rules, or (2) the data is too complex/big to analyze by rules in most of the cases we don't even know _how_ to formulate the rules. for example, if someone asks you, what makes an 'a' a 'a' ? what would be your response ?
nrao123 10y ago
I would recommend reading Paul Graham's Paul Graham's Plan for Spam- one of the early extremely useful ML approaches to fighting spam. http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html In the article he outlines the challenges of taking a rules based approach: The statistical approach is not usually the first one people try when they write spam filters. Most hackers' first instinct is to try to write software that recognizes individual properties of spam. You look at spams and you think, the gall of these guys to try sending me mail that begins "Dear Friend" or has a subject line that's all uppercase and ends in eight exclamation points. I can filter out that stuff with about one line of code. And so you do, and in the beginning it works. A few simple rules will take a big bite out of your incoming spam. Merely looking for the word "click" will catch 79.7% of the emails in my spam corpus, with only 1.2% false positives. I spent about six months writing software that looked for individual spam features before I tried the statistical approach. What I found was that recognizing that last few percent of spams got very hard, and that as I made the filters stricter I got more false positives. False positives are innocent emails that get mistakenly identified as spams. For most users, missing legitimate email is an order of magnitude worse than receiving spam, so a filter that yields false positives is like an acne cure that carries a risk of death to the patient. The more spam a user gets, the less likely he'll be to notice one innocent mail sitting in his spam folder. And strangely enough, the better your spam filters get, the more dangerous false positives become, because when the filters are really good, users will be more likely to ignore everything they catch.

mmanfrin 10y ago

As I understand it, ML is about the finding of those rules. The big example I can think of is stocks -- what signals are important in determining the fiscal health of a company or stock? Every trader or investor has their own ruleset they believe in, ML (and specifically Quant) is meant to dig through as much data as possible to find those signals that affect the as-best-understood quality of a pick. For example, a human can think 'oh I will look at numbers like P/E and growth and...' whereas with ML you can feed it all those, and things like number of times the CEO has tweeted in the past week or the number of PR articles or even the language in PRs released to see if there is some strong correlative signal in the bunch, or if it is all just noise.

pouta 10y ago

I can't wait to read the answer to this!

tnecniv 10y ago

Figuring out the rules is the hardest part of programming (after choosing good names, haha).

cttet 10y ago

You are absolute right about most field of applications of ML. But in some fields like computer vision, we humans fail(at least till now) to make rules better than black box neural networks. Also, there are another group of ML people deal with rules+data, I quite agree with this article about the two types of ML: http://projecteuclid.org/download/pdf_1/euclid.ss/1009213726%20 http://projecteuclid.org/download/pdf_1/euclid.ss/1009213726...

micro_cam 10y ago

ML is about creating a process that produces models from data in a way that is likely to generalize to new data. For example if you just start trying lots of rules by hand on your fraud data set there is a good chance you'll come up with a rule that looks good on your data but doesn't generalize to new data. The number of models (or rules or formulas) you try by hand increases this chance (this related to multiple testing) and worse the process that generates them isn't repeatable since one you know a feature worked on the data set you're biased towards finding it again. So in ML you try to come up with a model generating process that is entirely automated and repeatable. This means you can do it repeatedly in cross validation, over bootstraps, out of time etc on the same data set and be fairly sure that it will generalize. The goal can still be a simple rule or formula but you achieve this simplicity for simplicity by penalizing complexity (as in lasso) or doing explicit simplification (as in pruning). The reason that complex "black box" models are so popular is that they often have really nice statistical properties in terms of generalization. It's fairly intuitive that averaging over a bunch of slightelly perterbed simple models will give you a nice combined model as in a random forest, gbm or other ensemble. Deep neural networks are less intuitive but it's been hypothesized that the depth makes them less prone to overfitting than a shallow network. A shallow network will have one global optimal set of weights that's simple and really good but it also has lots of locally optimal states that aren't as good...and the chances are high your training procedure (or manual search for a simple model that works) will get caught in one of these. It's been shown that the for deeper networks these local optima tend to be much closer in performance to the global optima so in effect making the model more complex makes it less likely you'll end up with a bad model.

deleted 10y ago
[deleted]

deleted 10y ago

[deleted]

visarga 10y ago

> Forget ML, if you can't hardcode if-else, just use rules engine. What do you think? Load data in X, model.fit(X, y), predictions = model.predict(new_X). It's not more complicated to apply ML.

akg_67 10y ago

What you are describing is the basis for decision tree and random forest.

signa11 10y ago

> (1) we're too lazy to formulate the rules, or (2) the data is too complex/big to analyze by rules in most of the cases we don't even know _how_ to formulate the rules. for example, if someone asks you, what makes an 'a' a 'a' ? what would be your response ?

nrao123 10y ago

I would recommend reading Paul Graham's Paul Graham's Plan for Spam- one of the early extremely useful ML approaches to fighting spam. http://www.paulgraham.com/spam.html http://www.paulgraham.com/spam.html In the article he outlines the challenges of taking a rules based approach: The statistical approach is not usually the first one people try when they write spam filters. Most hackers' first instinct is to try to write software that recognizes individual properties of spam. You look at spams and you think, the gall of these guys to try sending me mail that begins "Dear Friend" or has a subject line that's all uppercase and ends in eight exclamation points. I can filter out that stuff with about one line of code. And so you do, and in the beginning it works. A few simple rules will take a big bite out of your incoming spam. Merely looking for the word "click" will catch 79.7% of the emails in my spam corpus, with only 1.2% false positives. I spent about six months writing software that looked for individual spam features before I tried the statistical approach. What I found was that recognizing that last few percent of spams got very hard, and that as I made the filters stricter I got more false positives. False positives are innocent emails that get mistakenly identified as spams. For most users, missing legitimate email is an order of magnitude worse than receiving spam, so a filter that yields false positives is like an acne cure that carries a risk of death to the patient. The more spam a user gets, the less likely he'll be to notice one innocent mail sitting in his spam folder. And strangely enough, the better your spam filters get, the more dangerous false positives become, because when the filters are really good, users will be more likely to ignore everything they catch.