4 ms·

Why use the term "AI" when the GP specifically used the less ambiguous Machine Learning? Machine Learning = statistics + linear algebra + computer science, mos

Why use the term "AI" when the GP specifically used the less ambiguous Machine Learning? Machine Learning = statistics + linear algebra + computer science, mostly. Naive Bayes and Graphical Models are pure statistics, but they are mostly used for toy problems. Machine Learning scales these approaches to high dimensionality problems, and tasks where data is abundant.

gaius 8y ago
If someone had something as basic as logistic regression or k-means clustering in a production app they are already ahead of the vast majority of ML practitioners
- peatmoss 8y ago
  Gods, maybe I should stop telling people that I don’t have much in the way of a machine learning background.
nkozyra 8y ago
> Naive Bayes and Graphical Models are pure statistics, but they are mostly used for toy problems. I think they just feel "toy" because they've been used with great accuracy for so long. Complex problems are rarely solved by a single approach. The harder the problem the more likely a suite will be used, and often naive bayes will be part of that in some capacity.
- PeterisP 8y ago
  Can you give one example where Naive Bayes has been used with great accuracy for so long? Naive Bayes is a common baseline solution because it's very simple to implement and very fast to run; however, pretty much any other ML method gives better accuracy than Naive Bayes. Sometimes the accuracy improvement is small and doesn't justify the highly increased computing cost, though, so Naive Bayes becomes the appropriate method to use, but it's not because of its accuracy.
skj 8y ago
> Machine Learning scales these approaches to high dimensionality problems, and tasks where data is abundant. Machine learning is the use of computers to make decisions (or classifications) based on data without human intervention.
freehunter 8y ago
Funny, I would consider many ML usages to be toy problems. Coming up with funny nonsensical Shakespeare plays or generating random Pantone colors or Pokémon names isn’t something you hear from Bayesian systems. That’s the domain of ML. Recommending related products is again simple statistical analysis. And I can generate a prediction based on past performance real easy with some SQL and a couple of “if” statements. Just because Bayesian isn’t over-hyped doesn’t mean it doesn’t solve real problems. Not everything can be solved with ML, unless the problem is getting more investor money.
- kahnjw 8y ago
  I agree with all of this, except are you really implementing bayesian models in SQL? I'm not sure that is what the author is advocating in the first place. There are tools for bayesian everything, I much prefer using those over hacking together some illegible SQL script.
  - freehunter 8y ago
    No, it’s just SQL statements pulling data into the if...then in your preferred language. I’ve never done actual logic with SQL, not even sure if it’s possible.
    - JetSpiegel 8y ago
      Each DB engine has it's own incompatible imperative stored procedure language. Having worked with Oracle's version, and seen entire systems buult out of that, I dread it.
- peatmoss 8y ago
  There are some methods that I don’t see used outside of the machine learning community, but there appears to be a fair degree of overlap in actual methods used between classical stats and machine learning. Now, the biggest difference that I can see is that, within machine learning contexts, people are more concerned with the quality of the predictions than the interpretation of the independent variables. That’s not hard and fast, but it seems to be a common thread. And for some problems, making a good prediction is really the right thing. In other cases, understanding the mechanisms you might use to effect an outcome is more important. Both are valid uses of statistical methods, depending on the problem. I think within the realm of classical statistics, Bayesian methods’ super power is in being able to generate results that are much easier to communicate to lay people. Also Bayesian methods are nice if you want to do sensitivity analysis in a principled up front kind of way. But I could imagine using those methods in an ML context even if they aren’t the current darlings of the methodological pantheon.
dvogel 8y ago
AI and ML have different goals, which sometimes overlap. The goal of ML is to discover relationships in data that can only be (easily) observed by a machine. That usually serves as decision support data for a human decision maker or another more conventional algorithm. This is usually based on an entire data set at once. Accuracy and precision are valued over quickness and robustness. AI is about substituting human decision makers or other conventional computer decision algorithms. AI is used for situational decision making. It needs to be quick (like the conventional algorithms) while also recognizing the nuance and multi-dimemaional nature of a lot of decisions. Robustness and quickness are often valued more than precision. IOW the GP is only using ML for everything because it is a buzzword.
- PeterisP 8y ago
  ML is one of subdomains in the AI field, so instead of "sometimes overlap" it's technically correct to state that ML is a strict subset of AI. What you use as description for AI tempts me to use the "use it as a buzzword" angle back at you here - it is a stereotypical description of some use cases of AI approaches; but there are others - [machine] learning, knowledge representation, planning and scheduling, reasoning (both formal logic reasoning and also reasoning under uncertainty e.g. Bayesian approaches), intelligent agent representation, etc, are all parts of the AI field.
- tkxxx7 8y ago
  This last line seems unnecessarily rude. GP seems to understand well the two ML methods mentioned.
PeterisP 8y ago
Naive Bayes is machine learning. A very simple machine learning method, but machine learning. Also, the argument that it's pure statistics doesn't really fly - support vector machines, random forests or deep learning multilayer perceptrons are just as much statistics as probabilistic graphical models.
ravenstine 8y ago
Aren't Bayesian models essentially statistics + linear algebra? It sounds like you're characterizing ML as something beyond Bayes, which isn't really the case. I don't know what you mean by "toy" problems... that might be kind of true for Naive Bayes, but a Bayesian statistical model can be built in a non-naive fashion and provide useful multi-dimensional probabilities. It's really a kind of underrated technique, though I suppose it's because you can't simply `import Bayes from 'bayes'` to build a non-naive model. Bayesian models are also advantageous in that they are much easier to describe to stakeholders. Neural networks, on the other hand, are highly difficult(if not impossible in many cases) to describe how they function.

gaius 8y ago

If someone had something as basic as logistic regression or k-means clustering in a production app they are already ahead of the vast majority of ML practitioners

peatmoss 8y ago
Gods, maybe I should stop telling people that I don’t have much in the way of a machine learning background.

nkozyra 8y ago

> Naive Bayes and Graphical Models are pure statistics, but they are mostly used for toy problems. I think they just feel "toy" because they've been used with great accuracy for so long. Complex problems are rarely solved by a single approach. The harder the problem the more likely a suite will be used, and often naive bayes will be part of that in some capacity.

PeterisP 8y ago
Can you give one example where Naive Bayes has been used with great accuracy for so long? Naive Bayes is a common baseline solution because it's very simple to implement and very fast to run; however, pretty much any other ML method gives better accuracy than Naive Bayes. Sometimes the accuracy improvement is small and doesn't justify the highly increased computing cost, though, so Naive Bayes becomes the appropriate method to use, but it's not because of its accuracy.

skj 8y ago

> Machine Learning scales these approaches to high dimensionality problems, and tasks where data is abundant. Machine learning is the use of computers to make decisions (or classifications) based on data without human intervention.

freehunter 8y ago

Funny, I would consider many ML usages to be toy problems. Coming up with funny nonsensical Shakespeare plays or generating random Pantone colors or Pokémon names isn’t something you hear from Bayesian systems. That’s the domain of ML. Recommending related products is again simple statistical analysis. And I can generate a prediction based on past performance real easy with some SQL and a couple of “if” statements. Just because Bayesian isn’t over-hyped doesn’t mean it doesn’t solve real problems. Not everything can be solved with ML, unless the problem is getting more investor money.

kahnjw 8y ago
I agree with all of this, except are you really implementing bayesian models in SQL? I'm not sure that is what the author is advocating in the first place. There are tools for bayesian everything, I much prefer using those over hacking together some illegible SQL script.
- freehunter 8y ago
  No, it’s just SQL statements pulling data into the if...then in your preferred language. I’ve never done actual logic with SQL, not even sure if it’s possible.
  - JetSpiegel 8y ago
    Each DB engine has it's own incompatible imperative stored procedure language. Having worked with Oracle's version, and seen entire systems buult out of that, I dread it.
peatmoss 8y ago
There are some methods that I don’t see used outside of the machine learning community, but there appears to be a fair degree of overlap in actual methods used between classical stats and machine learning. Now, the biggest difference that I can see is that, within machine learning contexts, people are more concerned with the quality of the predictions than the interpretation of the independent variables. That’s not hard and fast, but it seems to be a common thread. And for some problems, making a good prediction is really the right thing. In other cases, understanding the mechanisms you might use to effect an outcome is more important. Both are valid uses of statistical methods, depending on the problem. I think within the realm of classical statistics, Bayesian methods’ super power is in being able to generate results that are much easier to communicate to lay people. Also Bayesian methods are nice if you want to do sensitivity analysis in a principled up front kind of way. But I could imagine using those methods in an ML context even if they aren’t the current darlings of the methodological pantheon.

dvogel 8y ago

AI and ML have different goals, which sometimes overlap. The goal of ML is to discover relationships in data that can only be (easily) observed by a machine. That usually serves as decision support data for a human decision maker or another more conventional algorithm. This is usually based on an entire data set at once. Accuracy and precision are valued over quickness and robustness. AI is about substituting human decision makers or other conventional computer decision algorithms. AI is used for situational decision making. It needs to be quick (like the conventional algorithms) while also recognizing the nuance and multi-dimemaional nature of a lot of decisions. Robustness and quickness are often valued more than precision. IOW the GP is only using ML for everything because it is a buzzword.

PeterisP 8y ago
ML is one of subdomains in the AI field, so instead of "sometimes overlap" it's technically correct to state that ML is a strict subset of AI. What you use as description for AI tempts me to use the "use it as a buzzword" angle back at you here - it is a stereotypical description of some use cases of AI approaches; but there are others - [machine] learning, knowledge representation, planning and scheduling, reasoning (both formal logic reasoning and also reasoning under uncertainty e.g. Bayesian approaches), intelligent agent representation, etc, are all parts of the AI field.
tkxxx7 8y ago
This last line seems unnecessarily rude. GP seems to understand well the two ML methods mentioned.

PeterisP 8y ago

Naive Bayes is machine learning. A very simple machine learning method, but machine learning. Also, the argument that it's pure statistics doesn't really fly - support vector machines, random forests or deep learning multilayer perceptrons are just as much statistics as probabilistic graphical models.

ravenstine 8y ago

Aren't Bayesian models essentially statistics + linear algebra? It sounds like you're characterizing ML as something beyond Bayes, which isn't really the case. I don't know what you mean by "toy" problems... that might be kind of true for Naive Bayes, but a Bayesian statistical model can be built in a non-naive fashion and provide useful multi-dimensional probabilities. It's really a kind of underrated technique, though I suppose it's because you can't simply `import Bayes from 'bayes'` to build a non-naive model. Bayesian models are also advantageous in that they are much easier to describe to stakeholders. Neural networks, on the other hand, are highly difficult(if not impossible in many cases) to describe how they function.