5 ms·

Curious. OP gave a few examples for Ecommerce where SQL will do fine. Can you give a few where ML will do something otherwise impossible or harder with SQL?

smrtinsert 8y ago
Give me all the time frames when there was statistically significant bump in X.
throwawayjava 8y ago
controls and sensing.
firasd 8y ago
Product Recommendations. Trending Items (Top items being sold this week as opposed to last week, while filtering out items that are generally popular.) Much easier with Elasticsearch than SQL https://www.elastic.co/blog/significant-terms-aggregation https://www.elastic.co/blog/significant-terms-aggregation
- collyw 8y ago
  That's not really ML.
  - jon_richards 8y ago
    If I were ever to make ML bingo, "That's not really ML" would definitely be on it.
    - collyw 8y ago
      I see, so just how everything became "big data" a few years ago, now everything is ML. "Managing big data with MySQL" - the syllabus mentions nothing of clustering or sharing. Ten years ago that was just "using a database". I am getting to old for the faddish nature of this industry. https://www.coursera.org/learn/analytics-mysql https://www.coursera.org/learn/analytics-mysql
  - geebee 8y ago
    As a classic example, how about music or movie recommendations? You have a giant set of attributes about songs and a binary classification (thumbs up/thumbs down) from millions of listeners. You can use this this to train a random forest or neural net (or other algorithms) to make song recommendations to someone based on that person's pattern of thumbs up/down. I'd guess most recommendation systems do something along these lines, though I don't really know. I'm not sure how I'd do this with SQL. That certainly doesn't mean you couldn't take a SQL approach to the problem of "how do I make recommendations based on this training set and this particular individual's attributes" and do something creative and useful. But I do think you can, in this situation, get something out of ML that would be difficult (ok, I'll say impossible) to get with a pure SQL approach. To be clear, I love SQL. I still get a little flash of anger when I remember my arguments with "architects" who announced to a room full of non-technical people that I was pretty much a dinosaur unwilling to move away from an obsolete technology when they advocated no-sql approaches to data that was deeply relational. And I'm sure that there are plenty of people who think "need answer from data, must use ML" and end up somehow trying to train a neural net to perform a WHERE clause. Having seen the hype cycle several times, I am absolutely certain this has occurred in boardrooms/open office cubicle farms all across the you ess aye and beyond. But keep in mind, even though reports of SQL's death were remarkably exaggerated, "nosql" approaches often do make the most sense. Plenty of good ideas and technologies came from nosql, and many of the people who created them and advocated them continued to use (and argue for) SQL and relational databases depending on the problem. A graph database is a vastly superior approach to some types of network problems than standard SQL, and not helpful for others. When the hype cycle around ML dies down (and some people start writing articles that ML is "dead" or whatever), ML will continue to be used effectively in all kinds of places, as it is now.
    - collyw 8y ago
      I didn't say that it was SQL, I said that its not ML. Just aggregation / scoring as far as I can tell.
      - geebee 8y ago
        It was clear that you were only saying that it wasn't ML. I looked up a bit on elasticsearch's text classification, and it's interesting. There's a overview of "traditional" ML (similar to the song review classification I mentioned) and how elastic search differs. https://www.elastic.co/blog/text-classification-made-easy-with-elasticsearch https://www.elastic.co/blog/text-classification-made-easy-wi... "The MLT query is a very important query for text mining. How does it work? It can process arbitrary text, extract the top n keywords relative to the actual "model" and run a boolean match query with those keywords. This query is often used to gather similar documents." Yeah, looks like aggregation and scoring. It'd be interesting to see if/where this outperformed various ML algorithms (accuracy as well as performance).
AndyNemmity 8y ago
Pricing.
edraferi 8y ago
Anytime you need to make sense of unstructured data like images or text and drive an automated business process off it.
kahnjw 8y ago
Forecasting and prediction. Specific example: given a high quality dataset, forecasting sales is a good use case for machine learning. Of course, you could do this with SQL as well, but again, given a good dataset and a complex domain like sales, some relatively simple models will be hard to beat with a 10 line SQL query.
- zv 8y ago
  Sorry, I disagree with you. Let's take forecasting. This is a _very_ hard topic and being applied correctly using actuarial methods it will beat ML. To give you some example, you cannot take sales data for IPhone X and predict its future sales. Reason is simple - market will change, IPhone 11 will come out cannibalizing IPhone X sales. So, you need to have 2 more variables into play. One is the lifetime of a product (approximated) and second is, you need to "put in one basket" several products. Having this, you can predict your sales for some specific product group, no magic ML needed. However this requires you to define "similar products". This is hard because currently it is done semi-automatically using human labor. In fact ML is actually the best one to automagically categorize products into groups, however I have not seen any implementations. To summarize, ML do has value, however not where everybody is pointing out.
threeseed 8y ago
Feature extraction is the most obvious. I give you a problem e.g. "tell me what factors are influencing a customer's likelihood to leave a bad review". And with ML you can actually produce a list of sorted factors with weighted percentages. You simply can't do that with SQL. Also remember that SQL is for databases so you often can't do anything algorithmic e.g. K-means clustering, linear regression, random forest, decision trees etc. Where as ML encompasses non-databases as well e.g. from a Kafka stream of numbers predict the next number.
__m 8y ago
staying in business
raarts 8y ago
Getting funded
kthejoker2 8y ago
Well as I said it takes extremely intricate and not all that well performing SQL to perform signal processing and latent trend analysis, as it's not really designed with matrix multiplication or iterative transformation. So anything with noise, seasonality, ARIMA etc shouldn't be done in SQL, it should be done in a proper ML tool, because 99% of the time your developer will miss something and you'll improperly assign value to something that is in fact no pattern at all. You can't recreate the Netflix Prize in SQL, you can't really do NLP, you can't do pretty much any clustering or unsupervised learning. Again, "can't" meaning "can't do it easily or in a sustainable manner (i.e. shouldn't.)" And if we add in AI, I don't need to explain that SQL has no abilities to generate new content - it can't write a new song or headline, understand context and provide feedback, or drive a car. Now what OP is arguing is that you can get a lot of lift with SQL and basic exploratory data analysis and I totally agree. But ML can take you further out into the speculative realm and can work on a much less explicit and prescriptive model than SQL. Basically in SQL if you don't write it (whatever it is, a pattern, an anomaly, a correlation, a context) in your query you will never learn a single thing about it.

smrtinsert 8y ago

Give me all the time frames when there was statistically significant bump in X.

throwawayjava 8y ago

controls and sensing.

firasd 8y ago

Product Recommendations. Trending Items (Top items being sold this week as opposed to last week, while filtering out items that are generally popular.) Much easier with Elasticsearch than SQL https://www.elastic.co/blog/significant-terms-aggregation https://www.elastic.co/blog/significant-terms-aggregation

collyw 8y ago
That's not really ML.
- jon_richards 8y ago
  If I were ever to make ML bingo, "That's not really ML" would definitely be on it.
  - collyw 8y ago
    I see, so just how everything became "big data" a few years ago, now everything is ML. "Managing big data with MySQL" - the syllabus mentions nothing of clustering or sharing. Ten years ago that was just "using a database". I am getting to old for the faddish nature of this industry. https://www.coursera.org/learn/analytics-mysql https://www.coursera.org/learn/analytics-mysql
- geebee 8y ago
  As a classic example, how about music or movie recommendations? You have a giant set of attributes about songs and a binary classification (thumbs up/thumbs down) from millions of listeners. You can use this this to train a random forest or neural net (or other algorithms) to make song recommendations to someone based on that person's pattern of thumbs up/down. I'd guess most recommendation systems do something along these lines, though I don't really know. I'm not sure how I'd do this with SQL. That certainly doesn't mean you couldn't take a SQL approach to the problem of "how do I make recommendations based on this training set and this particular individual's attributes" and do something creative and useful. But I do think you can, in this situation, get something out of ML that would be difficult (ok, I'll say impossible) to get with a pure SQL approach. To be clear, I love SQL. I still get a little flash of anger when I remember my arguments with "architects" who announced to a room full of non-technical people that I was pretty much a dinosaur unwilling to move away from an obsolete technology when they advocated no-sql approaches to data that was deeply relational. And I'm sure that there are plenty of people who think "need answer from data, must use ML" and end up somehow trying to train a neural net to perform a WHERE clause. Having seen the hype cycle several times, I am absolutely certain this has occurred in boardrooms/open office cubicle farms all across the you ess aye and beyond. But keep in mind, even though reports of SQL's death were remarkably exaggerated, "nosql" approaches often do make the most sense. Plenty of good ideas and technologies came from nosql, and many of the people who created them and advocated them continued to use (and argue for) SQL and relational databases depending on the problem. A graph database is a vastly superior approach to some types of network problems than standard SQL, and not helpful for others. When the hype cycle around ML dies down (and some people start writing articles that ML is "dead" or whatever), ML will continue to be used effectively in all kinds of places, as it is now.
  - collyw 8y ago
    I didn't say that it was SQL, I said that its not ML. Just aggregation / scoring as far as I can tell.
    - geebee 8y ago
      It was clear that you were only saying that it wasn't ML. I looked up a bit on elasticsearch's text classification, and it's interesting. There's a overview of "traditional" ML (similar to the song review classification I mentioned) and how elastic search differs. https://www.elastic.co/blog/text-classification-made-easy-with-elasticsearch https://www.elastic.co/blog/text-classification-made-easy-wi... "The MLT query is a very important query for text mining. How does it work? It can process arbitrary text, extract the top n keywords relative to the actual "model" and run a boolean match query with those keywords. This query is often used to gather similar documents." Yeah, looks like aggregation and scoring. It'd be interesting to see if/where this outperformed various ML algorithms (accuracy as well as performance).

AndyNemmity 8y ago

Pricing.

edraferi 8y ago

Anytime you need to make sense of unstructured data like images or text and drive an automated business process off it.

kahnjw 8y ago

Forecasting and prediction. Specific example: given a high quality dataset, forecasting sales is a good use case for machine learning. Of course, you could do this with SQL as well, but again, given a good dataset and a complex domain like sales, some relatively simple models will be hard to beat with a 10 line SQL query.

zv 8y ago
Sorry, I disagree with you. Let's take forecasting. This is a _very_ hard topic and being applied correctly using actuarial methods it will beat ML. To give you some example, you cannot take sales data for IPhone X and predict its future sales. Reason is simple - market will change, IPhone 11 will come out cannibalizing IPhone X sales. So, you need to have 2 more variables into play. One is the lifetime of a product (approximated) and second is, you need to "put in one basket" several products. Having this, you can predict your sales for some specific product group, no magic ML needed. However this requires you to define "similar products". This is hard because currently it is done semi-automatically using human labor. In fact ML is actually the best one to automagically categorize products into groups, however I have not seen any implementations. To summarize, ML do has value, however not where everybody is pointing out.

threeseed 8y ago

Feature extraction is the most obvious. I give you a problem e.g. "tell me what factors are influencing a customer's likelihood to leave a bad review". And with ML you can actually produce a list of sorted factors with weighted percentages. You simply can't do that with SQL. Also remember that SQL is for databases so you often can't do anything algorithmic e.g. K-means clustering, linear regression, random forest, decision trees etc. Where as ML encompasses non-databases as well e.g. from a Kafka stream of numbers predict the next number.

__m 8y ago

staying in business

raarts 8y ago

Getting funded

kthejoker2 8y ago

Well as I said it takes extremely intricate and not all that well performing SQL to perform signal processing and latent trend analysis, as it's not really designed with matrix multiplication or iterative transformation. So anything with noise, seasonality, ARIMA etc shouldn't be done in SQL, it should be done in a proper ML tool, because 99% of the time your developer will miss something and you'll improperly assign value to something that is in fact no pattern at all. You can't recreate the Netflix Prize in SQL, you can't really do NLP, you can't do pretty much any clustering or unsupervised learning. Again, "can't" meaning "can't do it easily or in a sustainable manner (i.e. shouldn't.)" And if we add in AI, I don't need to explain that SQL has no abilities to generate new content - it can't write a new song or headline, understand context and provide feedback, or drive a car. Now what OP is arguing is that you can get a lot of lift with SQL and basic exploratory data analysis and I totally agree. But ML can take you further out into the speculative realm and can work on a much less explicit and prescriptive model than SQL. Basically in SQL if you don't write it (whatever it is, a pattern, an anomaly, a correlation, a context) in your query you will never learn a single thing about it.