3 ms·

Is There Such a Thing As Predictive Analytics?

10 points by skempe 14y ago

rywa 14y ago
Some systems are more complex than others. Others exhibit more apparent regularity. This also comes into play depending on what scale you're working at. Although it's difficult to predict the weather that doesn't mean it's equally to difficult to predict something like the load on an electric grid, which depends on the weather. Electricity companies have been doing this for decades.
- crusso 14y ago
  It's also important to understand the perceived success of predictive analytics in the context of the problem being solved. For example in predictive analytics used to determine product distribution at major retailers, the goal isn't to be perfect. The goal is to be a little bit better than what humans (or their previous methods) could do. If you go to Target management and prove that your predictive analytics is 3.3% better at predicting how many blenders they'll need in each of their stores, but that 3.3% netted them 5 million dollars, they'll sit up and listen.
crusso 14y ago
The author is right in one sense: there's a lot of black magic around predictive analytics, machine learning, modeling, etc. However, some of these techniques yield a great deal of paydirt. Look at the spam, or rather the lack thereof, in your inbox. Thank the enormous success of predictive analytics for eliminating a great deal of it. As he mentioned, weather forecasting is actually pretty good, all things considered. Kvetching about unexpected rainy days demonstrates a misunderstanding of how we should evaluate the success of weather forecasting techniques. As useful as these techniques are, there is a great deal of ignorance in where they are useful and where they aren't. People selling things and those with various political agendas use that ignorance to foist "modeling" on an unsuspecting public. No complex modeling used for predictive analytics is useful without a great deal of validation. You come up with a model and set of techniques, you feed in historical data, you compare the outputs of your predictions with subsequent historical data, you adjust your techniques to produce more accurate predictions, wash, rinse, repeat. There's no real getting around the need to perform this cycle ad nauseum if you want results that aren't just noise.
endersshadow 14y ago
I've done a lot of this thing called "predictive analytics," and I've found that what works best is machine-aided prognostication. It highlights some things that folks aren't aware of, and points to some seasonality that may not be particularly intuitive. I've found it particularly useful in predicting inventory and packaging distributions. For example, if you have a product like soda that you can package a myriad of different ways, predictive analytics is a great way to help determine the ratio and amount you need to put into each type of package. I've seen it used with great success, reducing the amount of excess inventory and increasing sales (because stock-outs don't happen). It's a tool, like any other. It's not a silver bullet that will tell you exactly what the future will hold, but it can give you a good guideline on what to expect, and from there, you can act upon that information.
dgreensp 14y ago
Philosophically speaking, he is defining the term "prediction" right out of existence! It's not that "real" predictions are "right" in some sense, even probabilistically. There is only ever such thing as one take on the future based on certain data from a certain perspective.
- msellout 14y ago
  Right. He's critiquing the term "predictive model" by saying we should use the term "hypothetical model" instead. Is he suggesting it's impossible to test a hypothesis? I thought Kant and Hume had this argument long ago.
msellout 14y ago
Taleb wrote better. Yes, prediction is hard. If you're interested in these topics, do some reading on over-fitting, stationarity, and nonlinear dynamics.

10 points by skempe 14y ago

rywa 14y ago

Some systems are more complex than others. Others exhibit more apparent regularity. This also comes into play depending on what scale you're working at. Although it's difficult to predict the weather that doesn't mean it's equally to difficult to predict something like the load on an electric grid, which depends on the weather. Electricity companies have been doing this for decades.

crusso 14y ago
It's also important to understand the perceived success of predictive analytics in the context of the problem being solved. For example in predictive analytics used to determine product distribution at major retailers, the goal isn't to be perfect. The goal is to be a little bit better than what humans (or their previous methods) could do. If you go to Target management and prove that your predictive analytics is 3.3% better at predicting how many blenders they'll need in each of their stores, but that 3.3% netted them 5 million dollars, they'll sit up and listen.

crusso 14y ago

The author is right in one sense: there's a lot of black magic around predictive analytics, machine learning, modeling, etc. However, some of these techniques yield a great deal of paydirt. Look at the spam, or rather the lack thereof, in your inbox. Thank the enormous success of predictive analytics for eliminating a great deal of it. As he mentioned, weather forecasting is actually pretty good, all things considered. Kvetching about unexpected rainy days demonstrates a misunderstanding of how we should evaluate the success of weather forecasting techniques. As useful as these techniques are, there is a great deal of ignorance in where they are useful and where they aren't. People selling things and those with various political agendas use that ignorance to foist "modeling" on an unsuspecting public. No complex modeling used for predictive analytics is useful without a great deal of validation. You come up with a model and set of techniques, you feed in historical data, you compare the outputs of your predictions with subsequent historical data, you adjust your techniques to produce more accurate predictions, wash, rinse, repeat. There's no real getting around the need to perform this cycle ad nauseum if you want results that aren't just noise.

endersshadow 14y ago

I've done a lot of this thing called "predictive analytics," and I've found that what works best is machine-aided prognostication. It highlights some things that folks aren't aware of, and points to some seasonality that may not be particularly intuitive. I've found it particularly useful in predicting inventory and packaging distributions. For example, if you have a product like soda that you can package a myriad of different ways, predictive analytics is a great way to help determine the ratio and amount you need to put into each type of package. I've seen it used with great success, reducing the amount of excess inventory and increasing sales (because stock-outs don't happen). It's a tool, like any other. It's not a silver bullet that will tell you exactly what the future will hold, but it can give you a good guideline on what to expect, and from there, you can act upon that information.

dgreensp 14y ago

Philosophically speaking, he is defining the term "prediction" right out of existence! It's not that "real" predictions are "right" in some sense, even probabilistically. There is only ever such thing as one take on the future based on certain data from a certain perspective.

msellout 14y ago
Right. He's critiquing the term "predictive model" by saying we should use the term "hypothetical model" instead. Is he suggesting it's impossible to test a hypothesis? I thought Kant and Hume had this argument long ago.

msellout 14y ago

Taleb wrote better. Yes, prediction is hard. If you're interested in these topics, do some reading on over-fitting, stationarity, and nonlinear dynamics.