6 ms·

These are all fantastic points, and I strongly agree that most people don't appreciate linear models nearly enough. Another one I would add that is very import

These are all fantastic points, and I strongly agree that most people don't appreciate linear models nearly enough. Another one I would add that is very important: Human beings, especially in groups, can only reasonably make linear decisions. That is, when we are in a meeting making decisions for the direction of the company we can only say things like "we need to increase ad spend, while reducing the other costs of acquisition such as discount vouchers". If you want to find the balance between "increasing ad spend" while "decreasing other costs" that's a simple linear model. Even if you have a great non-linear model, it's not even a matter of "interpretability" so much as "actionability". You can bring the results of a regression analysis to a meeting and very quickly model different strategies with reasonable directional confidence. I struggled communicating actionable insights upward until I started to really understand regression analysis. After that it became amazingly simple to quickly crack open and understand fairly complex business processes.

nextos 2y ago
If you add a multilevel structure to shrink your (generalized) linear predictors, this framework becomes incredibly powerful. There are entire statistics textbooks devoted to multilevel linear models, you can get really far with these. Shrinking through information sharing is really important to avoid overly optimistic predictions in the case of little data.
- aquafox 2y ago
  If you like shrinkage (I do), I highly recommend the work of Matthew Stephens, e.g. ashr [1] and vash [2] for shrinkage based on an empirically derived prior. [1] https://cran.r-project.org/web/packages/ashr/index.html https://cran.r-project.org/web/packages/ashr/index.html [2] https://github.com/mengyin/vashr https://github.com/mengyin/vashr
  - nextos 2y ago
    Yes, the article linked to ashr is quite famous.
- levocardia 2y ago
  Especially when you use the mixed model (aka MLM) framework to automatically select the smoothing penalty for your splines. So in one simple and very intuitive framework, you can estimate linear and nonlinear effects, account for repeated measurements and nested data, and model binary, count, or continuous outcomes (and more), all fitting the model in one shot, yielding statistically valid confidence intervals and p-values. R's mgcv package (which does all of the above) is probably the single reason I'm still using R as my primary stats language.
  - stevesimmons 2y ago
    Is there a Python eqivalent?
    - gpderetta 2y ago
      statsmodels is the closest thing in python to R. statsmodels has mixed model support, but mgcv apparently requires more. It is well above my paygrade, but this seems relevant: https://github.com/statsmodels/statsmodels/issues/8029 https://github.com/statsmodels/statsmodels/issues/8029 (i.e. no out of the box support, you might be able to build an approximation on your own).
madrox 2y ago
I have a degree in statistics yet I've never thought about the relationship between linear models and business decisions in this way. You're absolutely right. This is the best comment I've read all month.
- highfrequency 2y ago
  I don't follow - could you explain this with a couple of examples? What would a business proposal look like that is analogous to a nonlinear model vs. one that is analogous to a linear model?
  - resonious 2y ago
    I'm also curious about what a non-actionable non-linear suggestion would look like.
    - 317070 2y ago
      How I understand the comment: a non-linear suggestion is that the budget for X should be 300k. The (supposedly linear) alternative is that the budget for X should increase. What I think is the important part, is that it is better to ask decision makers for decisions on setting a continuous parameter, than to make binary yes/no or go/no-go decisions. When it's a decision by committee, I can see why that is.
  - wheelinsupial 2y ago
    I’m neither of the previous posters, so I may be off… For simplicity, I’m going to assume each variable in the model is independent of every other variable. We can interpret the coefficients in linear models. This relationship holds for the model for the range of values it is based on. This relationship is the same for the whole range of the model. (We can’t extrapolate outside of what’s been modeled.) y = c1x1 + c2x2 +…+ cnxn (excuse the poor formatting) The sign tells you the direction (+ means it will increase the value of y, - means it will decrease the value of y), the value of the coefficient tells you how much the y will change for a given 1-unit change in the x value. Since this is linear, you get the same change to the output for the relevant increases no matter your starting point. So, the regression model would say x1, x3, and x5 have positive coefficients and variables x2, x4 have negative coefficients. If you want y to increase, either start doing more of x1, x3, x5 or do less of x2, x4. Depending on what these are and your limited investment budget, for example, you may pick doing x3 if that is the largest positive coefficient. Again, since this is linear, you can keep on putting resources into the largest coefficient and get the same increase up until your model is no longer valid. For non-linear models, you can still interpret the coefficients, but the interpretation depends on your starting conditions and where you are on the graph. There may be asymptotes in your non-linear model, so there is a point of diminishing returns where if you keep putting resources into a variable with a positive coefficient, this will not keep getting you commensurate results. Sorry I don’t have any actual examples here and I don’t have time to go digging through my old textbooks to look for any.
addaon 2y ago
> Human beings, especially in groups, can only reasonably make linear decisions. There are absolutely decisions that need to get made, and do get made, that are not linear. Step functions are a great example. "We need to decide if we are going to accept this acquisition offer" is an example of a decision with step function utility. You can try to "linearize" it and then apply a threshold -- "let's agree on a model for the value at which we would accept an acquisition offer" -- but in many ways that obscures that the utility function can be arbitrarily non-linear.
- mturmon 2y ago
  A single decision could still be easily modeled by a 0/1 variable (as an input) and a real variable (as an output, like revenue for example). That 0/1 input variable could also have arbitrary interactions with other variables, which would also amount to “step function “ input effects. See for example the autism/age setup down thread.
  - eru 2y ago
    Discrete linear optimisation is infinitely more complicated than continuous linear optimisation. The former is NP complete, the latter is in P.
    - naasking 2y ago
      Which seems almost ironic, because continuous linear optimization almost certainly doesn't exist really because real numbers can only be approximated, and so we're always doing discrete linear optimization at some level.
      - eru 2y ago
        Who cares about real numbers in this context? If all the numbers that appear in your constraints are rational (p/q with finite p and q), then any solution is also a rational number (with finite nominator and finite denominator). (Well, any finite solution. Your solution could also be unbounded, then you might have infinities in there.) A computer can represent finite rational numbers just fine. See eg https://docs.python.org/3/library/fractions.html https://docs.python.org/3/library/fractions.html or https://hackage.haskell.org/package/base-4.20.0.1/docs/Data-Ratio.html https://hackage.haskell.org/package/base-4.20.0.1/docs/Data-... for some libraries. Though in most cases, people just use floating point numbers in practice, but that's of no philosophical concern.
highfrequency 2y ago
I don't follow - could you explain this with a couple of examples? What would a business proposal look like that is analogous to a nonlinear model vs. one that is analogous to a linear model?
antwerp1 2y ago
Quadratics might be more useful for optimizing (min/max problems)..
eru 2y ago
> Another one I would add that is very important: Human beings, especially in groups, can only reasonably make linear decisions. No, that's not true. Human groups are very able to make discrete decisions. Actually, often they tend to go for discrete decisions, when something continuous (and perhaps linear) would be a lot better. (Just to be clear: if you force your linear models to make discrete predictions, they are no longer linear in any sense of the word. That's why linear optimisation is a problem that can be solved in polynomial time, and integer linear optimisation is NP complete. Even convex optimisation, which is no longer linear but still continuous, can be solved in roughly polynomial time.) Often people demand more decisive decisions, of 'yes'/'no' or concrete action, not shades of grey and fiddling at the margins. Getting people to even appreciate linear models is already a step forward. Like it or not, your business strategy meetings are already a step ahead of what most people would naturally be inclined to.
- wiz21c 2y ago
  > Often people demand more decisive decisions, of 'yes'/'no' or concrete action, not shades of grey and fiddling at the margins. And working with these people is so painful.
  - eru 2y ago
    Yes. I also found that in many cases being able to turn problems that require discrete decisions into problems that admit continuous decisions, eg by re-arranging how the business works etc, can unlock a lot of business value. In my concrete cases I mostly saw that in the direct sense of being able to deploy more mathematics and operations research, eg for netting out (partially) offsetting financial instruments for a bank. But by introspection you can come up with more example. Eg that's a common selling point for running your servers on AWS instead of building your own hardware.
  - harperlee 2y ago
    I often make fun of McKinsey- style four quadrants when overused, but they really boil down to something that makes a lot of sense in communicating a problem space: a) carefully choose the two most important dimensions of concern (as Alan Kay said: the correct point of view is worth 80 Iq points) b) make them binary: are we happy here or do we need to change? In a way similar to the pareto ratio, you keep a surprising amount of value in something “so simple it cant be possibly so useful”.
jvans 2y ago
> Human beings, especially in groups, can only reasonably make linear decisions. This seems to be getting a lot of attention. I couldn't agree more, we assume linearity all the time because reasoning non-linearly is exceptionally difficult. Yes we can do it sometimes, but it is not the default. Reasoning linearly has its flaws, and we should recognize we are making an imperfect decision, but it is still extremely useful.

nextos 2y ago

If you add a multilevel structure to shrink your (generalized) linear predictors, this framework becomes incredibly powerful. There are entire statistics textbooks devoted to multilevel linear models, you can get really far with these. Shrinking through information sharing is really important to avoid overly optimistic predictions in the case of little data.

aquafox 2y ago
If you like shrinkage (I do), I highly recommend the work of Matthew Stephens, e.g. ashr [1] and vash [2] for shrinkage based on an empirically derived prior. [1] https://cran.r-project.org/web/packages/ashr/index.html https://cran.r-project.org/web/packages/ashr/index.html [2] https://github.com/mengyin/vashr https://github.com/mengyin/vashr
- nextos 2y ago
  Yes, the article linked to ashr is quite famous.
levocardia 2y ago
Especially when you use the mixed model (aka MLM) framework to automatically select the smoothing penalty for your splines. So in one simple and very intuitive framework, you can estimate linear and nonlinear effects, account for repeated measurements and nested data, and model binary, count, or continuous outcomes (and more), all fitting the model in one shot, yielding statistically valid confidence intervals and p-values. R's mgcv package (which does all of the above) is probably the single reason I'm still using R as my primary stats language.
- stevesimmons 2y ago
  Is there a Python eqivalent?
  - gpderetta 2y ago
    statsmodels is the closest thing in python to R. statsmodels has mixed model support, but mgcv apparently requires more. It is well above my paygrade, but this seems relevant: https://github.com/statsmodels/statsmodels/issues/8029 https://github.com/statsmodels/statsmodels/issues/8029 (i.e. no out of the box support, you might be able to build an approximation on your own).

madrox 2y ago

I have a degree in statistics yet I've never thought about the relationship between linear models and business decisions in this way. You're absolutely right. This is the best comment I've read all month.

highfrequency 2y ago
I don't follow - could you explain this with a couple of examples? What would a business proposal look like that is analogous to a nonlinear model vs. one that is analogous to a linear model?
- resonious 2y ago
  I'm also curious about what a non-actionable non-linear suggestion would look like.
  - 317070 2y ago
    How I understand the comment: a non-linear suggestion is that the budget for X should be 300k. The (supposedly linear) alternative is that the budget for X should increase. What I think is the important part, is that it is better to ask decision makers for decisions on setting a continuous parameter, than to make binary yes/no or go/no-go decisions. When it's a decision by committee, I can see why that is.
- wheelinsupial 2y ago
  I’m neither of the previous posters, so I may be off… For simplicity, I’m going to assume each variable in the model is independent of every other variable. We can interpret the coefficients in linear models. This relationship holds for the model for the range of values it is based on. This relationship is the same for the whole range of the model. (We can’t extrapolate outside of what’s been modeled.) y = c1x1 + c2x2 +…+ cnxn (excuse the poor formatting) The sign tells you the direction (+ means it will increase the value of y, - means it will decrease the value of y), the value of the coefficient tells you how much the y will change for a given 1-unit change in the x value. Since this is linear, you get the same change to the output for the relevant increases no matter your starting point. So, the regression model would say x1, x3, and x5 have positive coefficients and variables x2, x4 have negative coefficients. If you want y to increase, either start doing more of x1, x3, x5 or do less of x2, x4. Depending on what these are and your limited investment budget, for example, you may pick doing x3 if that is the largest positive coefficient. Again, since this is linear, you can keep on putting resources into the largest coefficient and get the same increase up until your model is no longer valid. For non-linear models, you can still interpret the coefficients, but the interpretation depends on your starting conditions and where you are on the graph. There may be asymptotes in your non-linear model, so there is a point of diminishing returns where if you keep putting resources into a variable with a positive coefficient, this will not keep getting you commensurate results. Sorry I don’t have any actual examples here and I don’t have time to go digging through my old textbooks to look for any.

addaon 2y ago

> Human beings, especially in groups, can only reasonably make linear decisions. There are absolutely decisions that need to get made, and do get made, that are not linear. Step functions are a great example. "We need to decide if we are going to accept this acquisition offer" is an example of a decision with step function utility. You can try to "linearize" it and then apply a threshold -- "let's agree on a model for the value at which we would accept an acquisition offer" -- but in many ways that obscures that the utility function can be arbitrarily non-linear.

mturmon 2y ago
A single decision could still be easily modeled by a 0/1 variable (as an input) and a real variable (as an output, like revenue for example). That 0/1 input variable could also have arbitrary interactions with other variables, which would also amount to “step function “ input effects. See for example the autism/age setup down thread.
- eru 2y ago
  Discrete linear optimisation is infinitely more complicated than continuous linear optimisation. The former is NP complete, the latter is in P.
  - naasking 2y ago
    Which seems almost ironic, because continuous linear optimization almost certainly doesn't exist really because real numbers can only be approximated, and so we're always doing discrete linear optimization at some level.
    - eru 2y ago
      Who cares about real numbers in this context? If all the numbers that appear in your constraints are rational (p/q with finite p and q), then any solution is also a rational number (with finite nominator and finite denominator). (Well, any finite solution. Your solution could also be unbounded, then you might have infinities in there.) A computer can represent finite rational numbers just fine. See eg https://docs.python.org/3/library/fractions.html https://docs.python.org/3/library/fractions.html or https://hackage.haskell.org/package/base-4.20.0.1/docs/Data-Ratio.html https://hackage.haskell.org/package/base-4.20.0.1/docs/Data-... for some libraries. Though in most cases, people just use floating point numbers in practice, but that's of no philosophical concern.

highfrequency 2y ago

I don't follow - could you explain this with a couple of examples? What would a business proposal look like that is analogous to a nonlinear model vs. one that is analogous to a linear model?

antwerp1 2y ago

Quadratics might be more useful for optimizing (min/max problems)..

eru 2y ago

> Another one I would add that is very important: Human beings, especially in groups, can only reasonably make linear decisions. No, that's not true. Human groups are very able to make discrete decisions. Actually, often they tend to go for discrete decisions, when something continuous (and perhaps linear) would be a lot better. (Just to be clear: if you force your linear models to make discrete predictions, they are no longer linear in any sense of the word. That's why linear optimisation is a problem that can be solved in polynomial time, and integer linear optimisation is NP complete. Even convex optimisation, which is no longer linear but still continuous, can be solved in roughly polynomial time.) Often people demand more decisive decisions, of 'yes'/'no' or concrete action, not shades of grey and fiddling at the margins. Getting people to even appreciate linear models is already a step forward. Like it or not, your business strategy meetings are already a step ahead of what most people would naturally be inclined to.

wiz21c 2y ago
> Often people demand more decisive decisions, of 'yes'/'no' or concrete action, not shades of grey and fiddling at the margins. And working with these people is so painful.
- eru 2y ago
  Yes. I also found that in many cases being able to turn problems that require discrete decisions into problems that admit continuous decisions, eg by re-arranging how the business works etc, can unlock a lot of business value. In my concrete cases I mostly saw that in the direct sense of being able to deploy more mathematics and operations research, eg for netting out (partially) offsetting financial instruments for a bank. But by introspection you can come up with more example. Eg that's a common selling point for running your servers on AWS instead of building your own hardware.
- harperlee 2y ago
  I often make fun of McKinsey- style four quadrants when overused, but they really boil down to something that makes a lot of sense in communicating a problem space: a) carefully choose the two most important dimensions of concern (as Alan Kay said: the correct point of view is worth 80 Iq points) b) make them binary: are we happy here or do we need to change? In a way similar to the pareto ratio, you keep a surprising amount of value in something “so simple it cant be possibly so useful”.

jvans 2y ago

> Human beings, especially in groups, can only reasonably make linear decisions. This seems to be getting a lot of attention. I couldn't agree more, we assume linearity all the time because reasoning non-linearly is exceptionally difficult. Yes we can do it sometimes, but it is not the default. Reasoning linearly has its flaws, and we should recognize we are making an imperfect decision, but it is still extremely useful.