5 ms·

I always feel this is too close to stochastic versus random. There is a lot of text that pushes an idea that regression is used to understand how well a model

by taeric 3y ago

I always feel this is too close to stochastic versus random. There is a lot of text that pushes an idea that regression is used to understand how well a model fits relationships between variables. But, I start to have major doubts when people push the idea that regression models are not also predictive models.

VHRanger 3y ago
I mean, regression models (GLMs in general) are interpretable. If you are using them to extrapolate (eg. Prediction) that should help you gauge how resilient you expect the model to be in prediction. Obviously, for ARIMA the AR and MA parameters aren't very informative. I use SARIMAX a decent amount, nonetheless.
- mr_toad 3y ago
  Taken individually the AR and MA terms are an average change and an average difference respectively. When you combine different AR and MA terms it does become less explicable.
Fomite 3y ago
One of the major time series prediction models my field uses is indeed a regression model.
rdli 3y ago
(Author here) One of the things that confused me is that regression models can be predictive, just like time series forecasting — they just do so in a different way. I tried to make this clear in the article (or maybe I’m not understanding what you’re saying). In a regression model, you’re predicting target variables from feature variables. In a time series, you’re predicting the same variable from its past behavior. This is a subtle but crucial difference. (And then you can do time series with covariates, which combines the two.)
- civilized 3y ago
  Many of the most important time series prediction models are called "autoregressive", meaning they are regression models predicting the target from (prior values of) itself. This suggests that statisticians don't really share the view that these domains are distinct, or that regression models should only predict with different variables from the target.
  - hackerlight 3y ago
    Right, AR(n) is a regression model, as are models which take only exogenous variables. My question is this. According to definitions, can the latter (f(X_t) = y_t) be a time series model if each row of data is a time step? It doesn't have any autoregressive terms in X, so I don't know if it categorically is a time-series model. Not that this question even matters, it's purely a taxonomy/terminology question.
    - nerdponx 3y ago
      Yes, it is. A time series model is any model where the data varies over time; that is, a time series model is any model of time series data. And timeseries data is broadly anything where the data for a single thing/entity varies over time. There are no strict definitions here, just common conventions.
      - hackerlight 3y ago
        Okay. And we can also say that there's some time series models that aren't regression models, right? For example, Kalman Filter is a "model" of a time series but isn't a regression.
        nerdponx 3y ago
        Correct. Although the term "regression" is a misnomer anyway, and often when people say "regression" they mean "linear model". And by "linear model", we mean specifically a model in which outputs/predictions are some fixed linear combination of the input. It is however possible to interpret the Kalman filter as a kind of dynamic regression model. Check out here if you want a good math workout on that topic: https://stats.stackexchange.com/q/330696 https://stats.stackexchange.com/q/330696 (Another somewhat distinct meaning of the term "regression" is any model with a "continuous" outcome variable. This is usually in contrast to "classification", which is any model that has a "categorical" or discrete outcome variable.)

by taeric 3y ago

VHRanger 3y ago

I mean, regression models (GLMs in general) are interpretable. If you are using them to extrapolate (eg. Prediction) that should help you gauge how resilient you expect the model to be in prediction. Obviously, for ARIMA the AR and MA parameters aren't very informative. I use SARIMAX a decent amount, nonetheless.

mr_toad 3y ago
Taken individually the AR and MA terms are an average change and an average difference respectively. When you combine different AR and MA terms it does become less explicable.

Fomite 3y ago

One of the major time series prediction models my field uses is indeed a regression model.

rdli 3y ago

(Author here) One of the things that confused me is that regression models can be predictive, just like time series forecasting — they just do so in a different way. I tried to make this clear in the article (or maybe I’m not understanding what you’re saying). In a regression model, you’re predicting target variables from feature variables. In a time series, you’re predicting the same variable from its past behavior. This is a subtle but crucial difference. (And then you can do time series with covariates, which combines the two.)

civilized 3y ago
Many of the most important time series prediction models are called "autoregressive", meaning they are regression models predicting the target from (prior values of) itself. This suggests that statisticians don't really share the view that these domains are distinct, or that regression models should only predict with different variables from the target.
- hackerlight 3y ago
  Right, AR(n) is a regression model, as are models which take only exogenous variables. My question is this. According to definitions, can the latter (f(X_t) = y_t) be a time series model if each row of data is a time step? It doesn't have any autoregressive terms in X, so I don't know if it categorically is a time-series model. Not that this question even matters, it's purely a taxonomy/terminology question.
  - nerdponx 3y ago
    Yes, it is. A time series model is any model where the data varies over time; that is, a time series model is any model of time series data. And timeseries data is broadly anything where the data for a single thing/entity varies over time. There are no strict definitions here, just common conventions.
    - hackerlight 3y ago
      Okay. And we can also say that there's some time series models that aren't regression models, right? For example, Kalman Filter is a "model" of a time series but isn't a regression.
      - nerdponx 3y ago
        Correct. Although the term "regression" is a misnomer anyway, and often when people say "regression" they mean "linear model". And by "linear model", we mean specifically a model in which outputs/predictions are some fixed linear combination of the input. It is however possible to interpret the Kalman filter as a kind of dynamic regression model. Check out here if you want a good math workout on that topic: https://stats.stackexchange.com/q/330696 https://stats.stackexchange.com/q/330696 (Another somewhat distinct meaning of the term "regression" is any model with a "continuous" outcome variable. This is usually in contrast to "classification", which is any model that has a "categorical" or discrete outcome variable.)