4 ms·

Right, AR(n) is a regression model, as are models which take only exogenous variables. My question is this. According to definitions, can the latter (f(X_t) =

Right, AR(n) is a regression model, as are models which take only exogenous variables. My question is this. According to definitions, can the latter (f(X_t) = y_t) be a time series model if each row of data is a time step? It doesn't have any autoregressive terms in X, so I don't know if it categorically is a time-series model. Not that this question even matters, it's purely a taxonomy/terminology question.

nerdponx 3y ago
Yes, it is. A time series model is any model where the data varies over time; that is, a time series model is any model of time series data. And timeseries data is broadly anything where the data for a single thing/entity varies over time. There are no strict definitions here, just common conventions.
- hackerlight 3y ago
  Okay. And we can also say that there's some time series models that aren't regression models, right? For example, Kalman Filter is a "model" of a time series but isn't a regression.
  - nerdponx 3y ago
    Correct. Although the term "regression" is a misnomer anyway, and often when people say "regression" they mean "linear model". And by "linear model", we mean specifically a model in which outputs/predictions are some fixed linear combination of the input. It is however possible to interpret the Kalman filter as a kind of dynamic regression model. Check out here if you want a good math workout on that topic: https://stats.stackexchange.com/q/330696 https://stats.stackexchange.com/q/330696 (Another somewhat distinct meaning of the term "regression" is any model with a "continuous" outcome variable. This is usually in contrast to "classification", which is any model that has a "categorical" or discrete outcome variable.)
    - hackerlight 3y ago
      I have a time series forecasting methodology question that I'll drop here. Suppose I have exogenous variables that vary over time, X(t). X is about 100 features. What are some methods I can apply onto X(t) to automatically engineer features that may be useful at predicting some noisy y(t)? I want to simultaneously capture interactions/interdependence between the columns of X, as well as the autocorrelation structure of X. If I treat X as merely tabular data, throwing it into a traditional regression model (e.g. XGBoost), it can capture the interdependence structure in X, but it will neglect the autocorrelation structure... Unless I manually engineer features that capture the autocorrelation structure in X (e.g. rolling/shifted/differenced features), but I want to explore methods that do that automatically.
      - disgruntledphd2 3y ago
        if the variables in X(t) have the same time steps, I'd probably look at the cross correlation function of the X vs y, and then built another model on the X to predict X(t+n) and use that as an input for Y(t).
        hackerlight 3y ago
        > built another model on the X to predict X(t+n) I like this idea. Practically, how would this look? Say X has 100 columns. Do we estimate 100 separate models f_{i}(X_{t}) = X_{i, t+1}, then generate 100 predictions for each time step, and then feed those 100 predictions into a regression to predict Y_{t}? > cross correlation function of the X vs y Is this supposed to be combined somehow with the f_{i} outputs?
        disgruntledphd2 3y ago
        > > cross correlation function of the X vs y Is this supposed to be combined somehow with the f_{i} outputs? I'd rank the variables by their CCF, and use the top(n) to try to predict the series of interest. Like, split Y in half, then use the X(1:(t/2)+n) to predict Y(t+n) to see if it works, and then if it works OK, actually model the top n X series and use them to really predict the Y. It's a pretty manual approach, but you could automate it once you have a better idea what you're aiming for.

nerdponx 3y ago

Yes, it is. A time series model is any model where the data varies over time; that is, a time series model is any model of time series data. And timeseries data is broadly anything where the data for a single thing/entity varies over time. There are no strict definitions here, just common conventions.

hackerlight 3y ago
Okay. And we can also say that there's some time series models that aren't regression models, right? For example, Kalman Filter is a "model" of a time series but isn't a regression.
- nerdponx 3y ago
  Correct. Although the term "regression" is a misnomer anyway, and often when people say "regression" they mean "linear model". And by "linear model", we mean specifically a model in which outputs/predictions are some fixed linear combination of the input. It is however possible to interpret the Kalman filter as a kind of dynamic regression model. Check out here if you want a good math workout on that topic: https://stats.stackexchange.com/q/330696 https://stats.stackexchange.com/q/330696 (Another somewhat distinct meaning of the term "regression" is any model with a "continuous" outcome variable. This is usually in contrast to "classification", which is any model that has a "categorical" or discrete outcome variable.)
  - hackerlight 3y ago
    I have a time series forecasting methodology question that I'll drop here. Suppose I have exogenous variables that vary over time, X(t). X is about 100 features. What are some methods I can apply onto X(t) to automatically engineer features that may be useful at predicting some noisy y(t)? I want to simultaneously capture interactions/interdependence between the columns of X, as well as the autocorrelation structure of X. If I treat X as merely tabular data, throwing it into a traditional regression model (e.g. XGBoost), it can capture the interdependence structure in X, but it will neglect the autocorrelation structure... Unless I manually engineer features that capture the autocorrelation structure in X (e.g. rolling/shifted/differenced features), but I want to explore methods that do that automatically.
    - disgruntledphd2 3y ago
      if the variables in X(t) have the same time steps, I'd probably look at the cross correlation function of the X vs y, and then built another model on the X to predict X(t+n) and use that as an input for Y(t).
      - hackerlight 3y ago
        > built another model on the X to predict X(t+n) I like this idea. Practically, how would this look? Say X has 100 columns. Do we estimate 100 separate models f_{i}(X_{t}) = X_{i, t+1}, then generate 100 predictions for each time step, and then feed those 100 predictions into a regression to predict Y_{t}? > cross correlation function of the X vs y Is this supposed to be combined somehow with the f_{i} outputs?
        disgruntledphd2 3y ago
        > > cross correlation function of the X vs y Is this supposed to be combined somehow with the f_{i} outputs? I'd rank the variables by their CCF, and use the top(n) to try to predict the series of interest. Like, split Y in half, then use the X(1:(t/2)+n) to predict Y(t+n) to see if it works, and then if it works OK, actually model the top n X series and use them to really predict the Y. It's a pretty manual approach, but you could automate it once you have a better idea what you're aiming for.