4 ms·
No no, linear just means some expression of the form y = b1*x1 + b2*x2 + ... + bp*xp which is in the heart of the model (perhaps more specific description
by ivansavz 3y ago
No no, linear just means some expression of the form
y = b1*x1 + b2*x2 + ... + bp*xp
which is in the heart of the model (perhaps more specific description would be linear combination). Whatever you call it, it's just some quantity y, that is constructed via an additive process from components x1...xp, and each component is multiplied by some constant (the coefficient of the linear combination).
This linear combination "core" of the model can be used directly (in which case it is geometrically a hyperplane), but you could also use non-linear transformation of the inputs or the outputs.
E.g. x1 could be a spline basis function, or log-transformed data, or anything.
Similarly, the output y can be passed though some non-linearity to force a particular output (like in logistic regression). Hence the statement is not vacuous
- EvgeniyZh 3y agoWell yeah, but what is the non trivial statement here? Linear model is linear, and if you add to your model non-linear steps (such as pre- or post- processing) then it is not longer linear and can handle non-linear data. In the same spirit, you could say that GPT4 is a linear model (token logits) with a very large preprocessing (transformer). Maybe what OP meant is that particular non linearities (splines) allow to keep some goodies from the linear model in non-linear settings. Still it's not clear for me in which settings exactly such models are good enough