3 ms·
It is interesting that an obviously "wrong" approach can have more success than expected. However, most people these days would use more modern forms of regular
by michaelochurch 12y ago
It is interesting that an obviously "wrong" approach can have more success than expected. However, most people these days would use more modern forms of regularization (e.g. ridge, Lasso, elastic net, stagewise with early stopping). Modern optimization software can handle it.
As I noted in my other comment, equal-weight constraints are still used in one domain: neural network architectures (e.g. convolutional neural nets). This is not quite the same thing because neural nets are so highly parameterized, and the weights interact in so many nonlinear ways, that individual weights don't have any meaning independent of others. It does show us that the idea of equal-weight regression has still lived on in one specific place.
- nkurz 12y agoEqual weights are also used in creating ensemble averages. It's always bothered me that the IPCC predictions for climate are done by averaging the results across models of highly varying quality, with each one weighted equally. Instinctively, I'd think you'd want to put greater trust in the 'better quality' models, but perhaps equal weighting is more defensible than it appears.