3 ms·
This is actually similar to a common approach in probabilistic modeling. Pick an initial model & set of probabilistic priors. Evaluate it with a "goodness of f
by glimcat 13y ago
This is actually similar to a common approach in probabilistic modeling.
Pick an initial model & set of probabilistic priors. Evaluate it with a "goodness of fit" heuristic function, then iterate & keep measuring while keeping the best solution discovered.
As long as the initial parameters are sort of reasonable, it will give you pretty good results for many problems.
It obviously doesn't get rid of the need for a better understanding of the problem. Improvements to how well your features describe important attributes of the problem tend to be strictly superior to your choice of learning algorithm (i.e. your exact process of iteration).
But as long as the initial solution is more or less on target? You can solve many problems by picking a solution that you know is inadequate, then iterating.
- ced 13y agoI'm fairly knowledgeable about Bayesian statistics, but I can't tell which technique you're referring to. It vaguely sounds like maximum likelihood, but ML doesn't iterate the model or priors. Could you explain further and give a reference?
- tprynn 13y agoIterative algorithms which attempt to solve maximum likelihood usually fall under the category of expectation maximization (a good example is K-means clustering). http://en.wikipedia.org/wiki/Expectation-maximization_algorithm http://en.wikipedia.org/wiki/Expectation-maximization_algori...
- ced 13y agoAh, I'd always thought of EM as iterating over the parameters given a model, but I suppose it makes sense to see it as a search for a model, including parameters, that fits the data. Cool.
- pavanred 13y agoPerhaps he/she is referring to conjugate priors [1], where the prior and the posterior are of the same family of distributions. Once you compute the posterior using a likelihood and prior, you obtain a posterior that is of the same family of distribution as the prior. This posterior can then be used as a prior for more samples. [1] http://en.wikipedia.org/wiki/Conjugate_prior http://en.wikipedia.org/wiki/Conjugate_prior