3 ms·

Also, it's not clear to me why this entry makes it to the front page over all the tons of other research paper / monographs. It's significance does not stand ou

by kleiba 9y ago

Also, it's not clear to me why this entry makes it to the front page over all the tons of other research paper / monographs. It's significance does not stand out to me, i.e., I cannot grasp why this is something I should look at instead of many other interesting reads from the field.

jpfr 9y ago
This was just released as part of the now series for machine learning. They usually publish superb reference material. Research that is foundational and/or has become a stable body of work, but is recent enough to be interesting. Read these and you are on the edge of what is currently happening. http://www.nowpublishers.com/MAL http://www.nowpublishers.com/MAL
mturmon 9y ago
There has been a big question about the place of convex relaxations for solving learning problems, and this responds to that question. That is, you have a learning problem with some fit criterion that is non-convex. Maybe you choose a different objective function, or different parameterization of the problem, so that it is convex. Then you can use a convex solver to find the guaranteed optimum of the "wrong" problem. Using convex relaxations has been a trend for perhaps 20 years - it avoids a whole bunch of nagging "what if my solution is just a local max" questions. Or, you can use older techniques, like expectation-maximization, or in general, alternating minimization, that are adapted for these kinds of learning problems. Learning problems often have observed variables, and hidden variables, and if you knew the hidden variables, the optimization problem would be easier. These algorithms work on such problems. But, they don't find the global maximum. So you have found a possibly-good solution to the "right" problem. Lately, some very interesting results have shown that, under some conditions, there are probabilistic guarantees that the second approach will find the global optimum [e.g., https://arxiv.org/abs/1408.2156 https://arxiv.org/abs/1408.2156]. Since the second approach often scales much better, people are naturally interested.

by kleiba 9y ago

jpfr 9y ago

This was just released as part of the now series for machine learning. They usually publish superb reference material. Research that is foundational and/or has become a stable body of work, but is recent enough to be interesting. Read these and you are on the edge of what is currently happening. http://www.nowpublishers.com/MAL http://www.nowpublishers.com/MAL

mturmon 9y ago

There has been a big question about the place of convex relaxations for solving learning problems, and this responds to that question. That is, you have a learning problem with some fit criterion that is non-convex. Maybe you choose a different objective function, or different parameterization of the problem, so that it is convex. Then you can use a convex solver to find the guaranteed optimum of the "wrong" problem. Using convex relaxations has been a trend for perhaps 20 years - it avoids a whole bunch of nagging "what if my solution is just a local max" questions. Or, you can use older techniques, like expectation-maximization, or in general, alternating minimization, that are adapted for these kinds of learning problems. Learning problems often have observed variables, and hidden variables, and if you knew the hidden variables, the optimization problem would be easier. These algorithms work on such problems. But, they don't find the global maximum. So you have found a possibly-good solution to the "right" problem. Lately, some very interesting results have shown that, under some conditions, there are probabilistic guarantees that the second approach will find the global optimum [e.g., https://arxiv.org/abs/1408.2156 https://arxiv.org/abs/1408.2156]. Since the second approach often scales much better, people are naturally interested.