4 ms·

Stochastic gradient descent and other things like genetic algorithms and simulated annealing are random search techniques specifically created and taught in the

by mlthoughts2018 6y ago

Stochastic gradient descent and other things like genetic algorithms and simulated annealing are random search techniques specifically created and taught in the context of machine learning.

kgwgk 6y ago
Simulated annealing goes back to the seventies and was definitely not "specifically created in the context of machine learning". Many (most?) optimization techniques have their origin in Operations Research.
- mlthoughts2018 6y ago
  Simulated annealing was developed for purposes of parameter fitting in physics modeling, based on Metropolis Hastings which was likewise developed in the context of parameter inference for model fitting. Simulated annealing for eg traveling salesman problem came later. I do agree some optimization algorithms are rooted in other fields. I wasn’t trying to say that machine learning is the only historic field from which optimization methods were developed. I just wanted to point out it is a major historic field where some highly respected search and optimization procedures were first created, since people often overlook how old machine learning is and the vast set of modeling procedures apart from neural networks that make up the core of machine learning.
cupofcoffee 6y ago
Lol. Are all optimization problems specifically created and taught in the context of machine learning? Do you know any other context than ML?
- mlthoughts2018 6y ago
  Your comment is not coherent. You ask, “ Are all optimization problems specifically created and taught in the context of machine learning?” but this has no logical or semantic connection to my comment in any way. It fails to be a valid response or question. Instead it seems you falsely believe you are writing with some sarcasm that endows rhetorical flair to undercut my comment. It’s very rude and juvenile in addition to being wholly ineffective.

Stochastic gradient descent and other things like genetic algorithms and simulated annealing are random search techniques specifically created and taught in the context of machine learning.

kgwgk 6y ago

Simulated annealing goes back to the seventies and was definitely not "specifically created in the context of machine learning". Many (most?) optimization techniques have their origin in Operations Research.

mlthoughts2018 6y ago
Simulated annealing was developed for purposes of parameter fitting in physics modeling, based on Metropolis Hastings which was likewise developed in the context of parameter inference for model fitting. Simulated annealing for eg traveling salesman problem came later. I do agree some optimization algorithms are rooted in other fields. I wasn’t trying to say that machine learning is the only historic field from which optimization methods were developed. I just wanted to point out it is a major historic field where some highly respected search and optimization procedures were first created, since people often overlook how old machine learning is and the vast set of modeling procedures apart from neural networks that make up the core of machine learning.

cupofcoffee 6y ago

Lol. Are all optimization problems specifically created and taught in the context of machine learning? Do you know any other context than ML?

mlthoughts2018 6y ago
Your comment is not coherent. You ask, “ Are all optimization problems specifically created and taught in the context of machine learning?” but this has no logical or semantic connection to my comment in any way. It fails to be a valid response or question. Instead it seems you falsely believe you are writing with some sarcasm that endows rhetorical flair to undercut my comment. It’s very rude and juvenile in addition to being wholly ineffective.