Optimization in the context of machine learning is a search function that searches a param space by repeated calculations of gradients (derivatives: eg, calcula

Optimization in the context of machine learning is a search function that searches a param space by repeated calculations of gradients (derivatives: eg, calculate the derivative and test the objective function) to arrive at some optimal function. Learning in this case generally means the creation of models. Optimization is the process by which this happens. A few examples: Unsupervised learning - typically clustering and grouping things Supervised Learning - Example based learning where you're trying to label something. This could be sentiment, spam classification, object recognition over pixels, even extending to sequence labeling. Prediction/Regression - Predicting values based on a learned function. Learning is more about the goal trying to be achieved. Optimization is more of numerically solving relative to an objective function (also called an error function)