3 ms·
The problem with looking for a theoretical as to why one method should be chosen over another is that you run into the "No Free Lunch theorem"[1]: any two opti
by nl 10y ago
The problem with looking for a theoretical as to why one method should be chosen over another is that you run into the "No Free Lunch theorem"[1]:
any two optimization algorithms are equivalent when their performance is averaged across all possible problems
Once you accept that, then you start looking at practical considerations.
Having said that, if you do want to do the math then you might like the course from Oxford/Nando DeFreitas (now at DeepMind/Oxford)[2]
[1] https://en.wikipedia.org/wiki/No_free_lunch_theorem https://en.wikipedia.org/wiki/No_free_lunch_theorem
[2] https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPoTuPcqmOV53Fu https://www.youtube.com/playlist?list=PLE6Wd9FR--EfW8dtjAuPo..., https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearning/ https://www.cs.ox.ac.uk/people/nando.defreitas/machinelearni...