5 ms·
Given the non-linear dynamics that proteins must live under, I like the usage of local minima as an analogy. It feels apt. Generally though, within the context
by lawrenceyan 6y ago
Given the non-linear dynamics that proteins must live under, I like the usage of local minima as an analogy. It feels apt.
Generally though, within the context of machine learning, one of the benefits of gradient descent, especially when stochastic, is that we can get past those local minima humps. Does this hold less true with respect to the process of sequence mutation that viruses go through?
- sudosysgen 6y agoThe difference in viruses is that if you get on the hump, you are outcompeted by other viruses that stayed in the minimum, which makes it a lot more difficult.
- lawrenceyan 6y agoThat makes a lot of sense. Thanks!
- jhrmnn 6y agoAnother way how to characterize the "context of [deep] machine learning" is that it is the regime of high temperature (stochasticity) as a result of small batches and/or large learning rates, and/or the regime of high momentum (Adam et al.). The evolutionary dynamics in the genetic space is (intuitively) at much lower temperature (shorter jumps) and with no momentum (because of the evolutionary pressure).