in that regard it’s mostly in the first few paragraphs but it’s subtley stated... If you have a model and somecparameter space on it your trying to optimise th

in that regard it’s mostly in the first few paragraphs but it’s subtley stated... If you have a model and somecparameter space on it your trying to optimise then you can’t do much other than search the whole parameter space or approximate such a search intelligently. However, if your model is differeniable with respect to its parameters then it means you’ve got gradient ... and if you’ve got gradient then you can cut out a lot of searching because you can use gradient ascent and other such techniques . So basically: differentiable means can be differentiated which means has gradient which means search less. The rest of the article basically says that’d be nice because there is a way to think about differentiability modularly and therefore there may be a route to Lego things together from across fields to make uber models which still optimise well but use specialist knowledge. The parameter space stuff just means that less knowledge means more data and more parameters which is bad.