I was thinking along similar lines, it does smell like search to me at its core. But do you think there is a meaningful differentiation for something more nuanc

I was thinking along similar lines, it does smell like search to me at its core. But do you think there is a meaningful differentiation for something more nuanced like AdamW? Does the latter fall into the "brute force search" category too though it tends to converge faster (and arguably more optimally than SGD per Jeremy Howard's observations)?