Yes, gradient-based hyperparameter optimization is one option for taking the human out of the loop to some extent. Of course, we can't avoid having to specify

Yes, gradient-based hyperparameter optimization is one option for taking the human out of the loop to some extent. Of course, we can't avoid having to specify hyper-hyper-parameters! More generally, automatic hyperparameter tuning (without gradients) is already a standard part of many researchers' pipelines. It's also part of good scientific practice, since it makes it harder to bias the results towards a particular method.