"hyperparameter descent by grad student" is a lot more efficient at first, much of the time the loss function has caveats which make it easy to fall into parts

"hyperparameter descent by grad student" is a lot more efficient at first, much of the time the loss function has caveats which make it easy to fall into parts of the search space which don't actually accomplish the task (for example when empty frame = true reduces the loss) something a grad student would easily figure out. Until you get to the point where you are fairly certain the search is within the right space its hard to ensure that throwing a lot of compute at that search will yield anything useful.