It's very easy - observe that each time the model is built it samples from the hyperparameters. So it's easy to have N workers all working in different hyperpar

It's very easy - observe that each time the model is built it samples from the hyperparameters. So it's easy to have N workers all working in different hyperparameters. To coordinate the testing of hyperparameters, you use something like vizier, which is a service that the workers can call into to get their hyperparameter assignment. If you have a distributed model you have to make sure all of your workers on a single variant get the same HPs. Again, vizier. (Hi, I'm job 25 worker 3, what are my HPs?)