This is very close to my differentiation between the two. Machine learning fundamentally cares about model performance on other data, like validation or test s

This is very close to my differentiation between the two. Machine learning fundamentally cares about model performance on other data, like validation or test sets. It is looking for models that perform well, but not necessarily model the underlying process. Bayesian statistics, like most statistics, wants to accurately estimate parameters in the model. It cares most about models that are portraying the underlying data-generating process.