Even though they'll use the dev set to tune hyperparameters, they'll end up making a "is this a good model for the paper" decision based on the test set. So, e.

Even though they'll use the dev set to tune hyperparameters, they'll end up making a "is this a good model for the paper" decision based on the test set. So, e.g., they'll try different architectures, tune hyperparameters on the dev set, then decide whether to use that architecture by looking at the test set. So that's part of it. AKA "gradient descent via grad student" The other part is, as you say, relatively constrained data sets.