I think the point is that if you want a model that gives the right answer, you should still use partial credit to figure out how far away you are from that goal

I think the point is that if you want a model that gives the right answer, you should still use partial credit to figure out how far away you are from that goal instead of binary correct-or-not accuracy. If you use a metric where improvement happens suddenly and unpredictably, you can't even estimate how much longer you need to train, because the ability might emerge spontaneously. But if the partial credit metric improves smoothly and predictably, you have a chance of extrapolating your training progress to see when you might hit your accuracy target, better than if you extrapolated accuracy directly. And if you find that the estimated time is too long and you decide to train a bigger model, you can try to extrapolate across model sizes to estimate how big of a model you might need.