Among other things, I'd expect that depends very much on the dataset. Imagine some data, where 10% of the examples are significantly harder than the rest. Then

Among other things, I'd expect that depends very much on the dataset. Imagine some data, where 10% of the examples are significantly harder than the rest. Then going from 91% to 92% test set accuracy may actually indicated double the performance on the hard examples.