Probably not even the model, but the preprocessing - MNIST-trained models expect all the digits to have the same preprocessing that MNIST had (because that's al

Probably not even the model, but the preprocessing - MNIST-trained models expect all the digits to have the same preprocessing that MNIST had (because that's all they saw), so if you apply a MNIST-optimized model directly to user-generated input without e.g. resizing and re-centering the images (or, alternatively, a "robustized" model trained with various data augmentations), then you're going to have horrible results.