There are a lot of solutions for the problems you mention. For overfitting you can do data augmentation, normalization, dropout and early stop with the test set

There are a lot of solutions for the problems you mention. For overfitting you can do data augmentation, normalization, dropout and early stop with the test set. (and probably improve your dataset) More neurons means more parameters to adjust to your data, so overfitting is more likely to happen. It is like interpolation a function, the more parameters you use the more overfitting you have. > if your data is somewhat different from your training data in what humans would call an insignificant way, your network may easily start to fail humans call it insignificant because we have a deep knowledge of a lot of domains, meanwhile a network has been trained for an specific domain. So if you train the network with a distribution and then test it with another distribution it is not going to work. That is like quite obvious I think Deep learning works incredible well. It works so well that it outperform humans in some domains. So may want to rethink what are you doing, because I think (but I may be wrong) the reason you are failing applying deep learning is something related with your process and not with deep learning