Couldn't agree more with your assessment. The idea of paying attention to 4.5 sigma events is crazy. The majority of classification tasks require broad accuracy

Couldn't agree more with your assessment. The idea of paying attention to 4.5 sigma events is crazy. The majority of classification tasks require broad accuracy and a measure of confidence. If the edge cases and boundaries are that important for your problem, use a one-class/exemplar svm to identify them. If I could add one more tip, it would be to make sure that all of your input and output features have a zero mean and a variance of 1 (something close to the normal distribution).