To be honest I haven't thought about measuring the performance of different approaches but I have thought about a metric which will signal poor performance and

To be honest I haven't thought about measuring the performance of different approaches but I have thought about a metric which will signal poor performance and right now I'm interested in eliminating poor performance in my simplistic methodology. What I mean is that if my output 'sentiment scores' are skewed towards the negative and centered around a negative number (~70% of my documents are scored as 'negative sentiment' using the above scheme) then I know that my model is broken because I know that my document "test set" is necessarily neutral (or even positively skewed). Mathematically, I need to ensure my 'cost function' is a reflection of reality which means my sentiment scores at the end of the day need to be symmetrical or slightly positive skew with a mean of approximately 0. I can use regularization to trick the model into looking like this but I don't want to overfit and I can't think of a theoretical reason that I'm getting negative bias for this simple model except perhaps that my positive corpus is missing 'true positive sentiment' (ala a parameter versus an estimate in frequentist statistics) which could be the byproduct of the simplistic 'bag of words' assumption (words are assumed independent) aka breaking down my analysis on the basis of 1-grams (single words). As per my other comment, 2-grams could be the more natural way to express positive sentiment while 1-gram negative words are more readily available. Perhaps this is a sign of our pessimism as a species haha :)