4 ms·
Would anybody explain me what is the takeaway from this approach? When I run the code I see something like: 38/100 = 42.0% - 38.0% accuracy, 90.47619% accurac
by haddr 10y ago
Would anybody explain me what is the takeaway from this approach?
When I run the code I see something like:
38/100 = 42.0% - 38.0% accuracy, 90.47619% accuracy of rated data.
90% is nice, but it looks to me like a tradeoff between precision and recall, where we just label those samples with the highest probability of pos/neg...
- KennyCason 10y agoSorry for the confusion. The primary take away was meant to be that a clustered Bayesian classifier (each trained on random samples of training data) offers significant improvements over traditional single Bayesian classifiers. Nothing revolutionary, just wanted to share a practical example that out performs most Bayesian implementations I have came across. I also wanted to share the results of model pruning and that tokenization matters. I originally coded this without much intent on sharing, so the post write-up could probably use a lot of work. :)
- malcolmgreaves 10y agoThis idea seems similar to local linear regression [1] in a way: approximate a non-linear model with multiple linear models that are fit to different data selections. Neat idea wrt. using a Naive Bayes classifier here! [1] https://en.wikipedia.org/wiki/Local_regression https://en.wikipedia.org/wiki/Local_regression
- KennyCason 10y agoInteresting, I actually implemented Local Regression from that very Wiki page! https://github.com/kennycason/ml/tree/master/src/main/java/kenny/ml/regression/nonlinear/local https://github.com/kennycason/ml/tree/master/src/main/java/k... This idea originally came to me after explaining Random Forests to someone as a way to improve their Decision Tree. On a whim, I built out a prototype and it offered improved results. :)
- haddr 10y agoThank you! How does it compare to the state of the art methods that used those datasets? Is it better or comparable?
- KennyCason 10y agoMy best runs using our proprietary tokenizers and much more training data (tens of thousands of tweets, reviews, and open source training data), yield results of: 87-93% accuracy while rating 50-70% of the data. I.e. If the classifier isn't "confident" then it doesn't rate the data. I'll run a quick test to demonstrate it's absolute accuracy. Though it definitely performs more than well enough to use in production, and is relatively little effort. It especially performs well in aggregate stats (>99% accuracy)
- deleted 10y ago[deleted]
- KennyCason 10y agoI guess I didn't completely answer. If you force the classifier to rate 100% of the data, I expect ~80%. If you set a reasonable confidence threshold, I expect mid 80s to low 90s. In both cases, the clustered Bayesian classifier outperforms the single classifier.