3 ms·
Sentiment mining is a hot topic, with many possible uses, but coders can usually do something decent without any crazy magic. The problems start when any gramm
by guylhem 14y ago
Sentiment mining is a hot topic, with many possible uses, but coders can usually do something decent without any crazy magic.
The problems start when any grammar processing happens - because algorithms are then dependant of the language
(What about this service? I didn't find many details about it.)
Meanwhile, if you stay at the dictionary level, you can get decent results - that's what things such as the General Inquirer let you do - and you won't need for an API to do basic stemming of words and summing up the sentiment values from a database - see http://www.wjh.harvard.edu/~inquirer/ http://www.wjh.harvard.edu/~inquirer/
I've played with it - that's more or less a 1 day project if you use the existing database.
Bonus points if you add simple things - such as replacing the stemming algorithm by a Levenshtein distance or a phonex-style algorithm (typos, etc), or use inverse term frequency to mitigate the influence of "very sentimental" words that are falling into common use, i.e. whose current sentimental ponderation differs from the general inquirer ponderation.
I once played with sentiment analysis of mailing lists, to automatically get red flags when sh*t happens :-)
- bryanh 14y agoThat is really interesting (great link too!). As coders ourselves we've had the same experience: spinning up something simple is usually pretty straightforward. What we are a little more curious about is the ability to bring simple sentiment mining to non-coders (IE: your support or social media guy) via off the shelf tools.
- guylhem 14y agoBased on my experience the sensibility is good (ie it often gives a red flag when something happens) but the specificity is poor (ie there are many false positives). My theory at the time was that some kind of semantic sliding was happening and led "sentiment loaded" words into common use, while removing their specific sentiment load (#)- which is why I tried TF-IDF, but even then the results where not that good and I was too lazy to dig into grammar analysis to correct for the other possible bias (negations and other complexities such as irony becoming more frequent) (Are you part of the team behind this product? Would you have some comments about sensibility and specificity of your results?) (#) : ex : teen talk ("OMG it's so crazy and fabulous!") is full of words that may have indicated a strong sentiment, but which that are devoid of it in the given "teen talk" context.
- bryanh 14y agoI am not a part of the Semantria team, instead I'm a Zapier cofounder (we offer some simple piping or glue that makes it easy to move data between APIs without code). Happy to put you in touch with the Semantria guys, they are very knowledgable.