4 ms·

Fascinating project, I worked for a data analytics company and we always had a basic test that we'd try to throw at any Sentiment analysis engine. // Negative

by joeblau 13y ago

Fascinating project, I worked for a data analytics company and we always had a basic test that we'd try to throw at any Sentiment analysis engine. // Negative > This is shit. // Positive > This is the shit. Most engines can't sort that out. I'm definitely going to take a more in-depth look at this.

kurige 13y ago
It doesn't look like this one was able to sort those two out either. It gave me strongly neutral signals for both sentences, but that may be because 'shit' isn't currently tagged.
pygy_ 13y ago
"This is the shit" is a very specific idiom. Unless the engine has been specifically trained to understand it, it is not surprising that it doesn't know what to do with it.
byoung2 13y ago
"This is shit" isn't inherently negative. Consider the question "what is the brown substance in the toilet?" and you can answer a neutral "this is shit".
- gfodor 13y ago
  Presumably if you throw a realistic corpus of written text you will have much more occurrences of the idiom.
- Cyranix 13y ago
  Sentiment analysis implies attitudes, which imply context. A robust sentiment analysis system must either be trained on a targeted corpus (in which entities/concepts and attitudes are well aligned) or include some way of establishing a more targeted context after having been trained on a general-purpose corpus. That is to say, people generally aren't describing the physical material occupying a toilet in a corpus of movie reviews.
  - byoung2 13y ago
    I was referring to the previous comment, where the two lines about shit were used as a test for any sentiment analysis engine. Your point about context is what I was trying to illustrate.
- dagw 13y ago
  I imagine that someone using the word 'shit' to describe the brown stuff in the toilet is probably negatively inclined towards it. Someone discussing it in a more neutral context (like a doctor or scientist) would probably be more likely to say 'feces'

by joeblau 13y ago

kurige 13y ago

It doesn't look like this one was able to sort those two out either. It gave me strongly neutral signals for both sentences, but that may be because 'shit' isn't currently tagged.

pygy_ 13y ago

"This is the shit" is a very specific idiom. Unless the engine has been specifically trained to understand it, it is not surprising that it doesn't know what to do with it.

byoung2 13y ago

"This is shit" isn't inherently negative. Consider the question "what is the brown substance in the toilet?" and you can answer a neutral "this is shit".

gfodor 13y ago
Presumably if you throw a realistic corpus of written text you will have much more occurrences of the idiom.
Cyranix 13y ago
Sentiment analysis implies attitudes, which imply context. A robust sentiment analysis system must either be trained on a targeted corpus (in which entities/concepts and attitudes are well aligned) or include some way of establishing a more targeted context after having been trained on a general-purpose corpus. That is to say, people generally aren't describing the physical material occupying a toilet in a corpus of movie reviews.
- byoung2 13y ago
  I was referring to the previous comment, where the two lines about shit were used as a test for any sentiment analysis engine. Your point about context is what I was trying to illustrate.
dagw 13y ago
I imagine that someone using the word 'shit' to describe the brown stuff in the toilet is probably negatively inclined towards it. Someone discussing it in a more neutral context (like a doctor or scientist) would probably be more likely to say 'feces'