5 ms·

Read this quickly, but my immediate thought when I saw the PCA result vectors was that they resembled the Fourier basis. Given a relatively slow-moving set of s

by fovc 10y ago

Read this quickly, but my immediate thought when I saw the PCA result vectors was that they resembled the Fourier basis. Given a relatively slow-moving set of signals (which I think these are, given the 10,000 word sliding window), wouldn't you expect the PCA to give you exactly this? In other words, what is the null hypothesis here and is it rejected?

ggggtez 10y ago
I think the idea is, of you compare short fiction to the Harry Potter graph, most stories have simple emotional arcs, rather than complicated back and forth. Though, if you limit the word length allowed... I wonder how they deal with "it was the best of times. it was the worst of times"...
- schoen 10y ago
  You could mess up pretty easily at a local level with missed negations. For example, John Lennon's song "Imagine" invites people to envision a world without various things; if you assign negative sentiment to those things, you might infer that the song overall had very negative affect. Or religious prophecies about a future era in which people wouldn't have to suffer from various afflictions.
  - mcguire 10y ago
    Jockers, in one of his posts, argues that it's less of a problem than you might think because it's a local problem that just adds noise to the data. Plus, his different sentiment analysis techniques seem to more-or-less agree. On the other hand, he tells of showing a graph to his son, of a book his son had read, and being told the graph was completely wrong for part of the book---it turns out that part of the book was written from the antagonist's viewpoint and the "emotional valence" was apparently inverted. See one of the follow-ups to http://www.matthewjockers.net/2015/02/02/syuzhet/ http://www.matthewjockers.net/2015/02/02/syuzhet/.
    - schoen 10y ago
      The longer discussion is http://www.matthewjockers.net/2015/03/04/some-thoughts-on-annies-thoughts-about-syuzhet/ http://www.matthewjockers.net/2015/03/04/some-thoughts-on-an... (which gives examples of negation that sentiment analysis packages miss, but then some empirical evidence that analysis of sentiment in several literary works doesn't seem to be affected very much by this issue). The antagonist's viewpoint issue appeared in footnote 1 of http://www.matthewjockers.net/2015/02/25/the-rest-of-the-story/ http://www.matthewjockers.net/2015/02/25/the-rest-of-the-sto... and it does seem like the sentiment analysis would clearly be backwards overall if this kind of thing continued for most of a book. (An example might be if an author depicted people enjoying themselves while committing horrible acts, and used more individual words related to the enjoyment than to the acts.) Thanks for the references.
      - schoen 10y ago
        > people enjoying themselves while committing horrible acts Now I'm kind of curious about sentiment analysis of something like Fight Club.
chombier 10y ago
I thought the very same. I guess the most interesting result they obtain is the slight asymmetry in some of the low-frequency modes, which the Fourier basis does not exhibit.
- Pyramus 10y ago
  Me too, interested in debunking this properly?

by fovc 10y ago

ggggtez 10y ago

I think the idea is, of you compare short fiction to the Harry Potter graph, most stories have simple emotional arcs, rather than complicated back and forth. Though, if you limit the word length allowed... I wonder how they deal with "it was the best of times. it was the worst of times"...

schoen 10y ago
You could mess up pretty easily at a local level with missed negations. For example, John Lennon's song "Imagine" invites people to envision a world without various things; if you assign negative sentiment to those things, you might infer that the song overall had very negative affect. Or religious prophecies about a future era in which people wouldn't have to suffer from various afflictions.
- mcguire 10y ago
  Jockers, in one of his posts, argues that it's less of a problem than you might think because it's a local problem that just adds noise to the data. Plus, his different sentiment analysis techniques seem to more-or-less agree. On the other hand, he tells of showing a graph to his son, of a book his son had read, and being told the graph was completely wrong for part of the book---it turns out that part of the book was written from the antagonist's viewpoint and the "emotional valence" was apparently inverted. See one of the follow-ups to http://www.matthewjockers.net/2015/02/02/syuzhet/ http://www.matthewjockers.net/2015/02/02/syuzhet/.
  - schoen 10y ago
    The longer discussion is http://www.matthewjockers.net/2015/03/04/some-thoughts-on-annies-thoughts-about-syuzhet/ http://www.matthewjockers.net/2015/03/04/some-thoughts-on-an... (which gives examples of negation that sentiment analysis packages miss, but then some empirical evidence that analysis of sentiment in several literary works doesn't seem to be affected very much by this issue). The antagonist's viewpoint issue appeared in footnote 1 of http://www.matthewjockers.net/2015/02/25/the-rest-of-the-story/ http://www.matthewjockers.net/2015/02/25/the-rest-of-the-sto... and it does seem like the sentiment analysis would clearly be backwards overall if this kind of thing continued for most of a book. (An example might be if an author depicted people enjoying themselves while committing horrible acts, and used more individual words related to the enjoyment than to the acts.) Thanks for the references.
    - schoen 10y ago
      > people enjoying themselves while committing horrible acts Now I'm kind of curious about sentiment analysis of something like Fight Club.

chombier 10y ago

I thought the very same. I guess the most interesting result they obtain is the slight asymmetry in some of the low-frequency modes, which the Fourier basis does not exhibit.

Pyramus 10y ago
Me too, interested in debunking this properly?