3 ms·

You claim that context is not important, but surely in a domain-specific corpus such as HN comments you have to take domain, if not context, into account? For

by jlees 15y ago

You claim that context is not important, but surely in a domain-specific corpus such as HN comments you have to take domain, if not context, into account? For example, words that are generally positive may be only used in a snarky context in hacker circles, whereas words that are generally negative may similarly be used with positive affect (e.g. the word "hacker" itself). Did you customise your lexicon at all for this genre? Also, in my experience, it's not the lexicon of words -- heck, SentiWordNet has existed for long enough -- it's what you do with that input signal that counts. Many "bag of words" approaches are on the shelf now, and very few of them are particularly accurate or clever, but most of them do the job just well enough that they get away with it.

tansey 15y ago
...surely in a domain-specific corpus such as HN comments you have to take domain, if not context, into account? That's a great insight and I believe you are correct. In general, I think the domain does have an impact and I am working on some automatic demographic profiling technology. However, for a first-look, I think our general word dictionary covers an abundant set of terms that do not vary much between domains. For instance, your comment has the following anxiety-evoking words: not, important, but, example, positive, used, hacker, negative, long, bag, but, away Of all those words, only "hacker" would likely need to be adjusted for the HN domain. For a first look, we'll chalk that up to some minor added noise in the data. EffectCheck does include the ability to automatically profile the document type (e.g., "Press Release" or "Motivational Speech") based on a training corpus. For example, most press releases have a high degree of confidence-- based on a corpus of press releases, we normalize the levels so that "Very High" on confidence for a press release requires much higher confidence elicited per-word than it would in a complaint letter.
- jlees 15y ago
  Is there really much of interest in press releases, or were they just a handy corpus you had around? I started my PhD off looking at affect in scientific papers, and changed tack because there basically is none. Anyway, I guess I'm not clearly understanding your definition of "anxiety" because few of those words evoke anxiety as I would define it -- not? bag? away? I'll try to dig into whatever you've published to deeper understand your system, but I'm not convinced by the standalone word approach yet, though I see what you're trying to do (I think). I'd really want to check out the bigrams/trigrams, even if you don't do full parsing or any intermediate sophistication. Obviously there's an efficiency aspect, so I guess if your results are good enough already and you have customers, no need to make sacrifices. On a tangent, how does your system interpret common HN words like "healthcare", "Obama" and "education", I wonder? Could you have an "intellectual" or "hacker-ish" dimension based on manual categorisation of words into "this is stuff HN should be talking about" vs "this is stuff that belongs on reddit"?
  - tansey 15y ago
    >Is there really much of interest in press releases, or were they just a handy corpus you had around? Actually, both. Corporate marketing departments spend a lot of effort crafting the proper press releases. Entire agencies exist to help companies with press releases. I think we make a great companion tool for professionals in that area. We have an example of how EffectCheck can help with press releases on our blog [1]. >On a tangent, how does your system interpret common HN words like "healthcare", "Obama" and "education", I wonder? Could you have an "intellectual" or "hacker-ish" dimension based on manual categorisation of words into "this is stuff HN should be talking about" vs "this is stuff that belongs on reddit"? We certainly can do that, and we offer it as a service to clients that want hand-crafted, demographic-specific dictionaries (like political campaigns). [1] http://blog.effectcheck.com/2011/04/14/delivering-bad-news-a-twitter-case-study/ http://blog.effectcheck.com/2011/04/14/delivering-bad-news-a...
Geee 15y ago
I'm pretty sure that even when we don't know how accurately it can measure the 'absolute' values, we can clearly see the change which is relative to the comments on the same context and domain. It can be argued if it really measures happiness etc., but I'm sure there's correlation in the change even if it's not accurate.

by jlees 15y ago

tansey 15y ago

...surely in a domain-specific corpus such as HN comments you have to take domain, if not context, into account? That's a great insight and I believe you are correct. In general, I think the domain does have an impact and I am working on some automatic demographic profiling technology. However, for a first-look, I think our general word dictionary covers an abundant set of terms that do not vary much between domains. For instance, your comment has the following anxiety-evoking words: not, important, but, example, positive, used, hacker, negative, long, bag, but, away Of all those words, only "hacker" would likely need to be adjusted for the HN domain. For a first look, we'll chalk that up to some minor added noise in the data. EffectCheck does include the ability to automatically profile the document type (e.g., "Press Release" or "Motivational Speech") based on a training corpus. For example, most press releases have a high degree of confidence-- based on a corpus of press releases, we normalize the levels so that "Very High" on confidence for a press release requires much higher confidence elicited per-word than it would in a complaint letter.

jlees 15y ago
Is there really much of interest in press releases, or were they just a handy corpus you had around? I started my PhD off looking at affect in scientific papers, and changed tack because there basically is none. Anyway, I guess I'm not clearly understanding your definition of "anxiety" because few of those words evoke anxiety as I would define it -- not? bag? away? I'll try to dig into whatever you've published to deeper understand your system, but I'm not convinced by the standalone word approach yet, though I see what you're trying to do (I think). I'd really want to check out the bigrams/trigrams, even if you don't do full parsing or any intermediate sophistication. Obviously there's an efficiency aspect, so I guess if your results are good enough already and you have customers, no need to make sacrifices. On a tangent, how does your system interpret common HN words like "healthcare", "Obama" and "education", I wonder? Could you have an "intellectual" or "hacker-ish" dimension based on manual categorisation of words into "this is stuff HN should be talking about" vs "this is stuff that belongs on reddit"?
- tansey 15y ago
  >Is there really much of interest in press releases, or were they just a handy corpus you had around? Actually, both. Corporate marketing departments spend a lot of effort crafting the proper press releases. Entire agencies exist to help companies with press releases. I think we make a great companion tool for professionals in that area. We have an example of how EffectCheck can help with press releases on our blog [1]. >On a tangent, how does your system interpret common HN words like "healthcare", "Obama" and "education", I wonder? Could you have an "intellectual" or "hacker-ish" dimension based on manual categorisation of words into "this is stuff HN should be talking about" vs "this is stuff that belongs on reddit"? We certainly can do that, and we offer it as a service to clients that want hand-crafted, demographic-specific dictionaries (like political campaigns). [1] http://blog.effectcheck.com/2011/04/14/delivering-bad-news-a-twitter-case-study/ http://blog.effectcheck.com/2011/04/14/delivering-bad-news-a...

Geee 15y ago

I'm pretty sure that even when we don't know how accurately it can measure the 'absolute' values, we can clearly see the change which is relative to the comments on the same context and domain. It can be argued if it really measures happiness etc., but I'm sure there's correlation in the change even if it's not accurate.