5 ms·

Visualizing Hacker News

74 points by r3570r3 15y ago

This is a visualization of various company and topic sentiments as they are discussed on Hacker News.

niels_olson 15y ago
interesting idea. Doc here. I would organize them as 1. Depresssion 2. Anxiety 3. Hostility 4. Happiness 5. Confidence 6. Compassion Anxiety and Depression definitely go together, in the same way that hostility and happiness are actually similar in terms type of emotion, in terms of level of intellect, and confidence and compassion would be at the highest end of that spectrum. I would also change the colors to group them: anxiety and depression as blues, hostile and happy as reds, and confidence and compassion as greens. Or something like that.
- tansey 15y ago
  Thanks. My co-founders [1][2] actually did spend a decent amount of time thinking about this. The emotions are first sorted into negative on the left and positive on the right. The order is for symmetry of left/right side words: - Anxiety is the opposite of Confidence - Hostility is the opposite of Compassion - Depression is the opposite of Happiness As for the colors, each one is correlated to the typical psychological association for that color. The exception is Happiness which should be yellow but yellow doesn't render well on websites. [1] Yanon Volcani (Clinical Psychologist): http://www.volcani.com http://www.volcani.com [2] David Fogel (AI Expert): http://www.natural-selection.com/people_dfogel.html http://www.natural-selection.com/people_dfogel.html
  - niels_olson 15y ago
    If you're trying to illustrate position on three spectrums, illustrate three spectrums. 6 bars on a graph implies, however vaguely, to the reader, despite your intent, that the abscissa means something. But it doesn't. It just confuses the picture. so get rid of that abscissa. In any case, anxiety and depression are generally a package deal in clinic. So I would put them together visually. Depression --------------------X------- Happiness Anxiety ----------------X----------- Confidence Hostility -------------------x-------- Compassion
ddphone 15y ago
It'd be nice to also see typical comments both with high and low scores in each category.
asdffdasdf 15y ago
It'd be nice to also see typical comments both with high and low scores in each category.
- r3570r3 15y ago
  Contact the dev on Twitter here: https://twitter.com/scottwisme https://twitter.com/scottwisme
- Stwerner 15y ago
  I was really busy leading up to the contest, so I had to cut back on a lot of the features I wanted in order to hit the deadline. I will be working on adding a more visualisations and detail as quickly as I can.
ihodes 15y ago
So cool! This is what I had started working on for the HNSearch API Contest, but I hadn't gotten far. I'm SO glad someone did this. I feel as though this could be made into a useful and viable product, if marketed correctly and accurate enough.
- Stwerner 15y ago
  Do you mean as like a dashboard for a bunch of different social news sites? What would be some killer features that you think would make it in to something people would pay for?
  - ihodes 15y ago
    Eventually, yes, adding sites like Reddit, even newspapers, trending topics on twitter (and then evaluating the mood of tweets with the keyword(s). Also, allowing them to see the popularity of certain trends (traffic, if you will). I am not familiar with the industry…maybe the product exists…but it seems incredibly useful! Well done, too.
r3570r3 15y ago
Too bad the EffectCheck API is not open for all. Looks like a well-parameterized sentiment analysis tool. What does this page tell us: http://effectcheck.com/pricing http://effectcheck.com/pricing It can be used for a stock analysis and dampening or amplification caused from other firehose like sources like Twitter.
- tansey 15y ago
  It hopefully tells you that we're happy to work with you if you'd like to use sentiment analysis or lexical impact analysis in your company. :) We are focused on B2B applications of our technology rather than the consumer/API side. However, if you have a cool idea for how you'd like to use EffectCheck, email me [1] and I'll be happy to discuss it with you. [1] wesley.tansey@effectcheck.com
gizzlon 15y ago
Do you expose the underlying data? For example, what are the most hostile articles about Microsoft?
- Stwerner 15y ago
  I've got a bunch of ideas on ways to add more data, check back over the next day or so!
ntoshev 15y ago
Can it work farther back in time? interesting if there are longer term trends. Happiness particularly can be indicative about the overall satisfaction from HN.
- tansey 15y ago
  I did an overall analysis of the HN community from 2007-2011 a few weeks ago [1]. [1] http://blog.effectcheck.com/2011/05/31/do-social-news-sites-deteriorate/ http://blog.effectcheck.com/2011/05/31/do-social-news-sites-...
tansey 15y ago
Wow, how did this get discovered? I'm a co-founder at EffectCheck and I was working closely with Scott this weekend as he was building this. It wasn't really ready for viewing yet, but okay... :) Please note that the top graph is currently a mixture of two sets of data. The older points were using a less sensitive and improperly calibrated HN comment model, hence why everything is drifting around near "Typical." The points from 14:47 onward are using the correct model. For those wondering why HN was so negative from 14:47-20:47, I believe the main topics of conversation were the Bitcoin and Sega debacles. Makes sense that people were really anxious given all that news.
- bane 15y ago
  Impressive work, the analysis among many different emotional vectors is interesting. How do you guys work with non-English languages as well?
- palish 15y ago
  I'd like to know the scientific basis of EffectCheck. The site offers no explanation. Could you point out specific research papers, or link to technical info? Thanks!
  - Stwerner 15y ago
    I'll let tansey go in to more detail when he gets back. But for now, heres a link to one of his comments from last time EffectCheck was on here. http://news.ycombinator.com/item?id=2603438 http://news.ycombinator.com/item?id=2603438
    - palish 15y ago
      So would you say this comment is an accurate description? http://news.ycombinator.com/item?id=2603825 http://news.ycombinator.com/item?id=2603825 "It seems that the point is to introduce their special-sauce black box, with an argument to authority about its methodology. I think the correlations you ask for are where the problems will lie, in that there is a value judgement that is being hidden. If I can put myself out on a limb here, I'd say that that measurement is going to be fundamentally unscientific." Also, this seems like an important question, if you claim to be based in science: "How can we falsify your claims? That is, what is a test we can perform that if it went a certain way, would show that your claims are false?" It occurs to me that human color perception is another "subjective but scientific" field. You might want to research the experiments they performed for e.g. color matching in varying illumination levels, then come up with an experiment with a similar structure. Specifically, if EffectCheck is accurate, then it should correlate with how 100 average people would classify e.g. a sample of 1,000 twitter messages.
robin_reala 15y ago
See also Rob Hawkes’ (from Mozilla) attempts to map Twitter sentiment: http://rawkes.com/blog/2011/04/19/finding-patterns-in-twitter-sentiment http://rawkes.com/blog/2011/04/19/finding-patterns-in-twitte...
Stwerner 15y ago
Wow, I slept through this whole thing. I'll be around now though. If anyone has any requests for different ways to interact with the data, let me know and I'll work on getting it added to the site.
SonicSoul 15y ago
are you also behind the http://www.wefeelfine.org/ http://www.wefeelfine.org/ project? It was neat when it came out years ago... Although still not sure how to make any meaningful use of this tool. I suppose it is more useful for content generators..
- tansey 15y ago
  That's really cool! I have never seen that project before. I checked our colors compared to the results of the study: Anxiety: #2288ff vs. #595884 (anxious) Hostility: #ee0000 vs. #E0192C (angry) -- hostile only had 100 samples Depression: #3300cc vs. #283152 (depressed) Confidence: #ee00cc vs. #FF7F00 (confident) Compassion: #11bb00 vs. #00696F (sorry) -- not sure on this word choice but compassionate wasn't available. Loving was #004E6F Happiness: #ff8800 vs. #FF7F00 (happy) So I would say we did well on all except maybe Confidence. However, confident shows up as almost the same color as happy, so we would have to differentiate somehow anyway.
Stwerner 15y ago
I'll be adding the link to the site once I get home from work today, but I've set up a mailing list for anyone who might be interested in getting an email when new features are pushed out: http://eepurl.com/emtQU http://eepurl.com/emtQU
potomak 15y ago
it'd be cool to integrate it with http://smood.it http://smood.it

74 points by r3570r3 15y ago

This is a visualization of various company and topic sentiments as they are discussed on Hacker News.

interesting idea. Doc here. I would organize them as 1. Depresssion 2. Anxiety 3. Hostility 4. Happiness 5. Confidence 6. Compassion Anxiety and Depression definitely go together, in the same way that hostility and happiness are actually similar in terms type of emotion, in terms of level of intellect, and confidence and compassion would be at the highest end of that spectrum. I would also change the colors to group them: anxiety and depression as blues, hostile and happy as reds, and confidence and compassion as greens. Or something like that.

tansey 15y ago
Thanks. My co-founders [1][2] actually did spend a decent amount of time thinking about this. The emotions are first sorted into negative on the left and positive on the right. The order is for symmetry of left/right side words: - Anxiety is the opposite of Confidence - Hostility is the opposite of Compassion - Depression is the opposite of Happiness As for the colors, each one is correlated to the typical psychological association for that color. The exception is Happiness which should be yellow but yellow doesn't render well on websites. [1] Yanon Volcani (Clinical Psychologist): http://www.volcani.com http://www.volcani.com [2] David Fogel (AI Expert): http://www.natural-selection.com/people_dfogel.html http://www.natural-selection.com/people_dfogel.html
- niels_olson 15y ago
  If you're trying to illustrate position on three spectrums, illustrate three spectrums. 6 bars on a graph implies, however vaguely, to the reader, despite your intent, that the abscissa means something. But it doesn't. It just confuses the picture. so get rid of that abscissa. In any case, anxiety and depression are generally a package deal in clinic. So I would put them together visually. Depression --------------------X------- Happiness Anxiety ----------------X----------- Confidence Hostility -------------------x-------- Compassion

ddphone 15y ago

It'd be nice to also see typical comments both with high and low scores in each category.

asdffdasdf 15y ago

It'd be nice to also see typical comments both with high and low scores in each category.

r3570r3 15y ago
Contact the dev on Twitter here: https://twitter.com/scottwisme https://twitter.com/scottwisme
Stwerner 15y ago
I was really busy leading up to the contest, so I had to cut back on a lot of the features I wanted in order to hit the deadline. I will be working on adding a more visualisations and detail as quickly as I can.

ihodes 15y ago

So cool! This is what I had started working on for the HNSearch API Contest, but I hadn't gotten far. I'm SO glad someone did this. I feel as though this could be made into a useful and viable product, if marketed correctly and accurate enough.

Stwerner 15y ago
Do you mean as like a dashboard for a bunch of different social news sites? What would be some killer features that you think would make it in to something people would pay for?
- ihodes 15y ago
  Eventually, yes, adding sites like Reddit, even newspapers, trending topics on twitter (and then evaluating the mood of tweets with the keyword(s). Also, allowing them to see the popularity of certain trends (traffic, if you will). I am not familiar with the industry…maybe the product exists…but it seems incredibly useful! Well done, too.

r3570r3 15y ago

Too bad the EffectCheck API is not open for all. Looks like a well-parameterized sentiment analysis tool. What does this page tell us: http://effectcheck.com/pricing http://effectcheck.com/pricing It can be used for a stock analysis and dampening or amplification caused from other firehose like sources like Twitter.

tansey 15y ago
It hopefully tells you that we're happy to work with you if you'd like to use sentiment analysis or lexical impact analysis in your company. :) We are focused on B2B applications of our technology rather than the consumer/API side. However, if you have a cool idea for how you'd like to use EffectCheck, email me [1] and I'll be happy to discuss it with you. [1] wesley.tansey@effectcheck.com

gizzlon 15y ago

Do you expose the underlying data? For example, what are the most hostile articles about Microsoft?

Stwerner 15y ago
I've got a bunch of ideas on ways to add more data, check back over the next day or so!

ntoshev 15y ago

Can it work farther back in time? interesting if there are longer term trends. Happiness particularly can be indicative about the overall satisfaction from HN.

tansey 15y ago
I did an overall analysis of the HN community from 2007-2011 a few weeks ago [1]. [1] http://blog.effectcheck.com/2011/05/31/do-social-news-sites-deteriorate/ http://blog.effectcheck.com/2011/05/31/do-social-news-sites-...

tansey 15y ago

Wow, how did this get discovered? I'm a co-founder at EffectCheck and I was working closely with Scott this weekend as he was building this. It wasn't really ready for viewing yet, but okay... :) Please note that the top graph is currently a mixture of two sets of data. The older points were using a less sensitive and improperly calibrated HN comment model, hence why everything is drifting around near "Typical." The points from 14:47 onward are using the correct model. For those wondering why HN was so negative from 14:47-20:47, I believe the main topics of conversation were the Bitcoin and Sega debacles. Makes sense that people were really anxious given all that news.

bane 15y ago
Impressive work, the analysis among many different emotional vectors is interesting. How do you guys work with non-English languages as well?
palish 15y ago
I'd like to know the scientific basis of EffectCheck. The site offers no explanation. Could you point out specific research papers, or link to technical info? Thanks!
- Stwerner 15y ago
  I'll let tansey go in to more detail when he gets back. But for now, heres a link to one of his comments from last time EffectCheck was on here. http://news.ycombinator.com/item?id=2603438 http://news.ycombinator.com/item?id=2603438
  - palish 15y ago
    So would you say this comment is an accurate description? http://news.ycombinator.com/item?id=2603825 http://news.ycombinator.com/item?id=2603825 "It seems that the point is to introduce their special-sauce black box, with an argument to authority about its methodology. I think the correlations you ask for are where the problems will lie, in that there is a value judgement that is being hidden. If I can put myself out on a limb here, I'd say that that measurement is going to be fundamentally unscientific." Also, this seems like an important question, if you claim to be based in science: "How can we falsify your claims? That is, what is a test we can perform that if it went a certain way, would show that your claims are false?" It occurs to me that human color perception is another "subjective but scientific" field. You might want to research the experiments they performed for e.g. color matching in varying illumination levels, then come up with an experiment with a similar structure. Specifically, if EffectCheck is accurate, then it should correlate with how 100 average people would classify e.g. a sample of 1,000 twitter messages.

robin_reala 15y ago

See also Rob Hawkes’ (from Mozilla) attempts to map Twitter sentiment: http://rawkes.com/blog/2011/04/19/finding-patterns-in-twitter-sentiment http://rawkes.com/blog/2011/04/19/finding-patterns-in-twitte...

Stwerner 15y ago

Wow, I slept through this whole thing. I'll be around now though. If anyone has any requests for different ways to interact with the data, let me know and I'll work on getting it added to the site.

SonicSoul 15y ago

are you also behind the http://www.wefeelfine.org/ http://www.wefeelfine.org/ project? It was neat when it came out years ago... Although still not sure how to make any meaningful use of this tool. I suppose it is more useful for content generators..

tansey 15y ago
That's really cool! I have never seen that project before. I checked our colors compared to the results of the study: Anxiety: #2288ff vs. #595884 (anxious) Hostility: #ee0000 vs. #E0192C (angry) -- hostile only had 100 samples Depression: #3300cc vs. #283152 (depressed) Confidence: #ee00cc vs. #FF7F00 (confident) Compassion: #11bb00 vs. #00696F (sorry) -- not sure on this word choice but compassionate wasn't available. Loving was #004E6F Happiness: #ff8800 vs. #FF7F00 (happy) So I would say we did well on all except maybe Confidence. However, confident shows up as almost the same color as happy, so we would have to differentiate somehow anyway.

Stwerner 15y ago

I'll be adding the link to the site once I get home from work today, but I've set up a mailing list for anyone who might be interested in getting an email when new features are pushed out: http://eepurl.com/emtQU http://eepurl.com/emtQU

potomak 15y ago

it'd be cool to integrate it with http://smood.it http://smood.it