6 ms·

Related to this, using: https://hnprofile.com/ https://hnprofile.com/ You can identify who is an expert related to each tool or any topic in general

weinzierl 8y ago
This is cool. After skimming a few profiles of folks I know and read mood detection is questionable but the "Topics of interest" is spot on.
- lettergram 8y ago
  For reference, this started as a "different kind of search engine", utilizing conversation. Mood wasn't exactly the focus, so it's robustness is limited - i.e. lots of room for improvement. The mood detection (based on my manual tagging) is roughly 80% accurate, which IMO was "good enough". It's more accurate as you get to the edges (very positive or very negative), the middle is kinda fuzzy, but that's kind of the nature of language.
dabber 8y ago
Ha, that's neet. I particularly like the "Probable Mood" addition.
- protomyth 8y ago
  I went from Happy to Unhappy while looking at it. I am so confused.
shawn 8y ago
The mood detection may as well be RNG. I have an even better way to identify who is an expert on HN. Say "It's silly that X still isn't a thing." If X is a thing, you'll get ten people telling you why you're wrong within ten minutes. Example: https://news.ycombinator.com/item?id=17941772 https://news.ycombinator.com/item?id=17941772 You'll have to trade some karma for the answer though.
- mattnewton 8y ago
  What else is karma for? :D I have seen this technique used to great effect inside large companies too. People will spend a lot of effort telling you why you are dead wrong.
  - mr_toad 8y ago
    https://www.xkcd.com/386/ https://www.xkcd.com/386/
- shawn 8y ago
  Just to raise the visibility of this: lettergram's HNProfile project is awesome. It identified both of my alt accounts: https://news.ycombinator.com/item?id=17944348 https://news.ycombinator.com/item?id=17944348
Kagerjay 8y ago
Well I had to go look up myself out of curiosity "youtube, everything, video, times, data, issues, software, experience, companies, point" Its not entirely wrong, but I don't know what "point", or "everything" means honestly
- cdubzzz 8y ago
  wife, example, data, service, github, phone, bit, sort, hah, area I should show this to my wife (:
- saagarjha 8y ago
  You're an expert at everything!
  - Kagerjay 8y ago
    haha I wish
warent 8y ago
The probable mood is very funny. > dang > Probable Mood: Tense The eternal life of an admin I suppose!
- yorwba 8y ago
  Just now it was "Happy". Probably because https://news.ycombinator.com/item?id=17944992 https://news.ycombinator.com/item?id=17944992 contains "grateful". Sentiment analysis is a finicky thing. (Maybe categories like "polite" or "argumentative" would be easier to detect?)
  - amelius 8y ago
    Conclusion: this user is emotionally not stable :D
protomyth 8y ago
Funny, I guess I am an expert in "heck". There is a certain rightness to that given the last couple of weeks. Is there a way to see the comments it deems 'happy'? [edit] I do wonder if the analysis of topics is affected by both the frequency something shows up on HN[1] and not repeating the keyword in a reply post. 1) less examples and more likely to miss a topic a person is expert on
- Fnoord 8y ago
  That's because it doesn't blacklist words which are commonly used in English without them being technical terms, nor doesn't it whitelist terms which are topics of interest. There's also no difference being made between referring to Wikipedia and being an expert about Wikipedia. If you regularly reference to Wikipedia (something I do) you then become an expert on the topic "wikipedia". As an example, check the expert topics from user tptacek [1] ("security, argument, fact, hn, thread, software, comment, story, google, sense"). The probable mood was rather... entertaining. The notion this can be used to find sockpuppets or throwaways does not surprise me. Without sufficient effort you still use certain words, in certain order, consistently. Such can even be used to check out what your native language is. So if you are e.g. a blackhat who wrote ransomware it makes sense to have someone who never wrote on the internet to paraphrase or deal with the communication, or perhaps use tools such as translators. Also, if you use delay (e.g. delay 10) it is going to appear on HN after you wrote it meaning it could appear in the next hour despite being written in the previous. Despite the above, I did find this an interesting project; I just very much doubt its accuracy and usefulness. [1] https://hnprofile.com/author_profiles?utf8=%E2%9C%93&search=user%3Atptacek https://hnprofile.com/author_profiles?utf8=%E2%9C%93&search=...
Waterluvian 8y ago
Can you share more about how the "mood" is determined? Definitely will be controversial and is deserving of a much more clear explanation. Why am I tense? Is it the sentences I formulate? Upvote vs. downvote patterns? RNG? I pretty strongly believe that you should take that piece rather seriously, even if you don't previously consider it a big deal. People generally don't like when a stranger tells the world what their mood will be. Especially bad when someone believes it's quite wrong. Also I love the opportunity for introspection. I'm trying to become a more whole, happy person who isn't defined whatsoever by his career. I would love to learn how some algorithms perceive me and why.
- lettergram 8y ago
  EDIT: Fixed the UI: I'm assuming you were looking at a topic, the "probable mood" used to render as it relates to the topic being searched, not current mood. Highly recommend reading the "how it works" section (I assume you already have): https://hnprofile.com/learn-more https://hnprofile.com/learn-more That section has a few examples and details. I'll add further details below. The system ONLY looks at comments sent to it. It, kind of looks like: { Author, Comment } From there, everything else is derived. Nothing else, no profiles, no voting, nothing. That's it. The system identifies: - Expertise - Knowledge - Interests - Relevant content - Promoter score (an improved net promoter score) - Trends - Mood - Related Topics - etc. To your question, there are multiple algorithms at work determining mood, most are pretty standard models. These are ones you can find in NLTK, spaCy, CoreNLP, etc. There's also a couple home-baked ones. These models than "vote" for the mood, with an averaged score. Typically these scores are 3 - 7 models voting at any one time. The aggregate score is roughly 80% accurate (based on my tags), and it tags the topics, sentence, and overall comment all separately. Finally, the models are much more accurate at the extremes: Very Positive and Very Negative, but in the middle, it's much fuzzier. This is in part, just how language works, it's subjective. All NLP models are based on a combination of open datasets with the tagged sentiment, as well as my own manually created dataset, and finally, a dataset that was manually created via looking at following comments (for instance, "no need to be negative", is used to tag the prior comment. The probable current mood is a prediction of your prior moods. I believe right now, it looks at the last time you discussed the given topic you searched (if you searched for a user, it is the most recent mood). If there is enough data, it'll also predict your mood, based on the time of day + topic being discussed. Finally, I should add - there are definitely improvements to be made, and I'm currently working on mood. Primarily, the focus was on expertise, knowledge, and interests. That is what enables the "related content" section and is now very accurate (given a decent number of data samples). Mood, as stated, is ~80% accurate based on my tags. I think I can do better (probably >90%), especially with more labeled data for this particular dataset. The goal of this is it can drop into any company and instantly can search for who and what is relevant to your company.
tonyedgecombe 8y ago
How do I opt out, I don't expect such an invasion of my privacy from posting on HN.
- mwwilson 8y ago
  The opt out currently, for privacy moving forward, is to cease contribution to the digital noosphere. I suppose if you desired you could consolidate your doxx point to Google and ISP's by running all your comments through a couple rounds of Google translate. There is no privacy possible in the global village. Expectation of such is technically naive.
xtrapolate 8y ago
I see there's no privacy policy, and no clear guidelines on opting out. Why? Privacy was the focal point of 2018, but you're somehow ignoring that altogether?
caseymarquis 8y ago
This is neat! But, it seems a bit disingenuous to say I'm looking at experts, as opposed to people who like to talk about something. My expertise is industrial automation and connectivity for example, but I come up when searching for vue, a javascript framework, just because it's a tool I like to promote since I think it could use some noise given react's popularity. Step 2 seems like having someone or something reading through comments and assigning expert status to profiles.
leejo 8y ago
Commenting here to boost my mood, or at least see how this works. I'm super happy at the moment, woohoo! Happy. Happy. Happy...

weinzierl 8y ago

This is cool. After skimming a few profiles of folks I know and read mood detection is questionable but the "Topics of interest" is spot on.

lettergram 8y ago
For reference, this started as a "different kind of search engine", utilizing conversation. Mood wasn't exactly the focus, so it's robustness is limited - i.e. lots of room for improvement. The mood detection (based on my manual tagging) is roughly 80% accurate, which IMO was "good enough". It's more accurate as you get to the edges (very positive or very negative), the middle is kinda fuzzy, but that's kind of the nature of language.

dabber 8y ago

Ha, that's neet. I particularly like the "Probable Mood" addition.

protomyth 8y ago
I went from Happy to Unhappy while looking at it. I am so confused.

shawn 8y ago

The mood detection may as well be RNG. I have an even better way to identify who is an expert on HN. Say "It's silly that X still isn't a thing." If X is a thing, you'll get ten people telling you why you're wrong within ten minutes. Example: https://news.ycombinator.com/item?id=17941772 https://news.ycombinator.com/item?id=17941772 You'll have to trade some karma for the answer though.

mattnewton 8y ago
What else is karma for? :D I have seen this technique used to great effect inside large companies too. People will spend a lot of effort telling you why you are dead wrong.
- mr_toad 8y ago
  https://www.xkcd.com/386/ https://www.xkcd.com/386/
shawn 8y ago
Just to raise the visibility of this: lettergram's HNProfile project is awesome. It identified both of my alt accounts: https://news.ycombinator.com/item?id=17944348 https://news.ycombinator.com/item?id=17944348

Kagerjay 8y ago

Well I had to go look up myself out of curiosity "youtube, everything, video, times, data, issues, software, experience, companies, point" Its not entirely wrong, but I don't know what "point", or "everything" means honestly

cdubzzz 8y ago
wife, example, data, service, github, phone, bit, sort, hah, area I should show this to my wife (:
saagarjha 8y ago
You're an expert at everything!
- Kagerjay 8y ago
  haha I wish

warent 8y ago

The probable mood is very funny. > dang > Probable Mood: Tense The eternal life of an admin I suppose!

yorwba 8y ago
Just now it was "Happy". Probably because https://news.ycombinator.com/item?id=17944992 https://news.ycombinator.com/item?id=17944992 contains "grateful". Sentiment analysis is a finicky thing. (Maybe categories like "polite" or "argumentative" would be easier to detect?)
- amelius 8y ago
  Conclusion: this user is emotionally not stable :D

protomyth 8y ago

Funny, I guess I am an expert in "heck". There is a certain rightness to that given the last couple of weeks. Is there a way to see the comments it deems 'happy'? [edit] I do wonder if the analysis of topics is affected by both the frequency something shows up on HN[1] and not repeating the keyword in a reply post. 1) less examples and more likely to miss a topic a person is expert on

Fnoord 8y ago
That's because it doesn't blacklist words which are commonly used in English without them being technical terms, nor doesn't it whitelist terms which are topics of interest. There's also no difference being made between referring to Wikipedia and being an expert about Wikipedia. If you regularly reference to Wikipedia (something I do) you then become an expert on the topic "wikipedia". As an example, check the expert topics from user tptacek [1] ("security, argument, fact, hn, thread, software, comment, story, google, sense"). The probable mood was rather... entertaining. The notion this can be used to find sockpuppets or throwaways does not surprise me. Without sufficient effort you still use certain words, in certain order, consistently. Such can even be used to check out what your native language is. So if you are e.g. a blackhat who wrote ransomware it makes sense to have someone who never wrote on the internet to paraphrase or deal with the communication, or perhaps use tools such as translators. Also, if you use delay (e.g. delay 10) it is going to appear on HN after you wrote it meaning it could appear in the next hour despite being written in the previous. Despite the above, I did find this an interesting project; I just very much doubt its accuracy and usefulness. [1] https://hnprofile.com/author_profiles?utf8=%E2%9C%93&search=user%3Atptacek https://hnprofile.com/author_profiles?utf8=%E2%9C%93&search=...

Waterluvian 8y ago

Can you share more about how the "mood" is determined? Definitely will be controversial and is deserving of a much more clear explanation. Why am I tense? Is it the sentences I formulate? Upvote vs. downvote patterns? RNG? I pretty strongly believe that you should take that piece rather seriously, even if you don't previously consider it a big deal. People generally don't like when a stranger tells the world what their mood will be. Especially bad when someone believes it's quite wrong. Also I love the opportunity for introspection. I'm trying to become a more whole, happy person who isn't defined whatsoever by his career. I would love to learn how some algorithms perceive me and why.

lettergram 8y ago
EDIT: Fixed the UI: I'm assuming you were looking at a topic, the "probable mood" used to render as it relates to the topic being searched, not current mood. Highly recommend reading the "how it works" section (I assume you already have): https://hnprofile.com/learn-more https://hnprofile.com/learn-more That section has a few examples and details. I'll add further details below. The system ONLY looks at comments sent to it. It, kind of looks like: { Author, Comment } From there, everything else is derived. Nothing else, no profiles, no voting, nothing. That's it. The system identifies: - Expertise - Knowledge - Interests - Relevant content - Promoter score (an improved net promoter score) - Trends - Mood - Related Topics - etc. To your question, there are multiple algorithms at work determining mood, most are pretty standard models. These are ones you can find in NLTK, spaCy, CoreNLP, etc. There's also a couple home-baked ones. These models than "vote" for the mood, with an averaged score. Typically these scores are 3 - 7 models voting at any one time. The aggregate score is roughly 80% accurate (based on my tags), and it tags the topics, sentence, and overall comment all separately. Finally, the models are much more accurate at the extremes: Very Positive and Very Negative, but in the middle, it's much fuzzier. This is in part, just how language works, it's subjective. All NLP models are based on a combination of open datasets with the tagged sentiment, as well as my own manually created dataset, and finally, a dataset that was manually created via looking at following comments (for instance, "no need to be negative", is used to tag the prior comment. The probable current mood is a prediction of your prior moods. I believe right now, it looks at the last time you discussed the given topic you searched (if you searched for a user, it is the most recent mood). If there is enough data, it'll also predict your mood, based on the time of day + topic being discussed. Finally, I should add - there are definitely improvements to be made, and I'm currently working on mood. Primarily, the focus was on expertise, knowledge, and interests. That is what enables the "related content" section and is now very accurate (given a decent number of data samples). Mood, as stated, is ~80% accurate based on my tags. I think I can do better (probably >90%), especially with more labeled data for this particular dataset. The goal of this is it can drop into any company and instantly can search for who and what is relevant to your company.

tonyedgecombe 8y ago

How do I opt out, I don't expect such an invasion of my privacy from posting on HN.

mwwilson 8y ago
The opt out currently, for privacy moving forward, is to cease contribution to the digital noosphere. I suppose if you desired you could consolidate your doxx point to Google and ISP's by running all your comments through a couple rounds of Google translate. There is no privacy possible in the global village. Expectation of such is technically naive.

xtrapolate 8y ago

I see there's no privacy policy, and no clear guidelines on opting out. Why? Privacy was the focal point of 2018, but you're somehow ignoring that altogether?

caseymarquis 8y ago

This is neat! But, it seems a bit disingenuous to say I'm looking at experts, as opposed to people who like to talk about something. My expertise is industrial automation and connectivity for example, but I come up when searching for vue, a javascript framework, just because it's a tool I like to promote since I think it could use some noise given react's popularity. Step 2 seems like having someone or something reading through comments and assigning expert status to profiles.

leejo 8y ago

Commenting here to boost my mood, or at least see how this works. I'm super happy at the moment, woohoo! Happy. Happy. Happy...