4 ms·

You could probably use NRC Emotion Lexicon[1] to categorize the titles into different emotions and sort it like that, as well and give it some normalized emotio

You could probably use NRC Emotion Lexicon[1] to categorize the titles into different emotions and sort it like that, as well and give it some normalized emotional score/category for X. Then use logistic regression for Y being before/after lockdown or linear regression for Y being (current_date - lockdown_date). The linear regression will assume that things get linearly worse over time (it can be smoothed out better probably). 1: https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm#:~:text=NRC%20Word%2DEmotion%20Association%20Lexicon,were%20manually%20done%20by%20crowdsourcing https://saifmohammad.com/WebPages/NRC-Emotion-Lexicon.htm#:~....

sunnysofa 6y ago
> You could probably use NRC Emotion Lexicon Thanks and good idea! How does this compare to some pre-trained ML model for sentiments analysis? > Then use logistic regression for Y being before/after lockdown ... Just googled logistic regression but would love to hear some examples in this context to get a better grasp on this. Thanks!
- avindroth 6y ago
  I would think you could use the pre-trained ML model and achieve the same result. If you really want to hack away at it, the Machine Learning course at Stanford has really good notes. http://cs229.stanford.edu/syllabus-fall2020.html http://cs229.stanford.edu/syllabus-fall2020.html I just thought of logistic regression because you are doing binary classification (is the title before or after the outbreak?) and it is the simplest (and most general) way to do it. But I think it would be more interesting to do linear regression (how long has it been since lockdown?) which would analyze how the sentiments change relative to the outbreak. So maybe take the time period between -2 months and +6 months relative to lockdown and then see what role time plays in the sentiment. You could easily extend this to newspapers and what not, I am sure people have tried this before! You can conceptually change the problem of "how did breakout affect HN titles" to "given some HN title, predict whether it's before or after breakout".

sunnysofa 6y ago

> You could probably use NRC Emotion Lexicon Thanks and good idea! How does this compare to some pre-trained ML model for sentiments analysis? > Then use logistic regression for Y being before/after lockdown ... Just googled logistic regression but would love to hear some examples in this context to get a better grasp on this. Thanks!

avindroth 6y ago
I would think you could use the pre-trained ML model and achieve the same result. If you really want to hack away at it, the Machine Learning course at Stanford has really good notes. http://cs229.stanford.edu/syllabus-fall2020.html http://cs229.stanford.edu/syllabus-fall2020.html I just thought of logistic regression because you are doing binary classification (is the title before or after the outbreak?) and it is the simplest (and most general) way to do it. But I think it would be more interesting to do linear regression (how long has it been since lockdown?) which would analyze how the sentiments change relative to the outbreak. So maybe take the time period between -2 months and +6 months relative to lockdown and then see what role time plays in the sentiment. You could easily extend this to newspapers and what not, I am sure people have tried this before! You can conceptually change the problem of "how did breakout affect HN titles" to "given some HN title, predict whether it's before or after breakout".