3 ms·
> let’s look at the labeling methodology described in the paper. To quote Section 3.3: > “Reddit comments were presented [to labelers] with no additional metad
by raverbashing 4y ago
> let’s look at the labeling methodology described in the paper. To quote Section 3.3:
> “Reddit comments were presented [to labelers] with no additional metadata (such as the author or subreddit).”
> “All raters are native English speakers from India.”
This does not look good even on paper. No wonder the errors were abundant
Also a labeling system that has no entry for sarcasm is totally going to work guys!!1 /s
- HWR_14 4y agoIsnt knowing the subreddit valuable information to determine sentiment?
- raverbashing 4y agocorrect, the article has some good examples about this But even without that, the context or just the topic of the discussion should help
- notahacker 4y agoYep. And familiarity with Reddit/subreddit memes and inside jokes. And entire subreddits devoted to parodying styles of comment which are utterly earnest on other reddits which I'm not sure how you'd even begin to classify...
- TremendousJudge 4y agoBut it also applies to the model. If you label data taking into account the context (which subreddit this was posted), your model also must take this into account, or it will be wrong as well. If the same sentence can be labeled two different ways depending on the source, then, the model must also know the source or it won't know what to do. But then you didn't create a general language model, you created a reddit language model.
- HWR_14 4y agoExcellent point.