5 ms·

Thanks! I used Huggingface's pretrained BERT.

jedwhite 5y ago
This is a really good application of it. Getting NER right for something like book titles with so much name collision with other domains and entity types is really hard, and this works great on something that most people would never realize would be so hard!
sillysaurusx 5y ago
Please write up how you did this! It may seem easy or straightforward, but I assure you it's black magic to a lot of people.
- stanislavb 5y ago
  That'd be great.
- tracyhenry 5y ago
  You can refer to the medium post in this comment, which does more or less what I did: https://news.ycombinator.com/item?id=28601741 https://news.ycombinator.com/item?id=28601741
malshe 5y ago
This is really impressive! Can you please elaborate more on the way you labeled the data? I think usually there is a lot to learn from labeling methods.
- tracyhenry 5y ago
  I generated training comments by matching book names. Roughly, there are one in five of those comments that actually have a book mention. Then I use the Doccano labeling tool to label the tokens in the comments.
  - malshe 5y ago
    Thanks! I will check out Doccano now
m3at 5y ago
Thanks for building this, it's useful and well presented! Replies to your pinned top comment seems disabled, so allow me to ask here (sorry for the hijack): > used a custom NLP-based ranking function to sort the comments Can you expand on this function? (I'm familiar with NLP and most SOTA models)
- tracyhenry 5y ago
  Ah right now ranking is really rudimentary: I just use TextBlob's SA library to get a sentiment score for the comment, and combine it with the comment length. In theory we can also use BERT to get a sentiment score, which should be more accurate I guess? Love to hear your thoughts.
  - m3at 5y ago
    Simple is good! Especially for ranking as the objective is hard to define. Having looked at some samples on your site, imo it's good enough :) Though if you wanted to try other things just for fun, maybe: - Count matching NER, comments with a lot of book recommendations tend not to detail why they like the specific one currently filtered - maybe down weight comments that are too short (after the matched NER is subtracted) as they seem to just have a title+link? Not ML but while I'm here, on Android the comment section appear fine but has an horizontal scroll that seems spurious, with lots of blank space to the right (Galaxy S10, chrome & firefox)

This is a really good application of it. Getting NER right for something like book titles with so much name collision with other domains and entity types is really hard, and this works great on something that most people would never realize would be so hard!

sillysaurusx 5y ago

Please write up how you did this! It may seem easy or straightforward, but I assure you it's black magic to a lot of people.

stanislavb 5y ago
That'd be great.
tracyhenry 5y ago
You can refer to the medium post in this comment, which does more or less what I did: https://news.ycombinator.com/item?id=28601741 https://news.ycombinator.com/item?id=28601741

malshe 5y ago

This is really impressive! Can you please elaborate more on the way you labeled the data? I think usually there is a lot to learn from labeling methods.

tracyhenry 5y ago
I generated training comments by matching book names. Roughly, there are one in five of those comments that actually have a book mention. Then I use the Doccano labeling tool to label the tokens in the comments.
- malshe 5y ago
  Thanks! I will check out Doccano now

m3at 5y ago

Thanks for building this, it's useful and well presented! Replies to your pinned top comment seems disabled, so allow me to ask here (sorry for the hijack): > used a custom NLP-based ranking function to sort the comments Can you expand on this function? (I'm familiar with NLP and most SOTA models)

tracyhenry 5y ago
Ah right now ranking is really rudimentary: I just use TextBlob's SA library to get a sentiment score for the comment, and combine it with the comment length. In theory we can also use BERT to get a sentiment score, which should be more accurate I guess? Love to hear your thoughts.
- m3at 5y ago
  Simple is good! Especially for ranking as the objective is hard to define. Having looked at some samples on your site, imo it's good enough :) Though if you wanted to try other things just for fun, maybe: - Count matching NER, comments with a lot of book recommendations tend not to detail why they like the specific one currently filtered - maybe down weight comments that are too short (after the matched NER is subtracted) as they seem to just have a title+link? Not ML but while I'm here, on Android the comment section appear fine but has an horizontal scroll that seems spurious, with lots of blank space to the right (Galaxy S10, chrome & firefox)