2 ms·
Dupe of https://news.ycombinator.com/item?id=34969623 https://news.ycombinator.com/item?id=34969623
by gwbas1c 4y ago
Dupe of https://news.ycombinator.com/item?id=34969623 https://news.ycombinator.com/item?id=34969623
- PaulHoule 4y agoI submitted one even before that.. Without the annoying popup this one has. Pro Tip: I have been looking for way to cluster multiple copies of articles on the same subject in RSS feeds and similar things for a long time. That was a critical function Google News had from the very beginning, without it you would just be spammed with 200 me-too articles for every event. I found LDA -> Spectral Clustering was great for creating topical clusters (separate out "reinforcement learning" from other areas of "deep learning" and sports from "the decline and fall of the UK") until I hit 50,000 documents and I started running out of RAM. I switched to k-Means and it still seems to work OK. I just found out that MiniLM -> DBSCAN does an amazingly good job of collapsing news articles about the same event, it even works pretty well on HN headlines.
- dang 4y agoLooks like https://news.ycombinator.com/item?id=34962145 https://news.ycombinator.com/item?id=34962145 was first, so we merged the threads thither.