3 ms·

I'd love a RSS reader that groups items on the same topic. For example, let's say I have feeds from 25 news sources and the President gives a speech on Ukraine

I'd love a RSS reader that groups items on the same topic. For example, let's say I have feeds from 25 news sources and the President gives a speech on Ukraine. I end up with 25 articles on it, spread across my item list; I need to process it 25 times. IME, most items are duplicated at least once - half my items are not needed. If the feed reader would group all the Biden speech articles then I could process it once - pick the item I want to read, delete the rest. That's also more efficient because it's easier to compare the items. Online news aggregators like Google News already do that, so the technology exists, but I haven't seen it in a feed reader. I also wonder if the LLMs could group them more accurately.

nbbaier 3y ago
I bet language models could definitely help here, yeah. Perhaps something like (1) get content offeed items as they come in, (2) embed the content, (3) use those embeddings to group items. Probably not that difficult to be honest
- gwern 3y ago
  Yep. That would be a classic sort of k-means problem. Just throw them all into a standard embedding, like the OA API embeddings, run k-means from sci-kit, then convert them into a list-of-lists: one RSS item (containing a list of title-URLs) per cluster.
  - woodglyst 3y ago
    The problem with this approach is determining the what k is for the k-means. But again, we could use the “elbow” technique to determine what’s the optimal k and then start grouping them together. I wonder if there are any automatic sophisticated clustering algorithms?
    - memhole 3y ago
      Hierarchical and DBSCAN don’t require upfront knowledge about the number of clusters.
gumby 3y ago
I used to use an RSS reader that had "smart folders" (filters) so I could group cross feeds and across servers but the author seems to be too busy to keep it going.
nreece 3y ago
>> I'd love a RSS reader that groups items on the same topic. We had the same concern with everyday industry news, and so did some of our customers, so we built it a while back! Our Newsroom feed reader uses NLP and ML to group similar articles for low-noise research/reading: https://newsloth.com/#features https://newsloth.com/#features Currently, it detects and groups articles from a few hundred popular global news sites, and gradually growing to cover more sources. We also have a feed builder/generator that turns unstructured public webpages into structured RSS feeds, to make it a flexible and unified feeds platform: https://newsloth.com/product/feed-builder https://newsloth.com/product/feed-builder
- wolverine876 3y ago
  That's great! I will take a look.
Gormo 3y ago
I've been using Liferea [1] for years, and while it doesn't do automated content analysis, it does allow you to group feeds into folders, then get an aggregated view of the folder as a whole. It also allows "search folders", which are saved cross-feed searches that are exposed through the same folder interface. I use Liferea as my desktop client, but with TT-RSS running on my VPS as the underlying data source. [1]: https://lzone.de/liferea/ https://lzone.de/liferea/ https://github.com/lwindolf/liferea https://github.com/lwindolf/liferea

nbbaier 3y ago

I bet language models could definitely help here, yeah. Perhaps something like (1) get content offeed items as they come in, (2) embed the content, (3) use those embeddings to group items. Probably not that difficult to be honest

gwern 3y ago
Yep. That would be a classic sort of k-means problem. Just throw them all into a standard embedding, like the OA API embeddings, run k-means from sci-kit, then convert them into a list-of-lists: one RSS item (containing a list of title-URLs) per cluster.
- woodglyst 3y ago
  The problem with this approach is determining the what k is for the k-means. But again, we could use the “elbow” technique to determine what’s the optimal k and then start grouping them together. I wonder if there are any automatic sophisticated clustering algorithms?
  - memhole 3y ago
    Hierarchical and DBSCAN don’t require upfront knowledge about the number of clusters.

gumby 3y ago

I used to use an RSS reader that had "smart folders" (filters) so I could group cross feeds and across servers but the author seems to be too busy to keep it going.

nreece 3y ago

>> I'd love a RSS reader that groups items on the same topic. We had the same concern with everyday industry news, and so did some of our customers, so we built it a while back! Our Newsroom feed reader uses NLP and ML to group similar articles for low-noise research/reading: https://newsloth.com/#features https://newsloth.com/#features Currently, it detects and groups articles from a few hundred popular global news sites, and gradually growing to cover more sources. We also have a feed builder/generator that turns unstructured public webpages into structured RSS feeds, to make it a flexible and unified feeds platform: https://newsloth.com/product/feed-builder https://newsloth.com/product/feed-builder

wolverine876 3y ago
That's great! I will take a look.

Gormo 3y ago

I've been using Liferea [1] for years, and while it doesn't do automated content analysis, it does allow you to group feeds into folders, then get an aggregated view of the folder as a whole. It also allows "search folders", which are saved cross-feed searches that are exposed through the same folder interface. I use Liferea as my desktop client, but with TT-RSS running on my VPS as the underlying data source. [1]: https://lzone.de/liferea/ https://lzone.de/liferea/ https://github.com/lwindolf/liferea https://github.com/lwindolf/liferea