3 ms·
<< there should be an app like an 'auto-RAG' that scrapes RSS feeds and URLs, I am not aware if that exists yet, but the challenge I see with it is rather simp
by A4ET8a8uTh0 2y ago
<< there should be an app like an 'auto-RAG' that scrapes RSS feeds and URLs,
I am not aware if that exists yet, but the challenge I see with it is rather simple: you get overwhelmed with information really quickly. In other words, you would still need human somewhere in that process to review those scrapes and the quality of that varies widely. For example, even on HN it is not a given a link will be pure gold ( you still want to check if it fits your use case ).
That said, as ideas goes, it sounds like a fun weekend project.
- be_erik 2y agoI do exactly this with hoarder. I passively build tagged knowledge bases with the archived pages and then feed it to a RAG setup.
- swyx 2y agohttps://github.com/hoarder-app/hoarder https://github.com/hoarder-app/hoarder for the mention
- A4ET8a8uTh0 2y agoThank you. And I just found out, there is a docker image ready for Unraid. Lets see if it will work for me.
- fallinditch 2y agoCool. Hoarder looks interesting, thanks for the tip. How is it working out for you? Are you using the feature for auto hoarding RSS feeds?
- be_erik 2y agoI am! It works great and it’s reasonably easy to snapshot sites without RSS on a cron.