5 ms·

RSS-Bridge – The RSS feed for websites missing it

195 points by detaro 5y ago

jcgoette 5y ago
https://news.ycombinator.com/item?id=27712657 https://news.ycombinator.com/item?id=27712657 https://news.ycombinator.com/item?id=19044295 https://news.ycombinator.com/item?id=19044295
karlicoss 5y ago
Wonder what's are the differences from https://github.com/DIYgod/RSSHub https://github.com/DIYgod/RSSHub and https://github.com/kickscondor/fraidycat https://github.com/kickscondor/fraidycat I guess projects might have different goals in mind, but IMO would be nice to share the scrapers among the projects
- noizejoy 5y ago
  I actually prefer a world with multiple alternatives rather than single instances. Life itself works that way and it has proven to make life very robust, even when individual life-forms are very fragile.
thirdplace_ 5y ago
Some rss feeds only include a snippet of the entire item text. One nice thing about rss-bridge is that bridges can expand an existing feed to include the entire item content. In essence the project is a scraper that generates web feeds.
- kartoshechka 5y ago
  ttrss has (an official?) plugin to do that (fetch full text from title only RSS feeds), and I'm sure other feed readers can do it from the box or via plugins
  - wfriesen 5y ago
    The official plugin uses a php port of Mozilla's Readability, which is used for Firefox Reader Mode. There is also the 3rd party FeedIron that is more configurable. https://github.com/feediron/ttrss_plugin-feediron https://github.com/feediron/ttrss_plugin-feediron
sitta 5y ago
Unfortunately, the Facebook bridge is very broken and no longer maintained. From the public pages I've tried, it only intermittently works on at most half of them. I don't blame them for not maintaining it though. I grabbed the code to see if I could get it working and realized Facebook has made their newer pages fiendishly difficult to scrape. Assuming one does put in the effort to make it work, how long will it last till it's broken again? I wish there was a viable alternative platform for organizations where "public" content is actually accessible publicly.
- trog 5y ago
  I have been wondering if, instead of bothering to try to scrape Facebook via DOM, it might be worth pursuing a basic image matching approach to divide a Facebook page up into regular "chunks" that are just saved as images. I'd personally be happy with an RSS feed of Facebook as just a series of images in a feed, but I guess once you've gone that far you could relatively simply run it through OCR to get most of the text.
  - Karrot_Kream 5y ago
    I've toyed around with scraping Facebook through a headless chrome myself but haven't done it yet. Have you taken a look at https://mbasic.facebook.com https://mbasic.facebook.com ? It might be simpler to parse.
- tfsh 5y ago
  Unfortunately the RSS bridge doesn't work and also seems to be unmaintained, this is surprising given it seems to be set as one of the "core" bridges. Data archival, retention and accessibility is absolutely fundamental and it's unfortunate to know that so many companies are hell bent on stopping individuals from using open-means to access such data (although I am not surprised, and from a rational perspective I can understand their reasons for making it difficult).
- doom277 5y ago
  > Unfortunately, the Facebook bridge is very broken and no longer maintained. True. There is no maintainer for FacebookBridge. As for now, the only way to fix FacebookBridge that I see is: 1. using existing Facebook account 2. fetch pages using Selenium (simply speaking - web browser) This is mentioned in https://github.com/RSS-Bridge/rss-bridge/issues/2400 https://github.com/RSS-Bridge/rss-bridge/issues/2400
hinkley 5y ago
I feel guilty running something like this on my own, but this seems like an excellent tool for a news aggregator to run.
doom277 5y ago
Hi, RSS-Bridge maintainer here. Feel free to ask questions.
- doom277 5y ago
  Prooflink: https://github.com/em92/rss-bridge-maintainer-reports/commit/5f5b45007ce64430205943d4ce76685dfceb5c3f https://github.com/em92/rss-bridge-maintainer-reports/commit... https://github.com/orgs/RSS-Bridge/people https://github.com/orgs/RSS-Bridge/people
- mhsomethingsout 5y ago
  Thank you Eugene for your work, I really like these kind of projects for a open web! I have a question: do you have a list of hosted instances, as Invidious has [0], or are you planning to do it in the future? So people like me can use it without opening their own instance? [0]: https://docs.invidious.io/Invidious-Instances.md https://docs.invidious.io/Invidious-Instances.md
  - thirdplace_ 5y ago
    Here is a list of public instances: https://github.com/RSS-Bridge/rss-bridge/wiki/Public-hosts https://github.com/RSS-Bridge/rss-bridge/wiki/Public-hosts
    - mhsomethingsout 5y ago
      Oh, thanks! I took a brief look at the wiki but did not see that!
- yellowkard 5y ago
  Great app, congratulations!
anthropodie 5y ago
The best part is at the bottom of that page >>> Dear so-called "social" websites. Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. FUCK YOU. You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds. We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have your service with your applications using your API force-feeding them. Friends must be free to choose whatever software and service they want. We are rebuilding bridges you have willfully destroyed. Get your shit together: Put RSS/Atom back in. <<< Not gonna happen but I fully agree with the sentiment.
- sebsauvage 5y ago
  Thanks (spoiler: I'm the author of this text. I'm the author of the first version of RSS-Bridge, which has since been beautifuly expanded by the community).
Vinnl 5y ago
Back when I showed my tool to generate RSS feeds [0], someone recommended a extensive list of similar tools [1]. In case this doesn't exactly meet your needs (e.g. needing to self-host), you might find a useful alternative there. [0] Show HN: RSS feeds for arbitrary websites using CSS selectors https://news.ycombinator.com/item?id=27739568 https://news.ycombinator.com/item?id=27739568 [1] All About RSS - RSS Feed Generation https://github.com/AboutRSS/ALL-about-RSS#-rss-feed-generation https://github.com/AboutRSS/ALL-about-RSS#-rss-feed-generati...

195 points by detaro 5y ago

https://news.ycombinator.com/item?id=27712657 https://news.ycombinator.com/item?id=27712657 https://news.ycombinator.com/item?id=19044295 https://news.ycombinator.com/item?id=19044295

karlicoss 5y ago

Wonder what's are the differences from https://github.com/DIYgod/RSSHub https://github.com/DIYgod/RSSHub and https://github.com/kickscondor/fraidycat https://github.com/kickscondor/fraidycat I guess projects might have different goals in mind, but IMO would be nice to share the scrapers among the projects

noizejoy 5y ago
I actually prefer a world with multiple alternatives rather than single instances. Life itself works that way and it has proven to make life very robust, even when individual life-forms are very fragile.

thirdplace_ 5y ago

Some rss feeds only include a snippet of the entire item text. One nice thing about rss-bridge is that bridges can expand an existing feed to include the entire item content. In essence the project is a scraper that generates web feeds.

kartoshechka 5y ago
ttrss has (an official?) plugin to do that (fetch full text from title only RSS feeds), and I'm sure other feed readers can do it from the box or via plugins
- wfriesen 5y ago
  The official plugin uses a php port of Mozilla's Readability, which is used for Firefox Reader Mode. There is also the 3rd party FeedIron that is more configurable. https://github.com/feediron/ttrss_plugin-feediron https://github.com/feediron/ttrss_plugin-feediron

sitta 5y ago

Unfortunately, the Facebook bridge is very broken and no longer maintained. From the public pages I've tried, it only intermittently works on at most half of them. I don't blame them for not maintaining it though. I grabbed the code to see if I could get it working and realized Facebook has made their newer pages fiendishly difficult to scrape. Assuming one does put in the effort to make it work, how long will it last till it's broken again? I wish there was a viable alternative platform for organizations where "public" content is actually accessible publicly.

trog 5y ago
I have been wondering if, instead of bothering to try to scrape Facebook via DOM, it might be worth pursuing a basic image matching approach to divide a Facebook page up into regular "chunks" that are just saved as images. I'd personally be happy with an RSS feed of Facebook as just a series of images in a feed, but I guess once you've gone that far you could relatively simply run it through OCR to get most of the text.
- Karrot_Kream 5y ago
  I've toyed around with scraping Facebook through a headless chrome myself but haven't done it yet. Have you taken a look at https://mbasic.facebook.com https://mbasic.facebook.com ? It might be simpler to parse.
tfsh 5y ago
Unfortunately the RSS bridge doesn't work and also seems to be unmaintained, this is surprising given it seems to be set as one of the "core" bridges. Data archival, retention and accessibility is absolutely fundamental and it's unfortunate to know that so many companies are hell bent on stopping individuals from using open-means to access such data (although I am not surprised, and from a rational perspective I can understand their reasons for making it difficult).
doom277 5y ago
> Unfortunately, the Facebook bridge is very broken and no longer maintained. True. There is no maintainer for FacebookBridge. As for now, the only way to fix FacebookBridge that I see is: 1. using existing Facebook account 2. fetch pages using Selenium (simply speaking - web browser) This is mentioned in https://github.com/RSS-Bridge/rss-bridge/issues/2400 https://github.com/RSS-Bridge/rss-bridge/issues/2400

hinkley 5y ago

I feel guilty running something like this on my own, but this seems like an excellent tool for a news aggregator to run.

doom277 5y ago

Hi, RSS-Bridge maintainer here. Feel free to ask questions.

doom277 5y ago
Prooflink: https://github.com/em92/rss-bridge-maintainer-reports/commit/5f5b45007ce64430205943d4ce76685dfceb5c3f https://github.com/em92/rss-bridge-maintainer-reports/commit... https://github.com/orgs/RSS-Bridge/people https://github.com/orgs/RSS-Bridge/people
mhsomethingsout 5y ago
Thank you Eugene for your work, I really like these kind of projects for a open web! I have a question: do you have a list of hosted instances, as Invidious has [0], or are you planning to do it in the future? So people like me can use it without opening their own instance? [0]: https://docs.invidious.io/Invidious-Instances.md https://docs.invidious.io/Invidious-Instances.md
- thirdplace_ 5y ago
  Here is a list of public instances: https://github.com/RSS-Bridge/rss-bridge/wiki/Public-hosts https://github.com/RSS-Bridge/rss-bridge/wiki/Public-hosts
  - mhsomethingsout 5y ago
    Oh, thanks! I took a brief look at the wiki but did not see that!
yellowkard 5y ago
Great app, congratulations!

anthropodie 5y ago

The best part is at the bottom of that page >>> Dear so-called "social" websites. Your catchword is "share", but you don't want us to share. You want to keep us within your walled gardens. That's why you've been removing RSS links from webpages, hiding them deep on your website, or removed feeds entirely, replacing it with crippled or demented proprietary API. FUCK YOU. You're not social when you hamper sharing by removing feeds. You're happy to have customers creating content for your ecosystem, but you don't want this content out - a content you do not even own. Google Takeout is just a gimmick. We want our data to flow, we want RSS or Atom feeds. We want to share with friends, using open protocols: RSS, Atom, XMPP, whatever. Because no one wants to have your service with your applications using your API force-feeding them. Friends must be free to choose whatever software and service they want. We are rebuilding bridges you have willfully destroyed. Get your shit together: Put RSS/Atom back in. <<< Not gonna happen but I fully agree with the sentiment.

sebsauvage 5y ago
Thanks (spoiler: I'm the author of this text. I'm the author of the first version of RSS-Bridge, which has since been beautifuly expanded by the community).

Vinnl 5y ago

Back when I showed my tool to generate RSS feeds [0], someone recommended a extensive list of similar tools [1]. In case this doesn't exactly meet your needs (e.g. needing to self-host), you might find a useful alternative there. [0] Show HN: RSS feeds for arbitrary websites using CSS selectors https://news.ycombinator.com/item?id=27739568 https://news.ycombinator.com/item?id=27739568 [1] All About RSS - RSS Feed Generation https://github.com/AboutRSS/ALL-about-RSS#-rss-feed-generation https://github.com/AboutRSS/ALL-about-RSS#-rss-feed-generati...