3 ms·
Articles given via RSS feeds usually don't contain ads, right? Might it be possible to use content with no ads from RSS and the same content from the home webpa
by bhl 7y ago
Articles given via RSS feeds usually don't contain ads, right? Might it be possible to use content with no ads from RSS and the same content from the home webpage with ads to build a text extraction model that works beyond RSS?
- gumby 7y agoThat is typically not the case. Some are just a headline or a headline and a short paragraph of text (the most common). Some are the full text without HTML (thus no ad links -- mostly indie blog pages). Some are the actual page.
- imduffy15 7y agoThere are AI solutions that can work on the HTML and manage to avoid the ads. Checkout https://blog.scrapinghub.com/extracting-clean-article-html-with-autoextract https://blog.scrapinghub.com/extracting-clean-article-html-w...