3 ms·
It mainly uses RSS feeds versus HTML scraping, but there are a couple of news sources where HTML is required.
by sterban 9y ago
It mainly uses RSS feeds versus HTML scraping, but there are a couple of news sources where HTML is required.
- e15ctr0n 9y agoYou wrote elsewhere that your output HTML is highly optimized. I viewed the source of the page and I can see that it contains the CSS inside <style> tags and images embedded as base64 encoded text - what software do you use to generate this output?
- sterban 9y agoThe python scraper base64 encodes all the images, so that data is prepared for the web server. The python library being used is PIL. The CSS is hand-written in a Go template. The goal was to have all the rendering done in a single request, so I choose to embed the CSS and relax the CSP on styles.
- e15ctr0n 9y agoThank you for sharing. This is seriously cool stuff. Please write all this up in a blog post and do a Show HN. :-)