5 ms·

Looks very cool! Could you explain the tech that you developed ? Did you create a new algorithm for extracting the content from the page or used an existing one

Looks very cool! Could you explain the tech that you developed ? Did you create a new algorithm for extracting the content from the page or used an existing one ?

k1m 9y ago
Can be broken down to roughly these steps: 1. The tool for extracting article content is Full-Text RSS[0]. It relies on arc90's original (open source) article detection algorithm, Readability, as well as a set of site-specific extraction rules which we maintain on Github[1]. 2. Our PDF Newspaper[2] application then cleans that extracted article output to normalise HTML/CSS styles so we can present the output in our own layout without things breaking. We use CSS for multi-column output. 3. Finally, we use Chromium's new headless API to request the generated output in step 2 and create three PDFs - injecting CSS to increase the font size each time. After we have the three PDF files, we check the PDFs in order of smallest to largest font size and as soon as a larger font size creates an extra new page, we discard that and keep the PDF we had before and return that. So if PDF 1 (small font size) has three pages, PDF 2 (medium font size) also three pages, but PDF 3 (larger font size) four pages, then we return PDF 2. [0] http://fivefilters.org/content-only/ http://fivefilters.org/content-only/ [1] https://github.com/fivefilters/ftr-site-config https://github.com/fivefilters/ftr-site-config [2] http://fivefilters.org/pdf-newspaper/ http://fivefilters.org/pdf-newspaper/
- roadbeats 9y ago
  cool stuff! I tried it in my blog and the result was excellent.
- gravypod 9y ago
  Does this support images? I tried to convert Ernest P. Chan's blog [0] into a pdf using this and the results were not well formatted. [0] - http://epchan.blogspot.com/ http://epchan.blogspot.com/
  - k1m 9y ago
    We'll add images soon. As for the blog post, should work if you use the article URL: http://pdf.fivefilters.org/simple-print/url.php?size=A4#http://epchan.blogspot.se/2018/02/fx-order-flow-as-predictor.html http://pdf.fivefilters.org/simple-print/url.php?size=A4#http...
    - everydayavi 9y ago
      Timeline for images? Well done!
- pbhjpbhj 9y ago
  Step two seems to be almost certainly copyright infringement in the UK, not sure about USA exactly but as you take money and then copy the content it seems likely to breech the fair use regulations. Has this been addressed? Do you have an article on it or a position statement?

Looks very cool! Could you explain the tech that you developed ? Did you create a new algorithm for extracting the content from the page or used an existing one ?

k1m 9y ago

Can be broken down to roughly these steps: 1. The tool for extracting article content is Full-Text RSS[0]. It relies on arc90's original (open source) article detection algorithm, Readability, as well as a set of site-specific extraction rules which we maintain on Github[1]. 2. Our PDF Newspaper[2] application then cleans that extracted article output to normalise HTML/CSS styles so we can present the output in our own layout without things breaking. We use CSS for multi-column output. 3. Finally, we use Chromium's new headless API to request the generated output in step 2 and create three PDFs - injecting CSS to increase the font size each time. After we have the three PDF files, we check the PDFs in order of smallest to largest font size and as soon as a larger font size creates an extra new page, we discard that and keep the PDF we had before and return that. So if PDF 1 (small font size) has three pages, PDF 2 (medium font size) also three pages, but PDF 3 (larger font size) four pages, then we return PDF 2. [0] http://fivefilters.org/content-only/ http://fivefilters.org/content-only/ [1] https://github.com/fivefilters/ftr-site-config https://github.com/fivefilters/ftr-site-config [2] http://fivefilters.org/pdf-newspaper/ http://fivefilters.org/pdf-newspaper/

roadbeats 9y ago
cool stuff! I tried it in my blog and the result was excellent.
gravypod 9y ago
Does this support images? I tried to convert Ernest P. Chan's blog [0] into a pdf using this and the results were not well formatted. [0] - http://epchan.blogspot.com/ http://epchan.blogspot.com/
- k1m 9y ago
  We'll add images soon. As for the blog post, should work if you use the article URL: http://pdf.fivefilters.org/simple-print/url.php?size=A4#http://epchan.blogspot.se/2018/02/fx-order-flow-as-predictor.html http://pdf.fivefilters.org/simple-print/url.php?size=A4#http...
  - everydayavi 9y ago
    Timeline for images? Well done!
pbhjpbhj 9y ago
Step two seems to be almost certainly copyright infringement in the UK, not sure about USA exactly but as you take money and then copy the content it seems likely to breech the fair use regulations. Has this been addressed? Do you have an article on it or a position statement?