3 ms·
> Are you using headless Chrome or similar to implement this? Yes, with pyppeteer[0] (Python 3 port of puppeteer). > I found that even pages smaller than 5MB
by boramalper 9y ago
> Are you using headless Chrome or similar to implement this?
Yes, with pyppeteer[0] (Python 3 port of puppeteer).
> I found that even pages smaller than 5MB get rendered to PDF.
Web-pages (URLs that contain no file-extensions, or certain ones such as ".php", ".aspx" etc.) are rendered into PDFs. Generated PDFs must also be less than 5 MiB[1].
> It doesn't seem to support chunked transfer-encoding:
True. I need to determine the size of a file before downloading it completely, and I used "content-length" header for that. I can see how it should also be possible for chunked transfer-encoding, but I'm not willing to implement it myself to be honest, and I doubt if it's supported by requests library.
Thanks for the feedback!
[0]: https://miyakogi.github.io/pyppeteer/ https://miyakogi.github.io/pyppeteer/
[1]: An arbitrary limit. Might change it in the future.
- jstanley 9y agoI'd be extremely surprised if chunked encoding is not supported by the standard python http requests libraries.
- boramalper 9y agoI meant, a way to get the file-size when chunked transfer-encoding is used. I cannot imagine any, and I don't think requests has some magic way to calculate it (or more likely STOP after downloading n bytes). Maybe `stream=True` and `.read(5 * 1024 * 1024)` and check EOF, if not, abort.