jancurn

11 ms·

1. 61.▲
   by jancurn 9y ago

   You can also use page.authenticate() for that - see a note at the bottom of the article. Also see https://github.com/GoogleChrome/puppeteer/pull/1732
2. 62.▲
   by jancurn 9y ago

   There's a workaround - https://blog.apify.com/how-to-make-headless-chrome-and-puppe...
3. 63.▲
   by jancurn 9y ago

   Yes we are! Please see https://www.apify.com/jobs
4. 64.▲
   by jancurn 9y ago

   You can use https://www.htbridge.com/ssl/ for that
5. 65.▲
   How to make headless Chrome and Puppeteer use a proxy server with authentication(medium.com)

   2 points by jancurn 9y ago|0 comments
6. 66.▲
   by jancurn 9y ago

   Apify ( https://www.apify.com ) is a web scraping and automation platform where you can extract data from any website using a few simple lines of JavaScript. It's using headless browsers, so that people can extract data from
7. 67.▲
   by jancurn 9y ago

   Hey guys, sorry for the bother, we certainly don't want to abuse HN. All the posts had entirely different topics and we believe there are people who might genuinely benefit from them. If that's not the case, people won't upvo
8. 68.▲
   by jancurn 9y ago

   I believe it's possible, by adding something like "scale: 1.5" to "pdfOptions" you might render an accessible PDF
9. 69.▲
   by jancurn 9y ago

   Not yet, but we can quite easily add these features. Just let me know at jan@apify.com what would you need
10. 70.▲
    by jancurn 9y ago

    If you'd like to add specific features, please let me know at jan@apify.com, I'm sure we'll figure it out
11. 71.▲
    by jancurn 9y ago

    We can easily add this feature. So you'd like to pass a single HTML file?
12. 72.▲
    by jancurn 9y ago

    Here we go - another Apify act that uses pdf2htmlEX to convert PDF to HTML: https://www.apify.com/jancurn/pdf-to-html
13. 73.▲
    by jancurn 9y ago

    Now there is - I've just added the "sleepMillis" input option.
14. 74.▲
    by jancurn 9y ago

    Actually we're simply using Puppeteer - see the source code at the bottom of https://www.apify.com/jancurn/url-to-pdf
15. 75.▲
    by jancurn 9y ago

    There's another Apify act that extracts text from a PDF using the pdf-text-extract NPM package - see https://www.apify.com/juansgaitan/pdf-scraping If there's any library or tool that can convert PDF to HTML,
16. 76.▲
    by jancurn 9y ago

    You can call the API from anywhere, from resource-constrained servers, Docker containers that cannot run headless Chrome, JavaScript on a website etc. Also, we'll keep adding new features to this act to make it worthwhile to use it, e.
17. 77.▲
    Show HN: Ready-to-use API to convert any web page to PDF using headless Chrome(apify.com)

    129 points by jancurn 9y ago|64 comments
18. 78.▲
    by jancurn 9y ago

    The crawler uses PhantomJS, but you can also create jobs that use a plain HTML downloader or even cheerio. The platform is very flexible.
19. 79.▲
    by jancurn 9y ago

    Yes, every user has to agree with the Terms of use when signing up.
20. 80.▲
    by jancurn 9y ago

    We consider ourselves only as service providers. We provide technical means to perform the crawling but the responsibility for the actual crawling is with our users. It's similar to Amazon AWS - they only provide infrastructure and it&
21. 81.▲
    by jancurn 9y ago

    Our crawler ( https://www.apify.com/docs/crawler ) currently runs on PhantomJS, while Actor ( https://www.apify.com/docs/actor ) can run arbitrary jobs, including jobs that use headless Chrome and&#x2
22. 82.▲
    by jancurn 9y ago

    Thank you :)
23. 83.▲
    by jancurn 9y ago

    Hey guys, this is Jan, co-founder of Apify. Two years ago we showed HN Apifier - a hosted web crawler for developers - during our time in the Y Combinator Fellowship. Today we're launching the largest upgrade of Apifier to date - a new
24. 84.▲
    Show HN: Apify – Turn any website into an API(apify.com)

    45 points by jancurn 9y ago|21 comments
25. 85.▲
    by jancurn 9y ago

    With a headless browser the web scraping script can be even simpler. For example, have a look at the same scraper for datawhatnow.com at https://www.apify.com/jancurn/YP4Xg-api-datawhatnow-com
26. 86.▲
    by jancurn 9y ago

    Yeah, but check https://www.example.com/another vs https://www.example.com/anything-else
27. 87.▲
    Does anyone know what's on this page?(example.com)

    1 points by jancurn 9y ago|2 comments
28. 88.▲
    by jancurn 9y ago

    same here, our systems running in US East are behaving weird
29. 89.▲
    API for Amazon.com(apifier.com)

    2 points by jancurn 10y ago|0 comments
30. 90.▲
    by jancurn 10y ago

    Well, https://www.apifier.com does essentially the same thing, plus it supports JavaScript, can crawl through the whole website etc. Disclaimer: I'm a cofounder there