4 ms·
Do you have to use Curl? It wouldn't render a lot of sites correctly anyway (anything that uses JS for rendering). Can you run a puppeteer/playwright instance
by solardev 2y ago
Do you have to use Curl? It wouldn't render a lot of sites correctly anyway (anything that uses JS for rendering).
Can you run a puppeteer/playwright instance (which control real browsers) and add an ad blocker to that? e.g. https://github.com/ghostery/adblocker https://github.com/ghostery/adblocker or https://github.com/microsoft/playwright-python/issues/782 https://github.com/microsoft/playwright-python/issues/782
- suramya_tomar 2y agoI don't have to use curl, but in the past when I have setup something that opens browser instances it has usually been a bit unstable in the sense that it would crash intermittently. I wanted something that I could kick-off as a daily cron job.
- solardev 2y agoHmm, can't you check for correctness somehow and retry on failure? Those headless browsers are often run in automated environments.