4 ms·
> can you write a web scraper in your browser? The answer is: YES, you can! So why is nobody doing it? Completely agree with this sentiment. I just spent the
by joshdavham 2y ago
> can you write a web scraper in your browser? The answer is: YES, you can! So why is nobody doing it?
Completely agree with this sentiment.
I just spent the last couple of months developing a chrome extension, but recently also did an unrleated web scraping project where I looked into all the common tools like beautiful soup, selenium, playwright, pupeteer, etc, etc.
All of these tools were needlessly complicated and I was having a ton of trouble with sites that required authentication. I then realized it would be way easier to write some javascript and paste it in my browser to do the scraping. Worked like a charm!
- metadat 2y agoYou might like Tamper monkey. You can add a button to kick it off or whatever your heart desires. Tampermoney also works around CORs issues with relative ease.
- crtasm 2y agoUserscripts for the win. Consider Violentmonkey over Tampermonkey though.
- metadat 2y ago> Consider Violentmonkey over Tampermonkey though. Why?
- MC995 2y agoProbably because it looks to be closed source these days. https://github.com/Tampermonkey/tampermonkey https://github.com/Tampermonkey/tampermonkey > This repository contains the source of the Tampermonkey extension up to version 2.9. All newer versions are distributed under a proprietary license.
- panphora 2y agoOpen source
- joshdavham 2y agoI love Tamper monkey! I've never made my own scripts though - just used other peoples'.
- moritzwarhier 2y agoIs Playwright really that complicated? I feel that when it has been set up, it's very straightforward to use. Maybe in contrast to other solutions you posted? Not sure about that though; having only brief experiences with both, Playwright seems like an improved Cypress to me.