3 ms·
I'm going to plug my app that does scraping with Electron: https://github.com/CGamesPlay/chronicler https://github.com/CGamesPlay/chronicler To the commenters
by CGamesPlay 8y ago
I'm going to plug my app that does scraping with Electron: https://github.com/CGamesPlay/chronicler https://github.com/CGamesPlay/chronicler
To the commenters who don't understand why this is necessary:
- It reliably loads linked resources in a WYSIWYG fashion, including embedded media and other things that have to be handled in an ad-hoc fashion when using something like BeautifulSoup.
- It handles resources loaded through JavaScript, including HTML5 History API changes.
- aloer 8y agoCould you explain what electron offers here over for example a browser plugin? I'm not that familiar with limitations of WebExtension APIs It looks like an interesting project but only for a few selected sites. For more random browsing I believe I would be too security conscious (https://electronjs.org/docs/tutorial/security https://electronjs.org/docs/tutorial/security) to allow it Might be unlikely that some random script on a random site will target Electron but you never know
- CGamesPlay 8y agoWell in Chrome you can't hook into the network layer to record/replay requests like I've done here (you could fake it by overriding the ServiceWorker, possibly, but this feels brittle and I'm not sure if it's possible either). I'm not familiar with Firefox extensions but I'm given to understand you likely could implement this project as a Firefox extension. Locking down Electron apps to be safe on the larger web is certainly one area where I think electron could do a lot better. I think my project has followed all of the recommendations and should be safe, but I agree with you that it feels like a bigger attack surface. I personally wanted it to support offline browsing of documentation sites, which are generally pretty "safe" from that perspective.