3 ms·
Okay, the way I see this is that using headless tools like puppeteer or selenium is tedious; just trying to... er scrape my HN account's favorites (AFAIK no API
by pcr910303 7y ago
Okay, the way I see this is that using headless tools like puppeteer or selenium is tedious; just trying to... er scrape my HN account's favorites (AFAIK no API) becomes a task when you have to automate login.
Just typing in and pressing the button is much easier than automating the task, so that's why the iframe is something useful. You can interact with the content (without code).
- oefrha 7y agoYou can use non-headless puppeteer to manually log in (or perform any other manual actions, or use the dev console in the puppeteer-controlled browser) and only automate parts you want to automate.
- hugs 7y agoThe irony is that the iframe approach is exactly how the first version of Selenium worked. It's a cool hack, but we abandoned that approach over time because automating iframes couldn't cover all automation use cases.
- elorant 7y agoSelenium allows you to load different browser profiles. So you create a profile manually, login to the target site and then everytime you load the profile automatically the cookies are set and you're logged-in. Works like a charm.
- 3xblah 7y agoIs login required to access favorites page; isnt the page public, e.g., https://news.ycombinator.com/favorites?id=pcr910303 https://news.ycombinator.com/favorites?id=pcr910303 In the case login was required, using tmux it's easy to automate login to HN with a text-only browser such as links and saving the desired text/html, etc. to a file. Takes me about 500 characters of script to non-interactively log in, grab some text and log out