5 ms·

Adventures with Dockerised Headed Chrome, Extensions and Selenium-Python

35 points by Thev00d00 7y ago

linsomniac 7y ago
A vendor recently switched from self-hosted to AWS, and the API I was using to backup our artifacts that are stored in their system stopped working. I submitted a support request, but they still haven't replied a month later. Not even a "We're looking into your ticket". So I created a Docker container that uses chromedriver+selenium and it works GREAT. My first attempt at Selenium. I did run into one problem though: Downloading large attachments failed. After a bit of searching, I found I needed to add the following docker argument: --shm-size=5g I had tried various other memory-related arguments (I figured it was memory related), but that's the one that did the trick.
- toomuchtodo 7y ago
  Do you have a repo you might be willing to share?
  - linsomniac 7y ago
    See the above comment, I'm putting it out here: https://gitlab.com/linsomniac/fogbugz-to-jira https://gitlab.com/linsomniac/fogbugz-to-jira
    - toomuchtodo 7y ago
      Thank you!
- LikeAnElephant 7y ago
  I'm very close to needing to do the same exact thing, would really appreciate a GitHub link or something do your Dockerfile
  - linsomniac 7y ago
    I'm definitely able to open it up, I just need to sanitize it to get my information out of it and clean up some things. I've started with the component I referenced here, but I do have a basically complete workflow that I'm able to release. I'm putting it here: https://gitlab.com/linsomniac/fogbugz-to-jira https://gitlab.com/linsomniac/fogbugz-to-jira
miohtama 7y ago
If you want to use Selenium with Python, Splinter library provides some of much needed abstraction and utility functions to make your test code shorter and more readable https://splinter.readthedocs.io/en/latest/why.html https://splinter.readthedocs.io/en/latest/why.html
nicois 7y ago
We have rolled our own wrapper around selenium implementing page models, using pytest. It feels as though we are reinventing the wheel, though I'm not aware of an existing public package providing this. There are many caveats for the unwary when beginning a selenium adventure: detecting and retrying stale element exceptions (you can't just retry the final (say) click operation; you need to retry a DOM interaction sequence), detecting when an async operation begins and reaches a steady state, reliable snapshots and video recording across browsers, injecting the correct args to the browser containers, etc.
- mikeyjk 7y ago
  Adding 'waits' in for UI elements to be available feels wrong. Seems like busy work.

35 points by Thev00d00 7y ago

A vendor recently switched from self-hosted to AWS, and the API I was using to backup our artifacts that are stored in their system stopped working. I submitted a support request, but they still haven't replied a month later. Not even a "We're looking into your ticket". So I created a Docker container that uses chromedriver+selenium and it works GREAT. My first attempt at Selenium. I did run into one problem though: Downloading large attachments failed. After a bit of searching, I found I needed to add the following docker argument: --shm-size=5g I had tried various other memory-related arguments (I figured it was memory related), but that's the one that did the trick.

toomuchtodo 7y ago
Do you have a repo you might be willing to share?
- linsomniac 7y ago
  See the above comment, I'm putting it out here: https://gitlab.com/linsomniac/fogbugz-to-jira https://gitlab.com/linsomniac/fogbugz-to-jira
  - toomuchtodo 7y ago
    Thank you!
LikeAnElephant 7y ago
I'm very close to needing to do the same exact thing, would really appreciate a GitHub link or something do your Dockerfile
- linsomniac 7y ago
  I'm definitely able to open it up, I just need to sanitize it to get my information out of it and clean up some things. I've started with the component I referenced here, but I do have a basically complete workflow that I'm able to release. I'm putting it here: https://gitlab.com/linsomniac/fogbugz-to-jira https://gitlab.com/linsomniac/fogbugz-to-jira

miohtama 7y ago

If you want to use Selenium with Python, Splinter library provides some of much needed abstraction and utility functions to make your test code shorter and more readable https://splinter.readthedocs.io/en/latest/why.html https://splinter.readthedocs.io/en/latest/why.html

nicois 7y ago

We have rolled our own wrapper around selenium implementing page models, using pytest. It feels as though we are reinventing the wheel, though I'm not aware of an existing public package providing this. There are many caveats for the unwary when beginning a selenium adventure: detecting and retrying stale element exceptions (you can't just retry the final (say) click operation; you need to retry a DOM interaction sequence), detecting when an async operation begins and reaches a steady state, reliable snapshots and video recording across browsers, injecting the correct args to the browser containers, etc.

mikeyjk 7y ago
Adding 'waits' in for UI elements to be available feels wrong. Seems like busy work.