6 ms·

I've been successfully using Chrome headless in a 500MB Docker container for dumping the DOM for https://www.prerender.cloud/ https://www.prerender.cloud/ for m

by jotto 10y ago

I've been successfully using Chrome headless in a 500MB Docker container for dumping the DOM for https://www.prerender.cloud/ https://www.prerender.cloud/ for months (rendering a large variety of sites without restarts for weeks at a time) Run it with: --js-flags="--max_old_space_size=500" to force the VM to keep it GC'd below 500 Chrome v55 was a 30% memory savings, before that I used 1GB containers. It's not perfect, but I am definitely pushing high volume (multiple tabs, concurrent activity) and I am not having any significant stability issues and I am pushing diverse sets of webpages.

19eightyfour 10y ago
That ought actually be: --js-flags="--max-old-space-size=500" Hyphens not underscores. A list of V8 flags can be found with: --js-flags="--help"
- brazzledazzle 10y ago
  I think it may accept both the underscore and hyphen and just normalizes it.
  - deleted 10y ago
    [deleted]
  - 19eightyfour 10y ago
    This is correct, it does accept both, and the help lists it as underscores not hyphens. So the comment about it ought to be hyphens not underscores is incorrect. Either is fine.
    - brazzledazzle 10y ago
      When I went digging through the code for v8 I believe I saw examples of it both ways. I'm not dead sure though because I don't know very much C. I just checked because I was hoping they weren't actually using an invalid flag for all that time.
vvoyer 10y ago
What the best strategy that worked for you on high volumes, multiple tabs or multiple docker instances? I am wondering if multiple tabs is as efficient as multiple windows/instances. Thanks.
- heipei 10y ago
  I'm using more than one Chrome process so I can kill the processes every so often (e.g. after timeout or when they get stuck). Inside each Chrome instance I use 16 tabs, there might be a number of factors at play: - Are you worried about same-origin pollution if you run multiple tabs from the same origin in the same process? If so -> Extra process - Do you have to take screenshots? You can only take screenshots of the tab that's in the foreground, so you have to activate it first to take the screenshot. This might fail if you have lots of tabs which roughly trigger at once. You can see what I've built at https://urlscan.io https://urlscan.io btw.
pacuna 10y ago
what Docker image are you using?

by jotto 10y ago

19eightyfour 10y ago

That ought actually be: --js-flags="--max-old-space-size=500" Hyphens not underscores. A list of V8 flags can be found with: --js-flags="--help"

brazzledazzle 10y ago
I think it may accept both the underscore and hyphen and just normalizes it.
- deleted 10y ago
  [deleted]
- 19eightyfour 10y ago
  This is correct, it does accept both, and the help lists it as underscores not hyphens. So the comment about it ought to be hyphens not underscores is incorrect. Either is fine.
  - brazzledazzle 10y ago
    When I went digging through the code for v8 I believe I saw examples of it both ways. I'm not dead sure though because I don't know very much C. I just checked because I was hoping they weren't actually using an invalid flag for all that time.

vvoyer 10y ago

What the best strategy that worked for you on high volumes, multiple tabs or multiple docker instances? I am wondering if multiple tabs is as efficient as multiple windows/instances. Thanks.

heipei 10y ago
I'm using more than one Chrome process so I can kill the processes every so often (e.g. after timeout or when they get stuck). Inside each Chrome instance I use 16 tabs, there might be a number of factors at play: - Are you worried about same-origin pollution if you run multiple tabs from the same origin in the same process? If so -> Extra process - Do you have to take screenshots? You can only take screenshots of the tab that's in the foreground, so you have to activate it first to take the screenshot. This might fail if you have lots of tabs which roughly trigger at once. You can see what I've built at https://urlscan.io https://urlscan.io btw.

pacuna 10y ago

what Docker image are you using?