4 ms·

Hey Joel–awesome service and post. Your tip about parallelizing via browser instances instead of tabs resonated with me as I've had a lot of headaches with one

Hey Joel–awesome service and post. Your tip about parallelizing via browser instances instead of tabs resonated with me as I've had a lot of headaches with one bad request taking down a full instance. That said, it does seem like the constant launch/close of puppeteer browser instances does have some sort of memory leak – Here's what happened when I switched from parallelizing via pages to browsers: https://imgur.com/a/rCz3oow https://imgur.com/a/rCz3oow Have you seen any similar behavior in your system? I initially thought I was forgetting to close the browser instance but triple checked my code and it does seem like I'm calling close() everywhere.

mtrunkat 8y ago
We have found 2 related issues: - Sometimes Promise returned by page.close() never resolves so it's good to call Promise.race() on that together with a Promise that resolves after some timeout period (30s?) - Sometimes Chrome process doesn't get killed so we are also manually killing remaining Chrome process after browser.close()
- dschnurr 8y ago
  Good tips. How are you killing remaining zombie Chrome processes? Just polling the processlist and running "kill" commands?
mrskitch 8y ago
Are you using something like dumb-init or equivalent? That's a definite requirement as _not_ having it results in zombie processes hanging around (which do retain some memory footprint). Read more about that here: https://github.com/Yelp/dumb-init#why-you-need-an-init-system https://github.com/Yelp/dumb-init#why-you-need-an-init-syste...
- dschnurr 8y ago
  I'm using dumb-init for the main node process in my docker container (node in-turn spawns chrome instances via puppeteer as requests come in). Is there a way to use dumb-init for each individual chrome instance as well? Or are you suggesting we make chrome the main container process and start new containers for each session?

mtrunkat 8y ago

We have found 2 related issues: - Sometimes Promise returned by page.close() never resolves so it's good to call Promise.race() on that together with a Promise that resolves after some timeout period (30s?) - Sometimes Chrome process doesn't get killed so we are also manually killing remaining Chrome process after browser.close()

dschnurr 8y ago
Good tips. How are you killing remaining zombie Chrome processes? Just polling the processlist and running "kill" commands?

mrskitch 8y ago

Are you using something like dumb-init or equivalent? That's a definite requirement as _not_ having it results in zombie processes hanging around (which do retain some memory footprint). Read more about that here: https://github.com/Yelp/dumb-init#why-you-need-an-init-system https://github.com/Yelp/dumb-init#why-you-need-an-init-syste...

dschnurr 8y ago
I'm using dumb-init for the main node process in my docker container (node in-turn spawns chrome instances via puppeteer as requests come in). Is there a way to use dumb-init for each individual chrome instance as well? Or are you suggesting we make chrome the main container process and start new containers for each session?