3 ms·

“developer skill problem”. Nice insult there, I’m tempted to respond in kind, but I’ll just point out that PHP doesn’t have much support for async operations of

by mikl 3y ago

“developer skill problem”. Nice insult there, I’m tempted to respond in kind, but I’ll just point out that PHP doesn’t have much support for async operations of any kind, and all the big frameworks are written in single-threaded “blocking” fashion, so the only way to have parallel processing is to run multiple instances of the program in parallel. And if you google for “php memory leak”, you’ll get a lot more than 0 results.

gumballindie 3y ago
Sorry for making it sound as an insult, i should have phrased it differently. Since php scripts are meant to be run once per request memory leaks are not an issue - any build up gets “reset” once execution ends. There are however shit frameworks that have circular references to objects and do all kinds of crap that builds memory up - easily avoided by parallel queue consumers. Threading and async are non issues given my statement above, hence me blaming on the dev’s skill and not understanding what the language does. For instance, you dont normally start a background process with an event loop to handle stuff. You start queue consumers at set intervals that run in parallel, or scripts that process batches of code - thus mimicking parallelism in a classic multi threaded app. This way you reduce the blast radius, are guaranteed that at least some data will always be processed and avoid memory leaks alltogether. No need for “conscious” async since its async by nature. However there is nuance in everything. Would you be able to provide a specific example of where you end up with such issues so i can better detail an example of how i would approach it?
- mikl 3y ago
  > php scripts are meant to be run once per request memory leaks are not an issue Yes, and that’s exactly why I said that PHP is awful for background processing, because there the process ideally runs continuously, or at least as long as there’s data to process. With PHP, I have to continuously re-spawn the processes to keep the memory leaks in check. If your application is so simple that it doesn’t need to process data outside of the request/response cycle, I’m sure PHP is fine. And sure, you can compensate for PHPs lack of async processing by spawning more processes, but that makes the RAM bloat problem even worse.
  - trog 3y ago
    We have scripts that have been running continuously for probably years at a time with none of these issues. The only time they stop is when servers are rebooted for updates. We actually had a big argument about building them like this for exactly the reasons you mentioned though - the preference for me and some others was to run them as once off's from cron, specifically to avoid the possibility of memory issues. Our tech lead wanted them to run as services (I think to improve responsiveness primarily). I didn't think it would work as well, but history has proven that it worked fine. I can't say it was better - it took a while to work out the kinks - but these things have been happily running for over a decade with basically zero issues relating to memory leaks, etc. They're not super complicated but not trivial either.

by mikl 3y ago

gumballindie 3y ago

Sorry for making it sound as an insult, i should have phrased it differently. Since php scripts are meant to be run once per request memory leaks are not an issue - any build up gets “reset” once execution ends. There are however shit frameworks that have circular references to objects and do all kinds of crap that builds memory up - easily avoided by parallel queue consumers. Threading and async are non issues given my statement above, hence me blaming on the dev’s skill and not understanding what the language does. For instance, you dont normally start a background process with an event loop to handle stuff. You start queue consumers at set intervals that run in parallel, or scripts that process batches of code - thus mimicking parallelism in a classic multi threaded app. This way you reduce the blast radius, are guaranteed that at least some data will always be processed and avoid memory leaks alltogether. No need for “conscious” async since its async by nature. However there is nuance in everything. Would you be able to provide a specific example of where you end up with such issues so i can better detail an example of how i would approach it?

mikl 3y ago
> php scripts are meant to be run once per request memory leaks are not an issue Yes, and that’s exactly why I said that PHP is awful for background processing, because there the process ideally runs continuously, or at least as long as there’s data to process. With PHP, I have to continuously re-spawn the processes to keep the memory leaks in check. If your application is so simple that it doesn’t need to process data outside of the request/response cycle, I’m sure PHP is fine. And sure, you can compensate for PHPs lack of async processing by spawning more processes, but that makes the RAM bloat problem even worse.
- trog 3y ago
  We have scripts that have been running continuously for probably years at a time with none of these issues. The only time they stop is when servers are rebooted for updates. We actually had a big argument about building them like this for exactly the reasons you mentioned though - the preference for me and some others was to run them as once off's from cron, specifically to avoid the possibility of memory issues. Our tech lead wanted them to run as services (I think to improve responsiveness primarily). I didn't think it would work as well, but history has proven that it worked fine. I can't say it was better - it took a while to work out the kinks - but these things have been happily running for over a decade with basically zero issues relating to memory leaks, etc. They're not super complicated but not trivial either.