3 ms·

My memory is hazy but I think I remember running into an actual memory leak in lighttpd circa 2008. We were serving dynamic content via FastCGI, IIRC. This wa

My memory is hazy but I think I remember running into an actual memory leak in lighttpd circa 2008. We were serving dynamic content via FastCGI, IIRC. This was a long time ago and I'm pretty hazy on the details, but I'm pretty sure I remember finding memory leak bug discussions on the lighttpd website around that time, and no clear answers on how to avoid it.

kijin 5y ago
Back in 2008, if you had a FastCGI backend that produced a large body, such as a file download, lighttpd would allocate the same amount of memory and forget to free it afterward. Everything would work fine for a while, but then someone attaches a large file on your busy PHP forum and bam! Your 720MB linode is thrashing like there's no tomorrow. The proper workaround was to send an X-Sendfile header to instruct lighttpd to fetch the file itself, instead of serving the content directly from the backend. It was also more efficient, but it required changes to backend code that made it less portable. I don't know if the bug was ever fixed as lighttpd development had slowed to a crawl and nginx arrived just in time to take away all the market share. The introduction of PHP-FPM around the same time was another factor that favored nginx, because lighttpd typically integrated with PHP using a more fragile setup called fcgi-wrapper. PHP-FPM was much nicer to work with, and nginx could even load-balance across FPM pools. A lot of WordPress sites switched to nginx and never looked back.
- Steltek 5y ago
  Just to say thank you for this closure ~15 years later. I was on team lighttpd back in the day (I think I liked the config syntax more? I honestly can't remember) but this bug caused a lot of grief and at the time, it appeared like no one knew the cause. Still, lighttpd/fcgi was still better than the epic custom Apache C++ module disaster in the previous iteration.
  - kijin 5y ago
    Yeah, the mod_fastcgi/mod_fcgid split on the Apache side was a total mess. The Apache modules also took on the thankless task of spawning and manageing PHP-CGI processes themselves, whereas nginx had FPM take care of it. IIRC lighty's spawn-fcgi wrapper design was halfway between these two approaches, but it wasn't particularly stable.
- NateEag 5y ago
  Thank you for explaining this so clearly. Like the other poster commented, this gives me a sense of closure.

kijin 5y ago

Back in 2008, if you had a FastCGI backend that produced a large body, such as a file download, lighttpd would allocate the same amount of memory and forget to free it afterward. Everything would work fine for a while, but then someone attaches a large file on your busy PHP forum and bam! Your 720MB linode is thrashing like there's no tomorrow. The proper workaround was to send an X-Sendfile header to instruct lighttpd to fetch the file itself, instead of serving the content directly from the backend. It was also more efficient, but it required changes to backend code that made it less portable. I don't know if the bug was ever fixed as lighttpd development had slowed to a crawl and nginx arrived just in time to take away all the market share. The introduction of PHP-FPM around the same time was another factor that favored nginx, because lighttpd typically integrated with PHP using a more fragile setup called fcgi-wrapper. PHP-FPM was much nicer to work with, and nginx could even load-balance across FPM pools. A lot of WordPress sites switched to nginx and never looked back.

Steltek 5y ago
Just to say thank you for this closure ~15 years later. I was on team lighttpd back in the day (I think I liked the config syntax more? I honestly can't remember) but this bug caused a lot of grief and at the time, it appeared like no one knew the cause. Still, lighttpd/fcgi was still better than the epic custom Apache C++ module disaster in the previous iteration.
- kijin 5y ago
  Yeah, the mod_fastcgi/mod_fcgid split on the Apache side was a total mess. The Apache modules also took on the thankless task of spawning and manageing PHP-CGI processes themselves, whereas nginx had FPM take care of it. IIRC lighty's spawn-fcgi wrapper design was halfway between these two approaches, but it wasn't particularly stable.
NateEag 5y ago
Thank you for explaining this so clearly. Like the other poster commented, this gives me a sense of closure.