4 ms·

The server got wedged: it couldn't write to the log file because there were too many files open. Still not sure why there were so many files open. I've never

by pg 18y ago

The server got wedged: it couldn't write to the log file because there were too many files open. Still not sure why there were so many files open. I've never seen that error before. But the program that notices when the server is wedged and kills it didn't catch it in this case, because instead of not responding it would respond but with an empty page.

shafqat 18y ago
What was the exact error? What kind of files were open? If you want to share, I'm sure lots of us would be interested to hear the details so we know what to look out for.
- jey 18y ago
  Sounds like the limit for the maximum number of open File Descriptors (FDs) was reached. On Linux, you can check the limit by doing: cat /proc/sys/fs/file-max You can change this limit: echo 100000 > /proc/sys/fs/file-max This limit is also available through the "fs.file-max" sysctl. The "lsof" tool is great for seeing what FDs a process has open. Note that FDs are used for more than just regular files: directories, sockets, pipes, and other file-like objects too.
  - abstractbill 18y ago
    netstat is also handy here (e.g. sometimes you find a single remote IP is holding open way more sockets than it should be).
  - axod 18y ago
    Also ulimit and /etc/security/limits.conf The default for max open files (1024) is ridiculously low. First thing to do when installing linux, up the max open file limits.
- pg 18y ago
  It looks as if most of the file descriptors in use at any given time refer to closed tcp connections. We're still trying to figure out what this means.
breck 18y ago
"I've never seen that error before." I feel like nothing better describes keeping a web app up and running than Murphy's Law. I amazed at how many little things have developed over time and temporarily brought down my sites over the years.
cstejerean 18y ago
Last time I saw too many open files errors was because of too many open sockets (some piece of code was leaking socket handles).
- tdavis 18y ago
  This was the bane of my existence at one point.

by pg 18y ago

shafqat 18y ago

What was the exact error? What kind of files were open? If you want to share, I'm sure lots of us would be interested to hear the details so we know what to look out for.

jey 18y ago
Sounds like the limit for the maximum number of open File Descriptors (FDs) was reached. On Linux, you can check the limit by doing: cat /proc/sys/fs/file-max You can change this limit: echo 100000 > /proc/sys/fs/file-max This limit is also available through the "fs.file-max" sysctl. The "lsof" tool is great for seeing what FDs a process has open. Note that FDs are used for more than just regular files: directories, sockets, pipes, and other file-like objects too.
- abstractbill 18y ago
  netstat is also handy here (e.g. sometimes you find a single remote IP is holding open way more sockets than it should be).
- axod 18y ago
  Also ulimit and /etc/security/limits.conf The default for max open files (1024) is ridiculously low. First thing to do when installing linux, up the max open file limits.
pg 18y ago
It looks as if most of the file descriptors in use at any given time refer to closed tcp connections. We're still trying to figure out what this means.

breck 18y ago

"I've never seen that error before." I feel like nothing better describes keeping a web app up and running than Murphy's Law. I amazed at how many little things have developed over time and temporarily brought down my sites over the years.

cstejerean 18y ago

Last time I saw too many open files errors was because of too many open sockets (some piece of code was leaking socket handles).

tdavis 18y ago
This was the bane of my existence at one point.