4 ms·

For servers, their choices are often bad. I recall how we had database failures in the middle of the night on the first of each month, until we figured out that

by cft 10y ago

For servers, their choices are often bad. I recall how we had database failures in the middle of the night on the first of each month, until we figured out that a monthly chron was running slocate indexing all files and IO jumped to 100% under load. I have many anecdotes like this. Also they don't seem to understand that during the update, rebooting a server is the last resort, unlike a laptop. In the server distribution patch instructions, it often says "reboot your computer", whereas one can just restart the services like in the latest openssl security update.

mwpmaybe 10y ago
Are you sure that was specific to Ubuntu and not general to Debian?
acdha 10y ago
locate has been a standard feature on Unix boxes for decades – it's certainly not specific to Ubuntu and the only way it causes I/O problem is either if the system is already close to breaking but nobody had been paying attention or the local configuration has been customized to do something like crawl NFS mounts. > In the server distribution patch instructions, it often says "reboot your computer", whereas one can just restart the services like in the latest openssl security update. It seems like a bad idea to criticize them for taking the safest tack in generic documentation intended for a wide audience. Some Ubuntu servers are run by veteran sysadmins but others run by people who are learning, primarily working on other things, etc. Restarting processes requires a decision per-patch to understand all of the affected components and safe restart strategies for all of them – e.g. in the case of OpenSSL, the library will be loaded not only by services but also other long-running jobs – cron tasks, anything a user has been running, etc. Yes, you can script looking for open file-handles and try to restart everything but if that goes wrong in any way, you're running with a known security hole which people will incorrectly think has been patched and may even claim that a scanner must be reporting a false-positive (I've personally seen that). If I was writing documentation to give to non-experts for operations, I'd make the same choice every time because it's simple and fails safely. Experts probably aren't going to read that documentation anyway and have enough knowledge to understand when they can make optimizations based on local knowledge.
- cft 10y ago
  I am all for having the mlocate program, but imposing a chron schedule on a server that causes heavy IO is stupid. Interestingly they have understood and disabled the mlocate chron in the server versions since at least 14.01. Have you administered a server with say 10m small files? It runs with negligible IO under normal traffic since recently accessed files are cached in memory, but running locate will kill its disks (sometimes physically). There were other features in Unix for "decades", such as Bash Shellshock bug for example.
  - acdha 10y ago
    Having a cron job which makes something useful out of the box is not stupid. It's useful for most people and any sysadmin who needs to customize what is indexed can easily do so if there's a path they want to exclude which isn't already in the default list. 10M files isn't a huge amount by post-90s standards and talking about disk access destroying hardware is pure hyperbole. If you have disks failing that frequently something is wrong with your server room, cooling, etc. A daily cron task is not a make or break problem there: I mean, 15 years ago I had that many small files backing a departmental email setup (maildir, so one file per message) — if you think locate is a problem, consider the I/O characteristics of hundreds of IMAP sessions and remember that RAM was a lot more expensive back then and many individual users had mailboxes larger than the aggregate total server memory). Drives failed, yes, but usually after years of heavy service.
    - JdeBP 10y ago
      You are erroneously conflating disc failures with database failures, which is what was actually written.
      - acdha 10y ago
        It's generally considered poor form around here to accuse people of not reading correctly with no further discussion. Perhaps you could enlighten us by explaining how the quote I was responding to, “will kill its disks (sometimes physically)”, is so clearly about databases rather than disks?

by cft 10y ago

mwpmaybe 10y ago

Are you sure that was specific to Ubuntu and not general to Debian?

acdha 10y ago

locate has been a standard feature on Unix boxes for decades – it's certainly not specific to Ubuntu and the only way it causes I/O problem is either if the system is already close to breaking but nobody had been paying attention or the local configuration has been customized to do something like crawl NFS mounts. > In the server distribution patch instructions, it often says "reboot your computer", whereas one can just restart the services like in the latest openssl security update. It seems like a bad idea to criticize them for taking the safest tack in generic documentation intended for a wide audience. Some Ubuntu servers are run by veteran sysadmins but others run by people who are learning, primarily working on other things, etc. Restarting processes requires a decision per-patch to understand all of the affected components and safe restart strategies for all of them – e.g. in the case of OpenSSL, the library will be loaded not only by services but also other long-running jobs – cron tasks, anything a user has been running, etc. Yes, you can script looking for open file-handles and try to restart everything but if that goes wrong in any way, you're running with a known security hole which people will incorrectly think has been patched and may even claim that a scanner must be reporting a false-positive (I've personally seen that). If I was writing documentation to give to non-experts for operations, I'd make the same choice every time because it's simple and fails safely. Experts probably aren't going to read that documentation anyway and have enough knowledge to understand when they can make optimizations based on local knowledge.

cft 10y ago
I am all for having the mlocate program, but imposing a chron schedule on a server that causes heavy IO is stupid. Interestingly they have understood and disabled the mlocate chron in the server versions since at least 14.01. Have you administered a server with say 10m small files? It runs with negligible IO under normal traffic since recently accessed files are cached in memory, but running locate will kill its disks (sometimes physically). There were other features in Unix for "decades", such as Bash Shellshock bug for example.
- acdha 10y ago
  Having a cron job which makes something useful out of the box is not stupid. It's useful for most people and any sysadmin who needs to customize what is indexed can easily do so if there's a path they want to exclude which isn't already in the default list. 10M files isn't a huge amount by post-90s standards and talking about disk access destroying hardware is pure hyperbole. If you have disks failing that frequently something is wrong with your server room, cooling, etc. A daily cron task is not a make or break problem there: I mean, 15 years ago I had that many small files backing a departmental email setup (maildir, so one file per message) — if you think locate is a problem, consider the I/O characteristics of hundreds of IMAP sessions and remember that RAM was a lot more expensive back then and many individual users had mailboxes larger than the aggregate total server memory). Drives failed, yes, but usually after years of heavy service.
  - JdeBP 10y ago
    You are erroneously conflating disc failures with database failures, which is what was actually written.
    - acdha 10y ago
      It's generally considered poor form around here to accuse people of not reading correctly with no further discussion. Perhaps you could enlighten us by explaining how the quote I was responding to, “will kill its disks (sometimes physically)”, is so clearly about databases rather than disks?