31 ms·

I'm curious now: what would be a more appropriate solution for each of the cases he gave (rsync, and mysqldump on a large database)?

by gchucky 16y ago

cagenut 16y ago
The first one would be hard to do without writing code. Thats probably what the googler intended, but of course not how a sysadmin is going to approach the problem. The second one would be a mix of xtrabackup and/or lvm (or san) snapshots on a slave. I've read this particular guy's blog a bunch of times before (its thin pickings out in the sysadmin blogosphere). His rush to the 90% answer bias is almost certainly a byproduct of his work environment, a small shop where ordering laptops is more a part of his day than performance tuning the website infrastructure. When your job covers a large variety and volume of relatively low-end problems, you would of course develop that reflex.
JoachimSchipper 16y ago
I think you could have these solutions when dealing with lots of data, but I'm not an expert in either: MySQL: replicate the database to one or more slaves, and either run mysqldump on a live slave (which should have sufficient capacity, since it's only keeping up with the INSERTs on the master and not handling queries) or take the slave offline, make a backup (EDIT: somehow - I don't really know MySQL all that well) and then make it catch back up (which may not be easy). rsync: sensible people only create paths of the form f/s/myfile if they are expecting a ton of files. In this case, keeping track of file attributes may be extremely valuable (if UNIX ctime hasn't changed, rsync doesn't need to look at the file at all, which saves a lot of disk I/O.) (EDIT:) I just remembered that rsync tries to preserve hard links, too - and therefore remembers the inode of every file. I'm pretty sure that you can run a machine out of memory with enough small files.

31 ms·

I'm curious now: what would be a more appropriate solution for each of the cases he gave (rsync, and mysqldump on a large database)?

by gchucky 16y ago

I'm curious now: what would be a more appropriate solution for each of the cases he gave (rsync, and mysqldump on a large database)?

cagenut 16y ago
The first one would be hard to do without writing code. Thats probably what the googler intended, but of course not how a sysadmin is going to approach the problem. The second one would be a mix of xtrabackup and/or lvm (or san) snapshots on a slave. I've read this particular guy's blog a bunch of times before (its thin pickings out in the sysadmin blogosphere). His rush to the 90% answer bias is almost certainly a byproduct of his work environment, a small shop where ordering laptops is more a part of his day than performance tuning the website infrastructure. When your job covers a large variety and volume of relatively low-end problems, you would of course develop that reflex.
JoachimSchipper 16y ago
I think you could have these solutions when dealing with lots of data, but I'm not an expert in either: MySQL: replicate the database to one or more slaves, and either run mysqldump on a live slave (which should have sufficient capacity, since it's only keeping up with the INSERTs on the master and not handling queries) or take the slave offline, make a backup (EDIT: somehow - I don't really know MySQL all that well) and then make it catch back up (which may not be easy). rsync: sensible people only create paths of the form f/s/myfile if they are expecting a ton of files. In this case, keeping track of file attributes may be extremely valuable (if UNIX ctime hasn't changed, rsync doesn't need to look at the file at all, which saves a lot of disk I/O.) (EDIT:) I just remembered that rsync tries to preserve hard links, too - and therefore remembers the inode of every file. I'm pretty sure that you can run a machine out of memory with enough small files.