4 ms·
So it was all the files in one go, presumably with `cp -r`? What about doing something with find/xargs/i-dunno to copy all the files, but break em into batches
by jrochkind1 12y ago
So it was all the files in one go, presumably with `cp -r`?
What about doing something with find/xargs/i-dunno to copy all the files, but break em into batches so you aren't asking cp to do it's bookkeeping for so many files in one process? Would that work better? Or worse in other ways?
- dangero 12y agoThe idea that you would need to do this seems fundamentally broken to me.
- pedrocr 12y agoIt should work much better provided you can easily divide it into batches where you know they don't have any shared hardlinked files. Otherwise you will duplicate those files wasting some space. That may be fine as well though.
- xchg_ax_ax 12y agoThis page may be useful: http://unix.stackexchange.com/questions/44247/how-to-copy-directories-with-preserving-hardlinks http://unix.stackexchange.com/questions/44247/how-to-copy-di... The main issue is that there's no api to get the list of files hard linked together: the only way is to check all the existing files and compare inodes. If you're doing a plain copy over 2 fs, you cannot choose which number the target inode will be, so you need to keep a map between inode numbers, or between inodes and file names ("cp" does the later).
- sounds 12y agopedrocr's comment above suggests a good solution: 1. Copy each file from the source volume to a single directory (e.g. /tmp) on the target volume, named for the source volume inode number. (edit: I suggest using a hierarchy of dirs to avoid the "too many dentry's" slowdown) 2. If the file has already been copied, it will already exist in /tmp - looking up the inode is a vanilla directory lookup 3. Create a hard link from /tmp to the actual path of the file 4. When all the files have been created on the target volume, delete the inode numbers in /tmp
- fspeech 12y agoHow do you keep hard links?