3 ms·

I don't know why any program would copy files, let alone twice, to un-archive them, but the usual extract-then-mv pattern is an important feature, usually well-

I don't know why any program would copy files, let alone twice, to un-archive them, but the usual extract-then-mv pattern is an important feature, usually well-worth the cost of the extra syscall: if the extraction is interrupted, it avoids leaving incomplete files that are indistinguishable from correctly-extracted files. I would consider extracting in place by default a design flaw.

deathanatos 7y ago
Extract and move is fine, but many programs extract to a temp dir first, and then copy the file. I don't think anyone minds an extra rename (and, as you say, it gets you a sort of atomicity with the extract) but moving a file out of a temp dir can mean doing a full copy, if the destination is not on the same FS. As a side note: for POSIX OS's, this is why I much prefer the FD based operations. I don't think it quite exists, but I really think the right answer here is to create a deleted file, and then give it a filename. That is, you write the data out to disk, and hardlink it into its final resting place. Atomic, but with no mess of figuring out a temporary name. This is the reverse of how temporary files work, where you create a temp file & then delete it, before writing to it. (Linux allows this; it'll clean up the file when it is closed. It's backed by disk, and since it lacks a name, it can't be inadvertently opened or messed with by other programs. It's a way of saying "give me anonymous disk space".) There's a syscall to create deleted files in one-shot, now, but I don't think there's a way to hardlink them, yet. (Note that you need to still provide either a path, or the FD to a directory, s.t. the OS can determine what FS should back the file, and that you have access to disk at all.)
- firethief 7y ago
  > Extract and move is fine, but many programs extract to a temp dir first, and then copy the file. I don't think anyone minds an extra rename (and, as you say, it gets you a sort of atomicity with the extract) but moving a file out of a temp dir can mean doing a full copy, if the destination is not on the same FS. The way many programs deal with this (Firefox, rsync by default I think) is to create a temp file in the destination directory with a throwaway initial name. This approach seems like a good tradeoff to me. I agree that using a temp directory for an arbitrarily-large file that is intended to be written to disk is a big mistake. I don't think any tools I regularly use do this, but my /tmp is a tmpfs so it wouldn't be too pessimal on my setup. > As a side note: for POSIX OS's, this is why I much prefer the FD based operations. I don't think it quite exists, but I really think the right answer here is to create a deleted file, and then give it a filename. That is, you write the data out to disk, and hardlink it into its final resting place. Atomic, but with no mess of figuring out a temporary name. This is the reverse of how temporary files work, where you create a temp file & then delete it, before writing to it. (Linux allows this; it'll clean up the file when it is closed. It's backed by disk, and since it lacks a name, it can't be inadvertently opened or messed with by other programs. It's a way of saying "give me anonymous disk space".) There's a syscall to create deleted files in one-shot, now, but I don't think there's a way to hardlink them, yet. (Note that you need to still provide either a path, or the FD to a directory, s.t. the OS can determine what FS should back the file, and that you have access to disk at all.) How would you specify which filesystem the deleted file should be created on?
longstation 7y ago
If an SSD is used, would the extract and copy approach cause more wear?

deathanatos 7y ago

Extract and move is fine, but many programs extract to a temp dir first, and then copy the file. I don't think anyone minds an extra rename (and, as you say, it gets you a sort of atomicity with the extract) but moving a file out of a temp dir can mean doing a full copy, if the destination is not on the same FS. As a side note: for POSIX OS's, this is why I much prefer the FD based operations. I don't think it quite exists, but I really think the right answer here is to create a deleted file, and then give it a filename. That is, you write the data out to disk, and hardlink it into its final resting place. Atomic, but with no mess of figuring out a temporary name. This is the reverse of how temporary files work, where you create a temp file & then delete it, before writing to it. (Linux allows this; it'll clean up the file when it is closed. It's backed by disk, and since it lacks a name, it can't be inadvertently opened or messed with by other programs. It's a way of saying "give me anonymous disk space".) There's a syscall to create deleted files in one-shot, now, but I don't think there's a way to hardlink them, yet. (Note that you need to still provide either a path, or the FD to a directory, s.t. the OS can determine what FS should back the file, and that you have access to disk at all.)

firethief 7y ago
> Extract and move is fine, but many programs extract to a temp dir first, and then copy the file. I don't think anyone minds an extra rename (and, as you say, it gets you a sort of atomicity with the extract) but moving a file out of a temp dir can mean doing a full copy, if the destination is not on the same FS. The way many programs deal with this (Firefox, rsync by default I think) is to create a temp file in the destination directory with a throwaway initial name. This approach seems like a good tradeoff to me. I agree that using a temp directory for an arbitrarily-large file that is intended to be written to disk is a big mistake. I don't think any tools I regularly use do this, but my /tmp is a tmpfs so it wouldn't be too pessimal on my setup. > As a side note: for POSIX OS's, this is why I much prefer the FD based operations. I don't think it quite exists, but I really think the right answer here is to create a deleted file, and then give it a filename. That is, you write the data out to disk, and hardlink it into its final resting place. Atomic, but with no mess of figuring out a temporary name. This is the reverse of how temporary files work, where you create a temp file & then delete it, before writing to it. (Linux allows this; it'll clean up the file when it is closed. It's backed by disk, and since it lacks a name, it can't be inadvertently opened or messed with by other programs. It's a way of saying "give me anonymous disk space".) There's a syscall to create deleted files in one-shot, now, but I don't think there's a way to hardlink them, yet. (Note that you need to still provide either a path, or the FD to a directory, s.t. the OS can determine what FS should back the file, and that you have access to disk at all.) How would you specify which filesystem the deleted file should be created on?

longstation 7y ago

If an SSD is used, would the extract and copy approach cause more wear?