4 ms·

> if you run GNU Tar with --sparse and a file shrinks while tar is reading it, tar fails to properly handle the resulting earlier than expected end of file. Is

by ezoe 8y ago

> if you run GNU Tar with --sparse and a file shrinks while tar is reading it, tar fails to properly handle the resulting earlier than expected end of file. Is that a bug? Yes. It's different than changing the data of file but still.

fanf2 8y ago
Clearly it’s a bug because tar goes into a mad loop reading nothing and outputting empty blocks forever, or until the file changes size again. It needs to check for eof to avoid this mistake, even if the eof is unexpected because the file size was different.
the8472 8y ago
I think most gnu utils are written so that they treat input files as a stream of data so you can also use pipes instead of regular files. This also means they can somewhat handle size changes since they will keep reading until they encounter an EOF without relying on an earlier snapshot of the file size or sparse chunks. Whether the results are useful depends on the file format, if it's just temporary padding as in this case it works.
newnewpdro 8y ago
Yes, it's expected for a general system utility like tar to handle such simultaneous file operations gracefully.
- hollerith 8y ago
  Agreed. OTOH, it is not a bug if the fact that tar is not instantaneous causes the tarball to contain combinations of files unlikely to occur during normal operation of the system. e.g., if a file is renamed while tar is running it is not a bug for the tarball to contain zero or two copies of that file.
  - newnewpdro 8y ago
    It is a bug for tar to enter an infinite loop writing an ever expanding file of trailing zeroes because the current file was truncated to a size smaller than its stat returned. It's basically a classic TOCTOU race, tar needs to honor the EOF returned from read() as authoritative. This is actually a very common trap for junior *nix programmers; treating stat() and a subsequent mmap() or read() loop as if they were atomic with regards to stat.st_size. The size returned by stat() can be used as an estimate, but otherwise can't be applied to subsequent operations.

by ezoe 8y ago

fanf2 8y ago

Clearly it’s a bug because tar goes into a mad loop reading nothing and outputting empty blocks forever, or until the file changes size again. It needs to check for eof to avoid this mistake, even if the eof is unexpected because the file size was different.

the8472 8y ago

I think most gnu utils are written so that they treat input files as a stream of data so you can also use pipes instead of regular files. This also means they can somewhat handle size changes since they will keep reading until they encounter an EOF without relying on an earlier snapshot of the file size or sparse chunks. Whether the results are useful depends on the file format, if it's just temporary padding as in this case it works.

newnewpdro 8y ago

Yes, it's expected for a general system utility like tar to handle such simultaneous file operations gracefully.

hollerith 8y ago
Agreed. OTOH, it is not a bug if the fact that tar is not instantaneous causes the tarball to contain combinations of files unlikely to occur during normal operation of the system. e.g., if a file is renamed while tar is running it is not a bug for the tarball to contain zero or two copies of that file.
- newnewpdro 8y ago
  It is a bug for tar to enter an infinite loop writing an ever expanding file of trailing zeroes because the current file was truncated to a size smaller than its stat returned. It's basically a classic TOCTOU race, tar needs to honor the EOF returned from read() as authoritative. This is actually a very common trap for junior *nix programmers; treating stat() and a subsequent mmap() or read() loop as if they were atomic with regards to stat.st_size. The size returned by stat() can be used as an estimate, but otherwise can't be applied to subsequent operations.