4 ms·

What about storing each stream as a separate logical file in the tar file? So you have the main file, with a property of "nt_stream_count=2", and the file name

What about storing each stream as a separate logical file in the tar file? So you have the main file, with a property of "nt_stream_count=2", and the file name for each stream would be the main filename followed by ":stream_id". That way each stream would show up as a separate file upon restore (assuming a standard tar is used for restoring), but the custom tar would know what to do. Also, unrecognized pax data would be ignored by a regular tar. And in my use case, when data is submitted by the client to the Snebu backend, it has its own tar implementation built in for separating file data from metadata, and re-synthesizes a tar file upon restore. (The reason that Snebu uses tar format for transfering, is it was a good way to make it agentless for normal Unix/Linux servers, yet extensible enough to support other use cases. Also I figured that most tar implementations were fairly well optimized, at least more so then I could do in a short amount of time). Thanks for your input, really appreciate it. Been a little bit tough learning the WIN APIs, after a couple decades lost in Unix land.

Const-me 9y ago
> What about storing each stream as a separate logical file in the tar file? I think that’ll work. Just be sure to name these separate streams in a way so they never conflict with other files that might be in the same directory or with other streams that may be in the same file. E.g. in Windows, there’re characters forbidden in file names which may work fine in Linux: https://stackoverflow.com/a/31976060/126995 https://stackoverflow.com/a/31976060/126995 > with a property of "nt_stream_count=2" Read this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa362667%28v=vs.85%29.aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... I don’t think just count is enough. Alternate data streams have names. Other data streams, extended attribute, security descriptor, etc., are essentially unnamed. You can implement some naming schemes for them, e.g. “file” for the main data, “file:xxx” for “xxx” alternative stream of the file, “file?sd” for security descriptor of the file, “file?sparse” for sparse file data. This can result in TAR backups that are more or less readable by the standard tools, but still allow your own tool to restore the complete thing. You have two problems to solve. 1. Directories. Quite often, they contain at least security descriptors. But AFAIK directories just aren’t stored in these TARs. 2. File names and their encoding. AFAIK modern Linux is UTF8 all the way down. Probably TAR is OK with that. But if UTF8 representation of the UCS2 name (WinNT isn’t quite UTF16, UCS2 is a subset of that) is longer than 100 bytes, you’ll have multiple files in your TAR with the same name, they’ll only differ by your extended PAX attributes that contain the complete name. If you combine that with the above fake names with security info & alternate streams, it becomes even more complex. P.S. I’ve been programming for windows for decades, only occasionally for linux or other platforms.

Const-me 9y ago

> What about storing each stream as a separate logical file in the tar file? I think that’ll work. Just be sure to name these separate streams in a way so they never conflict with other files that might be in the same directory or with other streams that may be in the same file. E.g. in Windows, there’re characters forbidden in file names which may work fine in Linux: https://stackoverflow.com/a/31976060/126995 https://stackoverflow.com/a/31976060/126995 > with a property of "nt_stream_count=2" Read this: https://msdn.microsoft.com/en-us/library/windows/desktop/aa362667%28v=vs.85%29.aspx https://msdn.microsoft.com/en-us/library/windows/desktop/aa3... I don’t think just count is enough. Alternate data streams have names. Other data streams, extended attribute, security descriptor, etc., are essentially unnamed. You can implement some naming schemes for them, e.g. “file” for the main data, “file:xxx” for “xxx” alternative stream of the file, “file?sd” for security descriptor of the file, “file?sparse” for sparse file data. This can result in TAR backups that are more or less readable by the standard tools, but still allow your own tool to restore the complete thing. You have two problems to solve. 1. Directories. Quite often, they contain at least security descriptors. But AFAIK directories just aren’t stored in these TARs. 2. File names and their encoding. AFAIK modern Linux is UTF8 all the way down. Probably TAR is OK with that. But if UTF8 representation of the UCS2 name (WinNT isn’t quite UTF16, UCS2 is a subset of that) is longer than 100 bytes, you’ll have multiple files in your TAR with the same name, they’ll only differ by your extended PAX attributes that contain the complete name. If you combine that with the above fake names with security info & alternate streams, it becomes even more complex. P.S. I’ve been programming for windows for decades, only occasionally for linux or other platforms.