7 ms·

The (second-)worst offense I'm aware of here is that alternate data stream names can have otherwise special characters in them, like backslashes. So if you (for

The (second-)worst offense I'm aware of here is that alternate data stream names can have otherwise special characters in them, like backslashes. So if you (for example) want to strip off the last path component, you technically cannot do this by just stripping everything after the last backslash. In fact this probably isn't the worst thing - it's even worse than this. Because you first need to strip off the prefix that represent the volume (like C:\) before you can look for a colon. But the prefix can be something like \\.\C:\ or \\.\HarddiskVolume2\, or even \\?\GLOBALROOT\DosDevices\HarddiskVolume2\. Or it can be any mount point inside another volume! (Remember that feature inside Disk Management?) Moreover you can't even assume the colon and alternate data streams are even a thing on the file system - it's an NTFS feature. So you gotta query the file system name first. And if the file system is something else with its own special syntax you don't know, then in general you can't find the file name and strip the last component at all. All of which I think means it's impossible to figure out the prefix length without performing syscalls on the target system, and that the answer might vary if the mounts change at run time.

flangola7 3y ago
What is an alternate data stream?
- dataflow 3y ago
  C:\foo has a default (primary) data stream; the name of that stream is empty, so it's omitted entirely when writing the name. But the file can also have C:\foo:bar on NTFS. It's a different stream that's part of the original. (Look up "NTFS ADS" or just "NTFS streams".) These are often used to store information tied to a file that shouldn't affect the file contents.
  - KMag 3y ago
    In the late 1990s, there was a bug in MS IIS where if you requested http://example.com/page.php http://example.com/page.php , it would execute the PHP script, but if you requested http://example.com/page.php http://example.com/page.php: , it would give you the PHP source code. Even more than today, it was common to hard-code database connection details, including passwords, into the source code.
- Calzifer 3y ago
  Any data stream which is not the first one. Data stream is basically the file content and on NTFS a file can have more than one. In practice it is comparable to extended attributes in the Linux world but somewhat superior. But like extended attributes it doesn't seem to have too much real world use. The only use case for alternate data streams I can remember are the "this file was downloaded from the internet, do you really want to run it" warnings. In such cases the browser attached a standardized marker as alternate data stream to the file.
  - dataflow 3y ago
    It's used for other things too. Like modern file compression (compact /exe:lzx).
    - Alupis 3y ago
      And used to be used to hide malicious software back in the early days.
    - Calzifer 3y ago
      Are you sure we mean the same thing? You seem to talk about a specific command line argument of the compact command with a Windows typical (and IMO ugly) option style with '/' instead of '--' as option marker and ':' instead of '=' as option value separator. But that would not be directly related to ADS and I cannot imagine a good use case where the compact command should use ADS.
      - dataflow 3y ago
        Yes, look up WofCompressedData. It's the stream name ultimately used by that command.
        Calzifer 3y ago
        Thanks, with this keyword I found https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=102597 https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=10... In context of ADS the first thing I imagined was storing the compressed and uncompressed file alongside. (which is rather silly, why compress at all) This use case is also kinda strange. Have the compressed content as ADS, the primary contend filled with 0 as sparse and fill it when needed/accessed. :/
cryptonector 3y ago
> Moreover you can't even assume the colon and alternate data streams are even a thing on the file system - it's an NTFS feature. So you gotta query the file system name first. And if the file system is something else with its own special syntax you don't know, then in general you can't find the file name and strip the last component at all. If the :stream syntax is not FS-specific then you can parse the data stream name out statically in almost every case. Yes, you have to work out the prefix, but you can mostly do that statically too, I think: > In fact this might not even be the worst thing - it's even worse than this because you first need to strip off the prefix that represent the volume (like C:\) before you can look for a colon. But the prefix can be something like \\.\C:\ or \\.\HarddiskVolume2\, or even \\?\GLOBALROOT\DosDevices\HarddiskVolume2\. Or it can be any mount point inside another volume! Which I think means it's impossible to figure out the prefix length without performing syscalls on the target system, and that the answer might vary if the mounts change at run time. The prefix of `\\.\C:\Foo:Bar` is `\\.\C:` as `C:` couldn't be a file name. The prefix of `\\.\HarddiskVolume2\Foo:Bar` is `\\.\HarddiskVolume2` because the volume name ends at the backslash. The prefix of `\\?\GLOBALROOT\DosDevices\HarddiskVolume2\Foo:Bar`... can be harder to determine but it doesn't matter because clearly there is no letter drive name in sight since a letter drive name would be... a single letter, but if the volume name were a single letter then it might require using system calls to resolve it (`\\?\GLOBALROOT\DosDevices\X\Y:Z\A:B` is harder to parse because X might be the volume name, or maybe Y: might be the letter drive and X might be part of the path prefix).
- dataflow 3y ago
  > If the :stream syntax is not FS-specific It is, I believe, as I alluded to in the comment. > `\\?\GLOBALROOT\DosDevices\X\Y:Z\A:B` is harder to parse As in, this is impossible to do statically in the general case - those names aren't guaranteed to look like that. See the note I had added about mount points. Remember C:\mnt can itself be the mount point of a volume instead of a drive letter. (Junctions present a similar problem, but at least for those, you can make an argument that they're intended to look like physical folders, and treat them similarly. With mount points, you might not have that intention - you might be just trying to go over 26 drive letters.)
ChrisSD 3y ago
A stream name is somewhat more limited than that: > All Unicode characters are legal in a streamname component except the following: > * The characters \ / : > * Control character 0x00. > * A streamname MUST be no more than 255 characters in length. > > A zero-length streamname denotes the default stream. https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/5953f072-b28c-4fbf-ae50-09b0173317b9 https://learn.microsoft.com/en-us/openspecs/windows_protocol...
- dataflow 3y ago
  Oops, thanks for the correction! I must've seen this with other characters (most likely double quotes) and not realized slashes and backslashes are an exception. Though ironically that still doesn't help you strip the last component, since it could still be a volume mount point. Like you don't want C:\mnt\..\foo to suddenly become C:\foo, just like how you don't want \\.\Server\Share1\..\Share2 to become \\.\Server\Share2, or for \\.\C:\..\HarddiskVolume1 to become \\.\HarddiskVolume1, etc.
andirk 3y ago
Those back slashes were annoying to me since before I knew what they were doing. Whereas the forward slash always made sense.

flangola7 3y ago

What is an alternate data stream?

dataflow 3y ago
C:\foo has a default (primary) data stream; the name of that stream is empty, so it's omitted entirely when writing the name. But the file can also have C:\foo:bar on NTFS. It's a different stream that's part of the original. (Look up "NTFS ADS" or just "NTFS streams".) These are often used to store information tied to a file that shouldn't affect the file contents.
- KMag 3y ago
  In the late 1990s, there was a bug in MS IIS where if you requested http://example.com/page.php http://example.com/page.php , it would execute the PHP script, but if you requested http://example.com/page.php http://example.com/page.php: , it would give you the PHP source code. Even more than today, it was common to hard-code database connection details, including passwords, into the source code.
Calzifer 3y ago
Any data stream which is not the first one. Data stream is basically the file content and on NTFS a file can have more than one. In practice it is comparable to extended attributes in the Linux world but somewhat superior. But like extended attributes it doesn't seem to have too much real world use. The only use case for alternate data streams I can remember are the "this file was downloaded from the internet, do you really want to run it" warnings. In such cases the browser attached a standardized marker as alternate data stream to the file.
- dataflow 3y ago
  It's used for other things too. Like modern file compression (compact /exe:lzx).
  - Alupis 3y ago
    And used to be used to hide malicious software back in the early days.
  - Calzifer 3y ago
    Are you sure we mean the same thing? You seem to talk about a specific command line argument of the compact command with a Windows typical (and IMO ugly) option style with '/' instead of '--' as option marker and ':' instead of '=' as option value separator. But that would not be directly related to ADS and I cannot imagine a good use case where the compact command should use ADS.
    - dataflow 3y ago
      Yes, look up WofCompressedData. It's the stream name ultimately used by that command.
      - Calzifer 3y ago
        Thanks, with this keyword I found https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=102597 https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=10... In context of ADS the first thing I imagined was storing the compressed and uncompressed file alongside. (which is rather silly, why compress at all) This use case is also kinda strange. Have the compressed content as ADS, the primary contend filled with 0 as sparse and fill it when needed/accessed. :/

cryptonector 3y ago

> Moreover you can't even assume the colon and alternate data streams are even a thing on the file system - it's an NTFS feature. So you gotta query the file system name first. And if the file system is something else with its own special syntax you don't know, then in general you can't find the file name and strip the last component at all. If the :stream syntax is not FS-specific then you can parse the data stream name out statically in almost every case. Yes, you have to work out the prefix, but you can mostly do that statically too, I think: > In fact this might not even be the worst thing - it's even worse than this because you first need to strip off the prefix that represent the volume (like C:\) before you can look for a colon. But the prefix can be something like \\.\C:\ or \\.\HarddiskVolume2\, or even \\?\GLOBALROOT\DosDevices\HarddiskVolume2\. Or it can be any mount point inside another volume! Which I think means it's impossible to figure out the prefix length without performing syscalls on the target system, and that the answer might vary if the mounts change at run time. The prefix of `\\.\C:\Foo:Bar` is `\\.\C:` as `C:` couldn't be a file name. The prefix of `\\.\HarddiskVolume2\Foo:Bar` is `\\.\HarddiskVolume2` because the volume name ends at the backslash. The prefix of `\\?\GLOBALROOT\DosDevices\HarddiskVolume2\Foo:Bar`... can be harder to determine but it doesn't matter because clearly there is no letter drive name in sight since a letter drive name would be... a single letter, but if the volume name were a single letter then it might require using system calls to resolve it (`\\?\GLOBALROOT\DosDevices\X\Y:Z\A:B` is harder to parse because X might be the volume name, or maybe Y: might be the letter drive and X might be part of the path prefix).

dataflow 3y ago
> If the :stream syntax is not FS-specific It is, I believe, as I alluded to in the comment. > `\\?\GLOBALROOT\DosDevices\X\Y:Z\A:B` is harder to parse As in, this is impossible to do statically in the general case - those names aren't guaranteed to look like that. See the note I had added about mount points. Remember C:\mnt can itself be the mount point of a volume instead of a drive letter. (Junctions present a similar problem, but at least for those, you can make an argument that they're intended to look like physical folders, and treat them similarly. With mount points, you might not have that intention - you might be just trying to go over 26 drive letters.)

ChrisSD 3y ago

A stream name is somewhat more limited than that: > All Unicode characters are legal in a streamname component except the following: > * The characters \ / : > * Control character 0x00. > * A streamname MUST be no more than 255 characters in length. > > A zero-length streamname denotes the default stream. https://learn.microsoft.com/en-us/openspecs/windows_protocols/ms-fscc/5953f072-b28c-4fbf-ae50-09b0173317b9 https://learn.microsoft.com/en-us/openspecs/windows_protocol...

dataflow 3y ago
Oops, thanks for the correction! I must've seen this with other characters (most likely double quotes) and not realized slashes and backslashes are an exception. Though ironically that still doesn't help you strip the last component, since it could still be a volume mount point. Like you don't want C:\mnt\..\foo to suddenly become C:\foo, just like how you don't want \\.\Server\Share1\..\Share2 to become \\.\Server\Share2, or for \\.\C:\..\HarddiskVolume1 to become \\.\HarddiskVolume1, etc.

andirk 3y ago

Those back slashes were annoying to me since before I knew what they were doing. Whereas the forward slash always made sense.