5 ms·

What is an alternate data stream?

dataflow 4y ago
C:\foo has a default (primary) data stream; the name of that stream is empty, so it's omitted entirely when writing the name. But the file can also have C:\foo:bar on NTFS. It's a different stream that's part of the original. (Look up "NTFS ADS" or just "NTFS streams".) These are often used to store information tied to a file that shouldn't affect the file contents.
- KMag 4y ago
  In the late 1990s, there was a bug in MS IIS where if you requested http://example.com/page.php http://example.com/page.php , it would execute the PHP script, but if you requested http://example.com/page.php http://example.com/page.php: , it would give you the PHP source code. Even more than today, it was common to hard-code database connection details, including passwords, into the source code.
Calzifer 4y ago
Any data stream which is not the first one. Data stream is basically the file content and on NTFS a file can have more than one. In practice it is comparable to extended attributes in the Linux world but somewhat superior. But like extended attributes it doesn't seem to have too much real world use. The only use case for alternate data streams I can remember are the "this file was downloaded from the internet, do you really want to run it" warnings. In such cases the browser attached a standardized marker as alternate data stream to the file.
- dataflow 4y ago
  It's used for other things too. Like modern file compression (compact /exe:lzx).
  - Alupis 4y ago
    And used to be used to hide malicious software back in the early days.
  - Calzifer 4y ago
    Are you sure we mean the same thing? You seem to talk about a specific command line argument of the compact command with a Windows typical (and IMO ugly) option style with '/' instead of '--' as option marker and ':' instead of '=' as option value separator. But that would not be directly related to ADS and I cannot imagine a good use case where the compact command should use ADS.
    - dataflow 4y ago
      Yes, look up WofCompressedData. It's the stream name ultimately used by that command.
      - Calzifer 4y ago
        Thanks, with this keyword I found https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=102597 https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=10... In context of ADS the first thing I imagined was storing the compressed and uncompressed file alongside. (which is rather silly, why compress at all) This use case is also kinda strange. Have the compressed content as ADS, the primary contend filled with 0 as sparse and fill it when needed/accessed. :/
- nereye 4y ago
  That standardized marker is also known as the 'Mark of the Web' (MOTW) in case you want to search for more details about it. In general, there is a tool which comes with the SysInternals suite that allows you to see which files have streams and their size: https://learn.microsoft.com/en-us/sysinternals/downloads/streams https://learn.microsoft.com/en-us/sysinternals/downloads/str...
- ajcoll5 4y ago
  If you have macOS clients connecting to an SMB file share hosed by a Windows server they use alternate data streams to store resource forks - like fonts. Makes for a fun 'oh shit' moment if you go to zip up files on Windows to archive, then realize you're missing data when you later unzip as most compression applications don't keep them.
  - LoganDark 4y ago
    > they use alternate data streams to store resource forks - like fonts macOS stores fonts in resource forks? I'm confused, what use does this have and what happens when you accidentally miss them?
    - skissane 4y ago
      > macOS stores fonts in resource forks? I'm confused, what use does this have and what happens when you accidentally miss them? Classic MacOS considers fonts to be a type of resource, and hence stores them in the resource fork. Contemporary macOS fonts are just ordinary files with a data fork only. I think grandparent is talking about the 1990s, although some of those machines remained in active use through the first few years of this century. Windows originally considered fonts to be a type of resource too – the original bitmap fonts used with Windows 1.x-3.x are stored as a resource–except unlike MacOS it embeds resources into EXE/DLL file data instead of putting them in a fork. In fact, a .FON file containing a Windows bitmap font is just an EXE with no code, only resources. Nobody really uses this any more, everything is TrueType now and TrueType uses its own file format not resources, but Windows still supports the old bitmap fonts for any legacy apps which still use them.
      - LoganDark 4y ago
        I originally thought you meant "macOS takes random fonts, stuffs them in resource forks for other non-font files, then bad things happen if the resource forks are ever lost" which makes zero sense to me. Anyway... so macOS fonts themselves were made of resource forks and therefore trying to transfer fonts themselves across a non-resource-fork-supporting network share will fail? As in, the resource forks were needed in order to use the font file?
- jra_samba 4y ago
  Oh contraire :-) .Alternate data streams are widely used by virus writers and spies using them to exfiltrate data from foreign (to the spy) government and corporate Windows IT systems. You think I jest ? Look up the leaked source code for the US government spy tooling. They hide data to be exfiltrated in an ADS on the root directory of the share :-). I finally realized ADS were the mother of bad ideas when Ted Tso responded to me asking why I couldn't have them in Linux for the umpteenth time by showing me a Windows task manager screenshot of Myfile.txt as an actively running process. If the ADS ends in .exe then Windows will happily run it :-).

C:\foo has a default (primary) data stream; the name of that stream is empty, so it's omitted entirely when writing the name. But the file can also have C:\foo:bar on NTFS. It's a different stream that's part of the original. (Look up "NTFS ADS" or just "NTFS streams".) These are often used to store information tied to a file that shouldn't affect the file contents.

KMag 4y ago
In the late 1990s, there was a bug in MS IIS where if you requested http://example.com/page.php http://example.com/page.php , it would execute the PHP script, but if you requested http://example.com/page.php http://example.com/page.php: , it would give you the PHP source code. Even more than today, it was common to hard-code database connection details, including passwords, into the source code.

Calzifer 4y ago

Any data stream which is not the first one. Data stream is basically the file content and on NTFS a file can have more than one. In practice it is comparable to extended attributes in the Linux world but somewhat superior. But like extended attributes it doesn't seem to have too much real world use. The only use case for alternate data streams I can remember are the "this file was downloaded from the internet, do you really want to run it" warnings. In such cases the browser attached a standardized marker as alternate data stream to the file.

dataflow 4y ago
It's used for other things too. Like modern file compression (compact /exe:lzx).
- Alupis 4y ago
  And used to be used to hide malicious software back in the early days.
- Calzifer 4y ago
  Are you sure we mean the same thing? You seem to talk about a specific command line argument of the compact command with a Windows typical (and IMO ugly) option style with '/' instead of '--' as option marker and ':' instead of '=' as option value separator. But that would not be directly related to ADS and I cannot imagine a good use case where the compact command should use ADS.
  - dataflow 4y ago
    Yes, look up WofCompressedData. It's the stream name ultimately used by that command.
    - Calzifer 4y ago
      Thanks, with this keyword I found https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=102597 https://devblogs.microsoft.com/oldnewthing/20190618-00/?p=10... In context of ADS the first thing I imagined was storing the compressed and uncompressed file alongside. (which is rather silly, why compress at all) This use case is also kinda strange. Have the compressed content as ADS, the primary contend filled with 0 as sparse and fill it when needed/accessed. :/
nereye 4y ago
That standardized marker is also known as the 'Mark of the Web' (MOTW) in case you want to search for more details about it. In general, there is a tool which comes with the SysInternals suite that allows you to see which files have streams and their size: https://learn.microsoft.com/en-us/sysinternals/downloads/streams https://learn.microsoft.com/en-us/sysinternals/downloads/str...
ajcoll5 4y ago
If you have macOS clients connecting to an SMB file share hosed by a Windows server they use alternate data streams to store resource forks - like fonts. Makes for a fun 'oh shit' moment if you go to zip up files on Windows to archive, then realize you're missing data when you later unzip as most compression applications don't keep them.
- LoganDark 4y ago
  > they use alternate data streams to store resource forks - like fonts macOS stores fonts in resource forks? I'm confused, what use does this have and what happens when you accidentally miss them?
  - skissane 4y ago
    > macOS stores fonts in resource forks? I'm confused, what use does this have and what happens when you accidentally miss them? Classic MacOS considers fonts to be a type of resource, and hence stores them in the resource fork. Contemporary macOS fonts are just ordinary files with a data fork only. I think grandparent is talking about the 1990s, although some of those machines remained in active use through the first few years of this century. Windows originally considered fonts to be a type of resource too – the original bitmap fonts used with Windows 1.x-3.x are stored as a resource–except unlike MacOS it embeds resources into EXE/DLL file data instead of putting them in a fork. In fact, a .FON file containing a Windows bitmap font is just an EXE with no code, only resources. Nobody really uses this any more, everything is TrueType now and TrueType uses its own file format not resources, but Windows still supports the old bitmap fonts for any legacy apps which still use them.
    - LoganDark 4y ago
      I originally thought you meant "macOS takes random fonts, stuffs them in resource forks for other non-font files, then bad things happen if the resource forks are ever lost" which makes zero sense to me. Anyway... so macOS fonts themselves were made of resource forks and therefore trying to transfer fonts themselves across a non-resource-fork-supporting network share will fail? As in, the resource forks were needed in order to use the font file?
jra_samba 4y ago
Oh contraire :-) .Alternate data streams are widely used by virus writers and spies using them to exfiltrate data from foreign (to the spy) government and corporate Windows IT systems. You think I jest ? Look up the leaked source code for the US government spy tooling. They hide data to be exfiltrated in an ADS on the root directory of the share :-). I finally realized ADS were the mother of bad ideas when Ted Tso responded to me asking why I couldn't have them in Linux for the umpteenth time by showing me a Windows task manager screenshot of Myfile.txt as an actively running process. If the ADS ends in .exe then Windows will happily run it :-).