5 ms·

Are PE executable files special in Windows, or could this bug have been hit with normal IO?

by Athas 9y ago

mehrdadn 9y ago
I'm not sure in what manner they would need to be special for this, but I would expect the significant part was the fact that it was memory-mapped (which isn't normal I/O), not that they were PE files. Edit (in response to comments): Actually, I don't think it has anything to do with PE files being optimized in some special fashion, because they say loading via LoadLibrary(Ex) can cause this issue too, and those are very much user-mode constructs, and hence can't mess with the caching/paging behavior (which are lower, in the kernel). AFAIK the only kernel-mode component to them is to just create a file mapping (NtCreateSection) and after that all they do is a bunch of user-mode juggling (fixing up relocations, loading other libraries that are depended on, calling entrypoints, etc.).
- ChrisSD 9y ago
  Indeed but being loaded as code seems important as well. It sounds like the two used together, in certain extreme conditions, causes the issue.
- blattimwind 9y ago
  Since it's only observed by execution, its entirely possible that the PE loader has been optimized in a low-level fashion leading to bugs that are not exposed in general I/O facilities.
  - deleted 9y ago
    [deleted]
  - gmueckl 9y ago
    My reading is that PE images are memory mapped into RAM instead of being read from disk. This is why they require write locks on the files during execution: the program image is backed from the corresponding file instead of the normal page file. So my speculation is that this bug is a race condition between the cleanup of a dirty memory mapped section of a file and the reading of the same file contents in a separate mapping.
- vardump 9y ago
  > memory-mapped (which isn't normal I/O) Memory mapped I/O is very much normal I/O. Normal, as in "common" or "usual".
  - mehrdadn 9y ago
    >> memory-mapped (which isn't normal I/O) > Memory mapped I/O is very much normal I/O. Normal, as in "common" or "usual". You know well what I meant.
    - vardump 9y ago
      What you said left an impression memory mapped I/O is somehow unusual or special, while it's actually a fairly common type of I/O. So while I did know what you meant, I found your comment misleading to someone who isn't familiar with the subject. Too many programmers misunderstand and are scared of memory mapping as it is.
- brucedawson 9y ago
  Both parts are special. As the post says, there are three conditions: 1) Write the file with memory-mapped I/O 2) Read the file as a PE file (CreateProcess, LoadLibrary, etc., I'm not positive of the exact requirements but it's something special that I believe only PE files can hit) 3) Very heavy load

Are PE executable files special in Windows, or could this bug have been hit with normal IO?

by Athas 9y ago

Are PE executable files special in Windows, or could this bug have been hit with normal IO?

I'm not sure in what manner they would need to be special for this, but I would expect the significant part was the fact that it was memory-mapped (which isn't normal I/O), not that they were PE files. Edit (in response to comments): Actually, I don't think it has anything to do with PE files being optimized in some special fashion, because they say loading via LoadLibrary(Ex) can cause this issue too, and those are very much user-mode constructs, and hence can't mess with the caching/paging behavior (which are lower, in the kernel). AFAIK the only kernel-mode component to them is to just create a file mapping (NtCreateSection) and after that all they do is a bunch of user-mode juggling (fixing up relocations, loading other libraries that are depended on, calling entrypoints, etc.).

ChrisSD 9y ago
Indeed but being loaded as code seems important as well. It sounds like the two used together, in certain extreme conditions, causes the issue.
blattimwind 9y ago
Since it's only observed by execution, its entirely possible that the PE loader has been optimized in a low-level fashion leading to bugs that are not exposed in general I/O facilities.
- deleted 9y ago
  [deleted]
- gmueckl 9y ago
  My reading is that PE images are memory mapped into RAM instead of being read from disk. This is why they require write locks on the files during execution: the program image is backed from the corresponding file instead of the normal page file. So my speculation is that this bug is a race condition between the cleanup of a dirty memory mapped section of a file and the reading of the same file contents in a separate mapping.
vardump 9y ago
> memory-mapped (which isn't normal I/O) Memory mapped I/O is very much normal I/O. Normal, as in "common" or "usual".
- mehrdadn 9y ago
  >> memory-mapped (which isn't normal I/O) > Memory mapped I/O is very much normal I/O. Normal, as in "common" or "usual". You know well what I meant.
  - vardump 9y ago
    What you said left an impression memory mapped I/O is somehow unusual or special, while it's actually a fairly common type of I/O. So while I did know what you meant, I found your comment misleading to someone who isn't familiar with the subject. Too many programmers misunderstand and are scared of memory mapping as it is.
brucedawson 9y ago
Both parts are special. As the post says, there are three conditions: 1) Write the file with memory-mapped I/O 2) Read the file as a PE file (CreateProcess, LoadLibrary, etc., I'm not positive of the exact requirements but it's something special that I believe only PE files can hit) 3) Very heavy load