5 ms·

I don’t think this is actually a corruption issue. I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot

I don’t think this is actually a corruption issue. I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean. What it seems to do is trigger a consistency check for the next reboot and some alarmist notifications. This is probably a simple check assertion somewhere which is seeing unexpected data passing through a call. That is a problem but significantly less of one than actual corruption. If anyone has evidence to the contrary and steps to reproduce it I’d be interested to try them.

sterlind 6y ago
Not doubting you, but the repro video in the bleepingcomputer article showed a green screen of death during chkdsk. Maybe there's something finicky?
- uncledave 6y ago
  I ran the test case and then chkdsk 5 times each on 1903, 2004 and 20H2 and it came back clean every time. It’s definitely possible I missed something but until it’s consistently reproducible then I’m on the fence. One complaint is I’d expect something at the level of MFT corruption to trigger a bugcheck (panic). Doesn’t seem safe behaviour to let the system carry on with a corrupt MFT.
londons_explore 6y ago
The article suggests a .url file could be found empty after the reboot. Perhaps the check assertion causes the filesystem to become read-only or discard the write cache or some other side effects that could lead to corruption? If it's something along these lines, it may not be reproducible in a VM (where the host is responsible for waiting around for flushing delays)
- uncledave 6y ago
  That’s certainly a possibility. Will go and try it. The MFT writes are transactional in theory from my understanding and small files are stored in the MFT and shortcuts are small files. Could be that the MFT is not flushed to disk after that event? That would suggest anything after the event would be broken. Also in line with my worries that this should be a bugcheck/panic rather than a soft assertion.
shakna 6y ago
> I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean. That is very frequently the case with these kinds of filesystem bugs. You need actual hardware, not a VM. FWIW, it does seem to be reproducible on hardware.
- uncledave 6y ago
  I’ve just tried it on my native T470 backup machine quick and same result. No action for chkdsk. This was a single test so the veracity of the outcome is somewhat debatable.
- rndgermandude 6y ago
  I don't see why this bug would work any different in a VM than on a real bare-metal OS or disk? It's an NTFS issue, i.e. a software issue. NTFS doesn't care if the underlying disk is virtual or bare metal. In fact, it kinda runs on virtual disks already even on bare metal, i.e. disk partitions (maybe even with a layer of encryption in between). There could be potential issues of course where you could get NTFS to trigger some hardware bugs, but that doesn't seem to be the case here, instead it is about accessing NTFS $I30 attributes.
  - steerablesafe 6y ago
    I'm not a filesystem developer, but assuming the corruption is caused by a race condition in the filesystem driver the difference can come down to many aspects: * Different timings accessing a raw device compared to a virtual drive. * The presence of the host scheduler can also alter timings. * CPU configuration on the guest that's not typical on a bare metal (single CPU VM?) It's not at all obvious that a bug must manifest the same way on a VM than on bare metal, although probably more often than not that's the case. If filesystem testing is also done on VMs then that would explain how this slipped through QC.
    - rndgermandude 6y ago
      Sure, there are race bugs that can in theory be affected by things like that. This seems to be a deterministic bug tho. Open "special" file path to an $I30 attribute stream, crash and burn.
stallman_is_ok 6y ago
I believe you are correct. I could not see this cause filesystem corruption. From looking around in ntfs.sys, it just seem to set a 'this is corrupt' flags on that code path.
steffanA 6y ago
Did not test this on physical hardware but in a one of the tests on a VM, it toasted my install. Tested it three times. Two of the tests, chkdsk ran and Windows booted after. Third time, after chkdsk it went into Windows startup repair, and ultimately led to a green screen where it wouldn't launch boot further.

sterlind 6y ago

Not doubting you, but the repro video in the bleepingcomputer article showed a green screen of death during chkdsk. Maybe there's something finicky?

uncledave 6y ago
I ran the test case and then chkdsk 5 times each on 1903, 2004 and 20H2 and it came back clean every time. It’s definitely possible I missed something but until it’s consistently reproducible then I’m on the fence. One complaint is I’d expect something at the level of MFT corruption to trigger a bugcheck (panic). Doesn’t seem safe behaviour to let the system carry on with a corrupt MFT.

londons_explore 6y ago

The article suggests a .url file could be found empty after the reboot. Perhaps the check assertion causes the filesystem to become read-only or discard the write cache or some other side effects that could lead to corruption? If it's something along these lines, it may not be reproducible in a VM (where the host is responsible for waiting around for flushing delays)

uncledave 6y ago
That’s certainly a possibility. Will go and try it. The MFT writes are transactional in theory from my understanding and small files are stored in the MFT and shortcuts are small files. Could be that the MFT is not flushed to disk after that event? That would suggest anything after the event would be broken. Also in line with my worries that this should be a bugcheck/panic rather than a soft assertion.

shakna 6y ago

> I spent a lot of time with this yesterday analysing it in a VM and I couldn’t find any cases where on reboot it caused chkdsk to actually take any actions. It always found the file system clean. That is very frequently the case with these kinds of filesystem bugs. You need actual hardware, not a VM. FWIW, it does seem to be reproducible on hardware.

uncledave 6y ago
I’ve just tried it on my native T470 backup machine quick and same result. No action for chkdsk. This was a single test so the veracity of the outcome is somewhat debatable.
rndgermandude 6y ago
I don't see why this bug would work any different in a VM than on a real bare-metal OS or disk? It's an NTFS issue, i.e. a software issue. NTFS doesn't care if the underlying disk is virtual or bare metal. In fact, it kinda runs on virtual disks already even on bare metal, i.e. disk partitions (maybe even with a layer of encryption in between). There could be potential issues of course where you could get NTFS to trigger some hardware bugs, but that doesn't seem to be the case here, instead it is about accessing NTFS $I30 attributes.
- steerablesafe 6y ago
  I'm not a filesystem developer, but assuming the corruption is caused by a race condition in the filesystem driver the difference can come down to many aspects: * Different timings accessing a raw device compared to a virtual drive. * The presence of the host scheduler can also alter timings. * CPU configuration on the guest that's not typical on a bare metal (single CPU VM?) It's not at all obvious that a bug must manifest the same way on a VM than on bare metal, although probably more often than not that's the case. If filesystem testing is also done on VMs then that would explain how this slipped through QC.
  - rndgermandude 6y ago
    Sure, there are race bugs that can in theory be affected by things like that. This seems to be a deterministic bug tho. Open "special" file path to an $I30 attribute stream, crash and burn.

stallman_is_ok 6y ago

I believe you are correct. I could not see this cause filesystem corruption. From looking around in ntfs.sys, it just seem to set a 'this is corrupt' flags on that code path.

steffanA 6y ago

Did not test this on physical hardware but in a one of the tests on a VM, it toasted my install. Tested it three times. Two of the tests, chkdsk ran and Windows booted after. Third time, after chkdsk it went into Windows startup repair, and ultimately led to a green screen where it wouldn't launch boot further.