3 ms·

Years ago when I used Dropbox on Windows, if would poll every single file on startup, which in the days of spinning disks rendered the machine unusable for seve

Years ago when I used Dropbox on Windows, if would poll every single file on startup, which in the days of spinning disks rendered the machine unusable for several minutes - does Dropbox on Windows still do this? Also, after the startup phase, IIRC Dropbox used the `FindFirstChangeNotification` Windows API function, which can miss changes under heavy load - do you still use this method, or have you moved to something else, such as using the NTFS USN journal, or a file system minifilter driver?

sujayakar 7y ago
we still have to check every file on startup, since we use the `ReadDirectoryChangesW` call on windows, but it should be less expensive on nucleus. but, now that we've rolled out nucleus, we can start exploring going beyond sync engine classic feature parity, like using the USN journal on windows.
- GordonS 7y ago
  For NTFS volumes (basically all Windows volumes), you could use the change journal to eliminate that painfully expensive startup check of all files - that would be huge for Windows users. Also, have you previously explored the possibility of using the NTFS change journal, and if so, what challenges did you face? (I don't mean to come across negative; I'm genuinely interested to learn why you didn't use this a decade ago, since there must be a reason).
  - rbtying 7y ago
    As I recall, this was investigated at one point on Sync Engine Classic and is definitely a thing that folks were thinking about for Nucleus, for pretty much exactly this reason. The biggest challenges with this kind of work (not specific to the USN journal) are in the lack of reliable cross-platform support for features which can be shimmed to look similar. The more unique the code path, the harder it is to test en masse. It's also the case that the execution environment on Windows machines tends to be very diverse due to a long history of backwards compatibility and the ability to set complex domain policies -- Dropbox strives for a good user experience, and "go talk to IT to have them change this setting" is rarely one of those. disclaimer: worked on sync at Dropbox; don't work on it anymore; don't have current context

sujayakar 7y ago

we still have to check every file on startup, since we use the `ReadDirectoryChangesW` call on windows, but it should be less expensive on nucleus. but, now that we've rolled out nucleus, we can start exploring going beyond sync engine classic feature parity, like using the USN journal on windows.

GordonS 7y ago
For NTFS volumes (basically all Windows volumes), you could use the change journal to eliminate that painfully expensive startup check of all files - that would be huge for Windows users. Also, have you previously explored the possibility of using the NTFS change journal, and if so, what challenges did you face? (I don't mean to come across negative; I'm genuinely interested to learn why you didn't use this a decade ago, since there must be a reason).
- rbtying 7y ago
  As I recall, this was investigated at one point on Sync Engine Classic and is definitely a thing that folks were thinking about for Nucleus, for pretty much exactly this reason. The biggest challenges with this kind of work (not specific to the USN journal) are in the lack of reliable cross-platform support for features which can be shimmed to look similar. The more unique the code path, the harder it is to test en masse. It's also the case that the execution environment on Windows machines tends to be very diverse due to a long history of backwards compatibility and the ability to set complex domain policies -- Dropbox strives for a good user experience, and "go talk to IT to have them change this setting" is rarely one of those. disclaimer: worked on sync at Dropbox; don't work on it anymore; don't have current context