3 ms·

There are two patches linked in the OP that switch to using F_FULLFSYNC on OSX. The OP says that people are still encountering db corruption even on branches wi

There are two patches linked in the OP that switch to using F_FULLFSYNC on OSX. The OP says that people are still encountering db corruption even on branches with these fixes. https://github.com/sipa/bitcoin/commit/b28d8b423bddc860c5858a9df2982ce825835350 https://github.com/sipa/bitcoin/commit/b28d8b423bddc860c5858... https://github.com/gmaxwell/bitcoin/commit/e7bad10c12ce9b5d424ac273c1c977b88469d46c https://github.com/gmaxwell/bitcoin/commit/e7bad10c12ce9b5d4...

pudquick 13y ago
We'll, I'm glad someone apparently IS reading :) But again - I'd point to the work of other longstanding database projects that are available on OS X as a source of "how we ensured data correctness".
pudquick 13y ago
Additional reply here since my other is too old to edit: Something a tester experiencing corruption at startup may want to try is using the 'purge' command from the Terminal. While a restart will indeed trigger caching files back to disk before the pending restart of the system, the 'purge' command will simulate a "cold boot"-like empty disk buffer by dropping existing file caches. https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/purge.8.html https://developer.apple.com/library/mac/documentation/Darwin... If using the command solves the problem of the database corruption without a restart, then you're definitely suffering from disk cache. Easy to confirm. Additionally, since this is an error on boot up of LevelDB, this is probably in regards to file reading - especially since on second bootup after a restart, no error is detected. F_FULLSYNC is for ensuring that a particular file's changes are 'fully written to disk' ... ... but the cache works both ways. A program could also end up reading the disk cache (which sounds like what's happening here), unless you used F_NOCACHE or F_GLOBAL_NOCACHE. Mind you, these don't prevent the accessing of files already in disk cache - they prevent a file from getting into the disk cache in the first place. http://lists.apple.com/archives/darwin-dev/2009/Oct/msg00165.html http://lists.apple.com/archives/darwin-dev/2009/Oct/msg00165... (Disclaimer: I can't think of a situation where the disk cache would get out of sync with the on-disk version of the file if F_FULLSYNC is used when accessed by a subsequent launch of a program except for in the case of faulty RAM on a machine which flipped bits. Your average file operation done by your average application in OS X isn't performing a checksum of data and generally wouldn't notice a single bit flip. It would be interesting to see which of these machines are using ECC RAM.)

pudquick 13y ago

We'll, I'm glad someone apparently IS reading :) But again - I'd point to the work of other longstanding database projects that are available on OS X as a source of "how we ensured data correctness".

pudquick 13y ago

Additional reply here since my other is too old to edit: Something a tester experiencing corruption at startup may want to try is using the 'purge' command from the Terminal. While a restart will indeed trigger caching files back to disk before the pending restart of the system, the 'purge' command will simulate a "cold boot"-like empty disk buffer by dropping existing file caches. https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man8/purge.8.html https://developer.apple.com/library/mac/documentation/Darwin... If using the command solves the problem of the database corruption without a restart, then you're definitely suffering from disk cache. Easy to confirm. Additionally, since this is an error on boot up of LevelDB, this is probably in regards to file reading - especially since on second bootup after a restart, no error is detected. F_FULLSYNC is for ensuring that a particular file's changes are 'fully written to disk' ... ... but the cache works both ways. A program could also end up reading the disk cache (which sounds like what's happening here), unless you used F_NOCACHE or F_GLOBAL_NOCACHE. Mind you, these don't prevent the accessing of files already in disk cache - they prevent a file from getting into the disk cache in the first place. http://lists.apple.com/archives/darwin-dev/2009/Oct/msg00165.html http://lists.apple.com/archives/darwin-dev/2009/Oct/msg00165... (Disclaimer: I can't think of a situation where the disk cache would get out of sync with the on-disk version of the file if F_FULLSYNC is used when accessed by a subsequent launch of a program except for in the case of faulty RAM on a machine which flipped bits. Your average file operation done by your average application in OS X isn't performing a checksum of data and generally wouldn't notice a single bit flip. It would be interesting to see which of these machines are using ECC RAM.)