4 ms·
I am also using checksums to detect bit-rot, but in order to tie them to the files they are stored in extended attributes of the files. Thus they do not depend
by adrian_b 4y ago
I am also using checksums to detect bit-rot, but in order to tie them to the files they are stored in extended attributes of the files. Thus they do not depend on the inode numbers.
OpenBSD also supports extended file attributes, so using them should be possible.
Using extended attributes on Linux or FreeBSD requires a few precautions, because there are still various copying/movement/archiving CLI commands or GUI applications that ignore the extended attributes and also some file systems that do not implement extended attributes, e.g. tmpfs on Linux (which supports only certain kinds of system extended attributes, not those defined by the users) or all not extremely new versions of NFS (only NFSv4 in Linux 5.9 or newer supports xattr, unlike samba, which has supported them for decades, mapping them correctly between different file systems, e.g. XFS on Linux to UFS on FreeBSD), so copying a file via those file systems would lose silently the extended attributes of the files.
The extended file attributes have been introduced in 1989, in HPFS for OS/2 version 1.2, and they have been brought to UNIX in XFS, in 1993.
30 years later, it is annoying to see that there are still some programs which pretend to make file copies or file archives, but which can lose the extended file attributes, without any warnings or errors.
- sdadams 4y agoOpenBSD removed support for extended attributes.
- minusf 4y agonot a 1:1 equivalent of the "classic" extended attr but there are special flags on files. https://man.openbsd.org/chflags https://man.openbsd.org/chflags
- adrian_b 4y agoI suppose that this has been done to ensure less work for the OpenBSD maintainers, but I do not consider it as a wise decision. Extended file attributes can be used to implement a large number of useful things, many of them enhancing security, e.g. access-control lists. However, IIRC OpenBSD has chosen to also not implement ACLs. In any case regardless how useful or not useful extended file attributes are considered to be, deciding to not implement them in the main file system used by an operating system has the immediate consequence of disqualifying this operating system for the use on a file server (a.k.a. NAS), which is an application domain where the *BSD operating systems have been traditionally very good. The reason is that whenever such a NAS would have Windows, Linux or FreeBSD clients, transiting any file through that NAS would potentially lose data. In general, in the documentation of any file system, the lack of support for features like extended attributes or access-control lists must be displayed very prominently, to warn any potential users about the risk of data loss during copy operations (because the file copy commands are usually stupid enough to not inform the users whenever they are stripping file metadata, so such a loss may be discovered only when it is too late).
- sdadams 4y agoIn another project of mine I considered using extended attributes to tag files into categories. With extended attributes not being universally supported and easily overlooked the conclusion I came to was that I should store that information in a database file. I didn't want to one day lose my (manually assigned) tags to an erroneous move command. Anything non-native to OpenBSD can still be stored in muxfs as an archive file. You would need to ensure to pass the right arguments to the archiver to preserve the attributes in this case.
- adrian_b 4y agoAmong the programs with good support for extended file attributes are any archivers based on the FreeBSD libarchive (e.g. bsdtar), rsync, samba and the Linux coreutils. However, when instead of being compiled from sources, precompiled binaries are used, one must verify if the support for extended attributes has not been disabled, as it may happen in some misguided Linux distributions. Besides such precompiled binaries where the xattr support had been disabled, I have also seen various GUI-based file managers that lacked support for xattr and which could strip them during copy or move operations. The older tar and cpio archive formats do not support extended attributes, and many older tar programs support xattr, but by using tar or pax format extensions that may be incompatible with other tar implementations. While it is important to be aware of these caveats, otherwise you may have unpleasant surprises, like I had many years ago, when I was copying files between different users via /tmp, and I could not understand where the files were losing metadata (and also their timestamps were truncated), until realizing that /tmp was on tmpfs, and copying to /tmp was silently stripping the extended attributes and truncating the timestamps (the latter might no longer be true today), with a few precautions it is possible to use extended attributes without problems on Linux and FreeBSD. Before using extended attributes, I had also used a database file, but that had the disadvantage of being updated continuously all the time, even for file operations that did not change the file content, e.g. when renaming or moving files.
- deleted 4y ago[deleted]