4 ms·

But what is text? Not everyone wants to use unicode. It is dependent of the platform, the region, the OS and on many other different things like LC_* variables

But what is text? Not everyone wants to use unicode. It is dependent of the platform, the region, the OS and on many other different things like LC_* variables on linux. Why should a filesystem depend on those too?

cryptonector 9y ago
ZFS has a per-dataset option to allow/reject non-UTF-8. For valid UTF-8 names ZFS implements normalization-preserving/insensitive behavior. That was the best compromise we could find, and it works really well.
deathanatos 9y ago
> and on many other different things like LC_ variables on linux.* The point is that it doesn't need to be. I would entertain that not everyone might not want to use Unicode: in that case, the FS should still have a well-defined encoding, such that I can still arrive at a string to display to the user. The point is not that "Unicode is best" but that storing file names as "bags of bytes" is incredibly user unfriendly, and there needs to be a straightforward, no bullshit method to display and transmit the names of files. But I would also argue that Unicode is the best we've got presently, and it would be pragmatic for a filesystem to simply adopt it outright. It's overwhelmingly the dominant character set in use today, especially if you ignore deprecated junk that Unicode is a strict superset of.
cryptonector 9y ago
Unicode is meant to replace all the other codesets. It's time to move on from them, except for compatibility reasons. Anyways, it's possible to allow UTF-8 and non-UTF-8 on the same filesystem, and still provide form-preserving/insensitive behavior... ZFS does it.
lurker456 9y ago
RDBMS's have gone through the same journey. First it was hardcoded, now for many of them we can specify the codepage or UTF encoding and collation of a DB, some even offer it per table or column, and so on. Filesystems should do the same. Either pick one way and stick to it (expose it as UTF8, with a sensible normalization and collation) or offer several options that can be specified when creating a volume and do on the fly conversions for clients that need it.

cryptonector 9y ago

ZFS has a per-dataset option to allow/reject non-UTF-8. For valid UTF-8 names ZFS implements normalization-preserving/insensitive behavior. That was the best compromise we could find, and it works really well.

deathanatos 9y ago

> and on many other different things like LC_ variables on linux.* The point is that it doesn't need to be. I would entertain that not everyone might not want to use Unicode: in that case, the FS should still have a well-defined encoding, such that I can still arrive at a string to display to the user. The point is not that "Unicode is best" but that storing file names as "bags of bytes" is incredibly user unfriendly, and there needs to be a straightforward, no bullshit method to display and transmit the names of files. But I would also argue that Unicode is the best we've got presently, and it would be pragmatic for a filesystem to simply adopt it outright. It's overwhelmingly the dominant character set in use today, especially if you ignore deprecated junk that Unicode is a strict superset of.

cryptonector 9y ago

Unicode is meant to replace all the other codesets. It's time to move on from them, except for compatibility reasons. Anyways, it's possible to allow UTF-8 and non-UTF-8 on the same filesystem, and still provide form-preserving/insensitive behavior... ZFS does it.

lurker456 9y ago

RDBMS's have gone through the same journey. First it was hardcoded, now for many of them we can specify the codepage or UTF encoding and collation of a DB, some even offer it per table or column, and so on. Filesystems should do the same. Either pick one way and stick to it (expose it as UTF8, with a sensible normalization and collation) or offer several options that can be specified when creating a volume and do on the fly conversions for clients that need it.