3 ms·

Is that not just treating the filename as a byte sequence?

by foota 9y ago

cryptonector 9y ago
Almost! But not quite: it aliases some possible filenames, though only they are valid UTF-8 encodings (or UTF-16 say, if one did this on Windows). In particular, it prevents the existence of two files with equivalent names.
- foota 9y ago
  Interesting, so it recognizes different encodings of the same Unicode code points?
  - foota 9y ago
    Or does it just map code points that are identical to each other?
    - cryptonector 9y ago
      Yes. But only for the purpose of string comparison and hashing (ZFS hashes directories for fast lookups).
  - cryptonector 9y ago
    Yes. Well, it recognizes different forms of the same characters. The encoding is and must be UTF-8.
- derefr 9y ago
  In other words, like most RDBMSes, ZFS differentiates "field type" from "collation." "Field type" in an RDBMS controls what can be written (e.g. "valid UTF8 strings"), what will be read back (e.g. the use of the Unicode replacement character), and what special values like NULL will cast to. An RDBMS field's collation controls how values in the field will compare for equality, and what will happen when you sort on that field. Unique constraints in RDBMSes function on top of collation—so if the collation says two values are equivalent, the unique constraint will prevent you from inserting the new one. Great properties, if you can get them. It's too bad that the filesystem API is as low-level as it is, actually; if POSIX had some concept of "readdir(2) pre-sorted by a given field" like e.g. Windows does, then you could require that userland programs rely on FS-level collation rather than allowing them to collate the resulting values however they feel like (usually meaning "naively.")
  - cryptonector 9y ago
    Well, readdir() in ZFS does not produce outputs in any order, since directories are just plain hash tables. And the n-i string comparison function produces a boolean if I remember correctly, not a trinary. So it doesn't define a collation. But it could define a collation, that's true. The problem with moving sorting into the kernel is that you now need to have the collations there (English? French? something else? "Unicode" is not enough), and user-land needs to tell the kernel what collation to use for any given process or thread or system call. That's ETOOMUCHWORK for everyone, so it doesn't happen.
    - adrianratnapala 9y ago
      Indeed! Moreover, like it should be EBADIDEA.

Is that not just treating the filename as a byte sequence?

by foota 9y ago

Is that not just treating the filename as a byte sequence?

Almost! But not quite: it aliases some possible filenames, though only they are valid UTF-8 encodings (or UTF-16 say, if one did this on Windows). In particular, it prevents the existence of two files with equivalent names.

foota 9y ago
Interesting, so it recognizes different encodings of the same Unicode code points?
- foota 9y ago
  Or does it just map code points that are identical to each other?
  - cryptonector 9y ago
    Yes. But only for the purpose of string comparison and hashing (ZFS hashes directories for fast lookups).
- cryptonector 9y ago
  Yes. Well, it recognizes different forms of the same characters. The encoding is and must be UTF-8.
derefr 9y ago
In other words, like most RDBMSes, ZFS differentiates "field type" from "collation." "Field type" in an RDBMS controls what can be written (e.g. "valid UTF8 strings"), what will be read back (e.g. the use of the Unicode replacement character), and what special values like NULL will cast to. An RDBMS field's collation controls how values in the field will compare for equality, and what will happen when you sort on that field. Unique constraints in RDBMSes function on top of collation—so if the collation says two values are equivalent, the unique constraint will prevent you from inserting the new one. Great properties, if you can get them. It's too bad that the filesystem API is as low-level as it is, actually; if POSIX had some concept of "readdir(2) pre-sorted by a given field" like e.g. Windows does, then you could require that userland programs rely on FS-level collation rather than allowing them to collate the resulting values however they feel like (usually meaning "naively.")
- cryptonector 9y ago
  Well, readdir() in ZFS does not produce outputs in any order, since directories are just plain hash tables. And the n-i string comparison function produces a boolean if I remember correctly, not a trinary. So it doesn't define a collation. But it could define a collation, that's true. The problem with moving sorting into the kernel is that you now need to have the collations there (English? French? something else? "Unicode" is not enough), and user-land needs to tell the kernel what collation to use for any given process or thread or system call. That's ETOOMUCHWORK for everyone, so it doesn't happen.
  - adrianratnapala 9y ago
    Indeed! Moreover, like it should be EBADIDEA.