3 ms·

> they are close to the underlying hardware for practical reasons Could you provide reference information to support this background assertion? I'm not totally

by adolph 1y ago

> they are close to the underlying hardware for practical reasons Could you provide reference information to support this background assertion? I'm not totally familiar with filesystems under the hood, but at this point doesn't storage hardware maintain an electrical representation relatively independent from the logical given things like wear leveling?

Ericson2314 1y ago
Yes I agree, that assertion doesn't pass muster. Mature database implementations also bypass a lot of kernel machinary to get closer to the underlying block devices. The layering of DB on top of FS is a failure.
- foobiekr 1y ago
  You are confusing that databases implement their own filesystem equivalent functionality in an application-specific way with the idea that FS's can or should be databases.
  - Ericson2314 1y ago
    I am not confusing any such thing. You need to define "database" such that "file system" doesn't include it. Common usage does this by convention, but that's just sloppy thinking and populist extentional definitining. I posit that any rigorous, thought-out, not overfit intentional definition of a database will, as a matter of course, also include file systems.
    - foobiekr 1y ago
      Perhaps so, but in the expansive definition you're using, even an in-memory binary tree qualifies as a database, which makes your point meaningless.
      - Ericson2314 1y ago
        I'm OK including that — tmpfs is similar, but we can easily exclude that by requiring persistence. The intentional definition doesn't need to be expansive the point of being useless!
      - aforwardslash 1y ago
        Nitpick: An in-memory hash table can be a filesystem :)
mrlongroots 1y ago
Some examples off the top of my head: - You can reason about block offsets. If your writes are 512B-aligned, you can be ensured minimal write amplification. - If your writes are append-only, log-structured, that makes SSD compaction a lot more straightforward - No caching guarantees by default. Again, even SSDs cache writes. Block writes are not atomic even with SSDs. The only way to guarantee atomicity is via write-ahead logs. - The NVMe layer exposes async submission/completion queues, to control the io_depth the device is subjected to, which is essential to get max perf from modern NVMe SSDs. Although you need to use the right interface to leverage it (libaio/io_uring/SPDK).
- Ericson2314 1y ago
  Is file alignment on disk guaranteed, or does it depend on the file system? The NVMe layer is not the same as the POSIX filesystem, there is no reason we need to throw that as part of knocking the POSIX filesystem off it's privileged position. Overall you are talking about individual files, but remember what really distinguishes the filesystem is directories. Other database, even relational ones, can have binary blob "leaf data" with the properties you speak about.
  - mrlongroots 1y ago
    > Is file alignment on disk guaranteed, or does it depend on the file system? I think "guaranteed" is too strong a word given the number of filesystems and flags out there, but "largely" you get aligned I/O. > The NVMe layer is not the same as the POSIX filesystem, there is no reason we need to throw that as part of knocking the POSIX filesystem off it's privileged position. I'd say that the POSIX filesystem lives in an ecosystem that makes leveraging NVMe layer characteristics a viable option. More along with the next point. > Overall you are talking about individual files, but remember what really distinguishes the filesystem is directories. Other database, even relational ones, can have binary blob "leaf data" with the properties you speak about. I think regardless of how you use a database, your interface is declarative. You always say "update this row" vs "fseek to offset 1048496 and fwrite 128 bytes and then fsync the page cache". Something needs to do the translation from "update this row" to the latter, and that layer will always be closer to hardware.

by adolph 1y ago

Ericson2314 1y ago

Yes I agree, that assertion doesn't pass muster. Mature database implementations also bypass a lot of kernel machinary to get closer to the underlying block devices. The layering of DB on top of FS is a failure.

foobiekr 1y ago
You are confusing that databases implement their own filesystem equivalent functionality in an application-specific way with the idea that FS's can or should be databases.
- Ericson2314 1y ago
  I am not confusing any such thing. You need to define "database" such that "file system" doesn't include it. Common usage does this by convention, but that's just sloppy thinking and populist extentional definitining. I posit that any rigorous, thought-out, not overfit intentional definition of a database will, as a matter of course, also include file systems.
  - foobiekr 1y ago
    Perhaps so, but in the expansive definition you're using, even an in-memory binary tree qualifies as a database, which makes your point meaningless.
    - Ericson2314 1y ago
      I'm OK including that — tmpfs is similar, but we can easily exclude that by requiring persistence. The intentional definition doesn't need to be expansive the point of being useless!
    - aforwardslash 1y ago
      Nitpick: An in-memory hash table can be a filesystem :)

mrlongroots 1y ago

Some examples off the top of my head: - You can reason about block offsets. If your writes are 512B-aligned, you can be ensured minimal write amplification. - If your writes are append-only, log-structured, that makes SSD compaction a lot more straightforward - No caching guarantees by default. Again, even SSDs cache writes. Block writes are not atomic even with SSDs. The only way to guarantee atomicity is via write-ahead logs. - The NVMe layer exposes async submission/completion queues, to control the io_depth the device is subjected to, which is essential to get max perf from modern NVMe SSDs. Although you need to use the right interface to leverage it (libaio/io_uring/SPDK).

Ericson2314 1y ago
Is file alignment on disk guaranteed, or does it depend on the file system? The NVMe layer is not the same as the POSIX filesystem, there is no reason we need to throw that as part of knocking the POSIX filesystem off it's privileged position. Overall you are talking about individual files, but remember what really distinguishes the filesystem is directories. Other database, even relational ones, can have binary blob "leaf data" with the properties you speak about.
- mrlongroots 1y ago
  > Is file alignment on disk guaranteed, or does it depend on the file system? I think "guaranteed" is too strong a word given the number of filesystems and flags out there, but "largely" you get aligned I/O. > The NVMe layer is not the same as the POSIX filesystem, there is no reason we need to throw that as part of knocking the POSIX filesystem off it's privileged position. I'd say that the POSIX filesystem lives in an ecosystem that makes leveraging NVMe layer characteristics a viable option. More along with the next point. > Overall you are talking about individual files, but remember what really distinguishes the filesystem is directories. Other database, even relational ones, can have binary blob "leaf data" with the properties you speak about. I think regardless of how you use a database, your interface is declarative. You always say "update this row" vs "fseek to offset 1048496 and fwrite 128 bytes and then fsync the page cache". Something needs to do the translation from "update this row" to the latter, and that layer will always be closer to hardware.