4 ms·

It doesn't look like PolarFS has distributed metadata. HopsFS (HDFS derivative work) stores distributed metadata, enabling 16X throughput improvements over HD

It doesn't look like PolarFS has distributed metadata. HopsFS (HDFS derivative work) stores distributed metadata, enabling 16X throughput improvements over HDFS - https://www.usenix.org/conference/fast17/technical-sessions/presentation/niazi https://www.usenix.org/conference/fast17/technical-sessions/... http://www.hops.io http://www.hops.io Technically, the hard part is correct concurrent and consistent operations across shards (partitions). Disclaimer: one of the designers of HopsFS.

ddorian43 8y ago
What do you think are some gotchas/limitations when using mysql-ndb-cluster ?
- jamesblonde 8y ago
  It's become very stable with the years. It's somehow never penetrated the silicon valley echo chamber, probably due to instability in its earlier years. Operationally, it's got most things you need - on-line add node, rolling upgrades, monitoring, etc. We use NDB to store both the metadata in-memory, but also to store small files on NVMe disks. We had talks with lots of other DB vendors, but, frankly, none of them have high performance support for cross-partition transactions, which is needed. DBs like VoltDB, MemSQL, NuoDB all have promise, but serialize cross-shard operations.
  - ryanworl 8y ago
    Have you explored using FoundationDB? It doesn't have a SQL interface (anymore), which might mean it isn't suitable if SQL is required.
    - jamesblonde 8y ago
      Not yet, it's only recently been open-sourced. We have a startup now commercializing Hops, called Logical Clocks (of course), so we're busy with that.
      - ryanworl 8y ago
        The community is very helpful over on the forums (https://forums.foundationdb.org https://forums.foundationdb.org), so if you do end up having time to check it out, come on over! I'm sure the team at Apple (I'm not affiliated with them) would love to talk about it.
        ddorian43 8y ago
        They're against coprocessors (having functions in the db) so you'll always be somewhat limited. Doing with shared-memory-etc will be hard. Also you can shard many tables as hash, so most hot-path transactions be inside a server, which you can't guarantee with range-sharding.

ddorian43 8y ago

What do you think are some gotchas/limitations when using mysql-ndb-cluster ?

jamesblonde 8y ago
It's become very stable with the years. It's somehow never penetrated the silicon valley echo chamber, probably due to instability in its earlier years. Operationally, it's got most things you need - on-line add node, rolling upgrades, monitoring, etc. We use NDB to store both the metadata in-memory, but also to store small files on NVMe disks. We had talks with lots of other DB vendors, but, frankly, none of them have high performance support for cross-partition transactions, which is needed. DBs like VoltDB, MemSQL, NuoDB all have promise, but serialize cross-shard operations.
- ryanworl 8y ago
  Have you explored using FoundationDB? It doesn't have a SQL interface (anymore), which might mean it isn't suitable if SQL is required.
  - jamesblonde 8y ago
    Not yet, it's only recently been open-sourced. We have a startup now commercializing Hops, called Logical Clocks (of course), so we're busy with that.
    - ryanworl 8y ago
      The community is very helpful over on the forums (https://forums.foundationdb.org https://forums.foundationdb.org), so if you do end up having time to check it out, come on over! I'm sure the team at Apple (I'm not affiliated with them) would love to talk about it.
      - ddorian43 8y ago
        They're against coprocessors (having functions in the db) so you'll always be somewhat limited. Doing with shared-memory-etc will be hard. Also you can shard many tables as hash, so most hot-path transactions be inside a server, which you can't guarantee with range-sharding.