4 ms·

First they picked up infinidb a while back and have been working to mainline "mariadb columnstore". Now the acquisition of Clustrix and discussion of mainlining

First they picked up infinidb a while back and have been working to mainline "mariadb columnstore". Now the acquisition of Clustrix and discussion of mainlining that also. It looks like MySQLs separation of the storage engines is paying off in their ability to keep one interface but allowing significantly different backends to meet the different workload requirements. There has been a lot of work expanding the storage engine API so columnstore can be mainlined and work so spider can progress. I imagine that progress has showed them that they are likely to be able to integrate a much larger and less connected (to existing mariadb) code base like clusterix. I can only open they decide to open source (BSL license?) the clusterix solution eventually. Between MyRocks (replacing tokudb), columnstore, spider and eventually clustrix seems that mariadb is trying to making the case that they can handle any size workload being through at it.

no1youknowz 8y ago
How is this any different from what MemSQL is already doing?
karmakaze 8y ago
I had high hopes for MyRocks, then I got a chance to use it. The limitations, mainly being 5.6 and no coexistence with InnoDB made me reevaluate TokuDB and it was a better choice for a write heavy, low update, workload especially with interval flushing (non-fsync durable) commits.
- zepearl 8y ago
  I am using as well MariaDB + TokuDB with "a write heavy, low update, workload especially with interval flushing (non-fsync durable) commits" => do you have maybe a short list of the limitations of MyRocks in MariaDB for this area? I tried to use MyRocks (never used it before) in MariaDB some months ago but couldn't find almost any docs and ultimately didn't understand which parameters were supposed to be set how under which condition... .
  - sethhochberg 8y ago
    IMO this is one of the biggest issues with the alternative storage engines for MySQL-family databases... we've also experimented with TokuDB for log-like data but found that, ultimately, the shortage of detailed documentation and operational issues like needing to develop homegrown tooling for things like backups overpowered the performance benefits. InnoDB isn't perfect, but it _is_ exhaustively documented and pretty well-understood, with a great set of related tools from Percona, etc, for simplifying operations. That goes a long way. Recently we've switched back to using InnoDB for ingestion on one of our write-heavy tables and aggressively archiving the data out of it and into Clickhouse (InnoDB deals with the high volume of concurrent inserts, data is loaded into Clickhouse in large batches for querying). By comparison to Toku or RocksDB, Clickhouse is refreshingly well-documented and its easy for us to make consistent backups with ZFS snapshots.
  - mdcallag 8y ago
    There are many options but most don't have to be set. We need to improve the tuning experience. See http://smalldatum.blogspot.com/2018/09/5-things-to-set-when-configuring.html http://smalldatum.blogspot.com/2018/09/5-things-to-set-when-...
- mdcallag 8y ago
  MyRocks isn't 5.6 only. It is in Percona 5.7 and MariaDB 10.3 I agree that tuning is too complex and we should do much better there. This explains where to ask for advice - http://smalldatum.blogspot.com/2018/05/where-to-ask-questions-about-myrocks.html http://smalldatum.blogspot.com/2018/05/where-to-ask-question...

no1youknowz 8y ago

How is this any different from what MemSQL is already doing?

karmakaze 8y ago

I had high hopes for MyRocks, then I got a chance to use it. The limitations, mainly being 5.6 and no coexistence with InnoDB made me reevaluate TokuDB and it was a better choice for a write heavy, low update, workload especially with interval flushing (non-fsync durable) commits.

zepearl 8y ago
I am using as well MariaDB + TokuDB with "a write heavy, low update, workload especially with interval flushing (non-fsync durable) commits" => do you have maybe a short list of the limitations of MyRocks in MariaDB for this area? I tried to use MyRocks (never used it before) in MariaDB some months ago but couldn't find almost any docs and ultimately didn't understand which parameters were supposed to be set how under which condition... .
- sethhochberg 8y ago
  IMO this is one of the biggest issues with the alternative storage engines for MySQL-family databases... we've also experimented with TokuDB for log-like data but found that, ultimately, the shortage of detailed documentation and operational issues like needing to develop homegrown tooling for things like backups overpowered the performance benefits. InnoDB isn't perfect, but it _is_ exhaustively documented and pretty well-understood, with a great set of related tools from Percona, etc, for simplifying operations. That goes a long way. Recently we've switched back to using InnoDB for ingestion on one of our write-heavy tables and aggressively archiving the data out of it and into Clickhouse (InnoDB deals with the high volume of concurrent inserts, data is loaded into Clickhouse in large batches for querying). By comparison to Toku or RocksDB, Clickhouse is refreshingly well-documented and its easy for us to make consistent backups with ZFS snapshots.
- mdcallag 8y ago
  There are many options but most don't have to be set. We need to improve the tuning experience. See http://smalldatum.blogspot.com/2018/09/5-things-to-set-when-configuring.html http://smalldatum.blogspot.com/2018/09/5-things-to-set-when-...
mdcallag 8y ago
MyRocks isn't 5.6 only. It is in Percona 5.7 and MariaDB 10.3 I agree that tuning is too complex and we should do much better there. This explains where to ask for advice - http://smalldatum.blogspot.com/2018/05/where-to-ask-questions-about-myrocks.html http://smalldatum.blogspot.com/2018/05/where-to-ask-question...