4 ms·

MariaDB ColumnStore on Intel Optane SSD

79 points by Rafuino 8y ago

Twirrim 8y ago
To some degree this feels like "Fast storage turns out to be fast storage", but still, it's an interesting post and good to see some numbers.
- imaginenore 8y ago
  Optane is more like slow RAM, but lots of it.
- Rafuino 8y ago
  Poster here. The tweak here is that MariaDB tested the Optane SSDs with Intel Memory Drive Technology (basically presenting the SSDs as memory to the OS) and as fast storage. Pretty interesting stuff, and it's been fun to help developers get their hands on the tech and give it all a go.
manigandham 8y ago
Does anyone use MariaDB in production? If so, why over the competitors?
- joeyphoen 8y ago
  ServiceNow and Alibaba. At ServiceNow, our stack was built on top of MySQL from the beginning. It works for us. We left Oracle® MySQL for MariaDB largely because MariaDB actually seems to work to improve things for its bigger customers. Also, our SVP of Engineering is now on the MariaDB board.
  - morgo 8y ago
    AFAIU Alibaba uses AliSQL, based on MySQL 5.6 (not MariaDB): https://github.com/alibaba/AliSQL https://github.com/alibaba/AliSQL
- SEJeff 8y ago
  Yes, because MariaDB support is excellent and relatively affordable. On the other hand, you have Oracle... Also, we like the percona performance patchset that made it into MariaDB and isn't in MySQL proper along with the pt-* toolset from percona.
  - morgo 8y ago
    > the percona performance patchset that made it into MariaDB and isn't in MySQL Many Percona patches did make it into MySQL 5.7 and MySQL 8.0.
- fabianfabian 8y ago
  Is this a serious question?
  - manigandham 8y ago
    Yes, and I would recommend always assuming so on HN. I don't quite see the unique advantages of MariaDB compared to current MySQL, PostgreSQL, SQL Server, CockroachDB, MemSQL, etc.
    - sjwright 8y ago
      I don’t see the unique advantages of MySQL compared to current MariaDB, PostgreSQL, SQL Server, CockroachDB, MemSQL, etc. In my case we switched to MariaDB because it fixed a number of glaring query optimizer failings much sooner than MySQL. And we've stuck with MariaDB because it has been much smoother and confidence-inspiring than MySQL when we used that. We also use the TokuDB storage engine extensively, which is well supported by MariaDB. Whereas MySQL 8 is going in a very opinionated direction that I personally don't agree with.
      - morgo 8y ago
        Former product manager. Could you give some examples of what you mean by opinionated direction?
- nijave 8y ago
  Github's on MySQL https://githubengineering.com/mysql-high-availability-at-github/ https://githubengineering.com/mysql-high-availability-at-git...
- Illli1l1iIli1I 8y ago
  https://vizhub.healthdata.org/gbd-compare/ https://vizhub.healthdata.org/gbd-compare/ runs on MariaDB ColumnStore as its main DB for ~6TB of data and the largest database ~6 billion rows. Easy to setup. Stable. Great and affordable support. MaxScale is cool.
Johnny555 8y ago
Is the Optane SSD storage referenced in this article different from this Optane storage that GCP just announced support for? https://cloud.google.com/blog/topics/partners/available-first-on-google-cloud-intel-optane-dc-persistent-memory https://cloud.google.com/blog/topics/partners/available-firs...
- Rafuino 8y ago
  Disclosure: I help manage the lab (hosted by Packet) where we made available the hardware for this testing. If you're interested in testing and sharing your results, check it out here: https://www.acceleratewithoptane.com/ https://www.acceleratewithoptane.com/ Yes, they are different products but are both based on Intel 3D XPoint memory media. GCP announced support for Intel Optane DC Persistent Memory, which sits in memory DIMM channels (see https://newsroom.intel.com/news/intel-optane-dc-persistent-memory-readies-widespread-deployment/ https://newsroom.intel.com/news/intel-optane-dc-persistent-m... for a visual). The Intel Optane DC SSD referenced and tested in the posted article is an NVMe SSD but with different characteristics than NAND-based SSDs you can read about here: https://www.intel.com/content/www/us/en/solid-state-drives/optane-ssd-dc-p4800x-brief.html https://www.intel.com/content/www/us/en/solid-state-drives/o.... GCP's announcement is pretty exciting, and as more and more developers get their hands on both types of products, we'll see more interesting usages like MariaDB ColumnStore, Memcached, RabbitMQ, Postgres, etc. We have our own ideas, of course, but it's been fun making this all available to developers directly and seeing new ideas come up.

79 points by Rafuino 8y ago

To some degree this feels like "Fast storage turns out to be fast storage", but still, it's an interesting post and good to see some numbers.

imaginenore 8y ago
Optane is more like slow RAM, but lots of it.
Rafuino 8y ago
Poster here. The tweak here is that MariaDB tested the Optane SSDs with Intel Memory Drive Technology (basically presenting the SSDs as memory to the OS) and as fast storage. Pretty interesting stuff, and it's been fun to help developers get their hands on the tech and give it all a go.

manigandham 8y ago

Does anyone use MariaDB in production? If so, why over the competitors?

joeyphoen 8y ago
ServiceNow and Alibaba. At ServiceNow, our stack was built on top of MySQL from the beginning. It works for us. We left Oracle® MySQL for MariaDB largely because MariaDB actually seems to work to improve things for its bigger customers. Also, our SVP of Engineering is now on the MariaDB board.
- morgo 8y ago
  AFAIU Alibaba uses AliSQL, based on MySQL 5.6 (not MariaDB): https://github.com/alibaba/AliSQL https://github.com/alibaba/AliSQL
SEJeff 8y ago
Yes, because MariaDB support is excellent and relatively affordable. On the other hand, you have Oracle... Also, we like the percona performance patchset that made it into MariaDB and isn't in MySQL proper along with the pt-* toolset from percona.
- morgo 8y ago
  > the percona performance patchset that made it into MariaDB and isn't in MySQL Many Percona patches did make it into MySQL 5.7 and MySQL 8.0.
fabianfabian 8y ago
Is this a serious question?
- manigandham 8y ago
  Yes, and I would recommend always assuming so on HN. I don't quite see the unique advantages of MariaDB compared to current MySQL, PostgreSQL, SQL Server, CockroachDB, MemSQL, etc.
  - sjwright 8y ago
    I don’t see the unique advantages of MySQL compared to current MariaDB, PostgreSQL, SQL Server, CockroachDB, MemSQL, etc. In my case we switched to MariaDB because it fixed a number of glaring query optimizer failings much sooner than MySQL. And we've stuck with MariaDB because it has been much smoother and confidence-inspiring than MySQL when we used that. We also use the TokuDB storage engine extensively, which is well supported by MariaDB. Whereas MySQL 8 is going in a very opinionated direction that I personally don't agree with.
    - morgo 8y ago
      Former product manager. Could you give some examples of what you mean by opinionated direction?
nijave 8y ago
Github's on MySQL https://githubengineering.com/mysql-high-availability-at-github/ https://githubengineering.com/mysql-high-availability-at-git...
Illli1l1iIli1I 8y ago
https://vizhub.healthdata.org/gbd-compare/ https://vizhub.healthdata.org/gbd-compare/ runs on MariaDB ColumnStore as its main DB for ~6TB of data and the largest database ~6 billion rows. Easy to setup. Stable. Great and affordable support. MaxScale is cool.

Johnny555 8y ago

Is the Optane SSD storage referenced in this article different from this Optane storage that GCP just announced support for? https://cloud.google.com/blog/topics/partners/available-first-on-google-cloud-intel-optane-dc-persistent-memory https://cloud.google.com/blog/topics/partners/available-firs...

Rafuino 8y ago
Disclosure: I help manage the lab (hosted by Packet) where we made available the hardware for this testing. If you're interested in testing and sharing your results, check it out here: https://www.acceleratewithoptane.com/ https://www.acceleratewithoptane.com/ Yes, they are different products but are both based on Intel 3D XPoint memory media. GCP announced support for Intel Optane DC Persistent Memory, which sits in memory DIMM channels (see https://newsroom.intel.com/news/intel-optane-dc-persistent-memory-readies-widespread-deployment/ https://newsroom.intel.com/news/intel-optane-dc-persistent-m... for a visual). The Intel Optane DC SSD referenced and tested in the posted article is an NVMe SSD but with different characteristics than NAND-based SSDs you can read about here: https://www.intel.com/content/www/us/en/solid-state-drives/optane-ssd-dc-p4800x-brief.html https://www.intel.com/content/www/us/en/solid-state-drives/o.... GCP's announcement is pretty exciting, and as more and more developers get their hands on both types of products, we'll see more interesting usages like MariaDB ColumnStore, Memcached, RabbitMQ, Postgres, etc. We have our own ideas, of course, but it's been fun making this all available to developers directly and seeing new ideas come up.