5 ms·

The elephant in the room is that there is no mention of TokuDB here. My personal experience is that InnoDB performance drops off a cliff as tables grow and par

The elephant in the room is that there is no mention of TokuDB here. My personal experience is that InnoDB performance drops off a cliff as tables grow and parts of tables stop fitting in RAM. TokuDB just keeps on going. TokuDB also compresses your data really well. My own stats: https://williamedwardscoder.tumblr.com/post/160628887798/compressing-mysql-databases https://williamedwardscoder.tumblr.com/post/160628887798/com...

falcolas 9y ago
InnoDB performance drops off if you don't have hotspots in your data access, and is still really good otherwise. This doesn't take away from TokuDB though - which is pretty awesome in any case.
fweespeech 9y ago
> My personal experience is that InnoDB performance drops off a cliff as tables grow and parts of tables stop fitting in RAM. TokuDB just keeps on going. Yes but at least in my experience as things fit in RAM, InnoDB is still the correct choice. TokuDB is great once you run out of RAM-related solutions but given the price point RAM is at these days, we are talking 100GB+ tables which probably shouldn't be one a single machine anyway.
- deleted 9y ago
  [deleted]
- willvarfar 9y ago
  Ha! A few gb ram and a few tb disk is real cheap in the cloud. If your dataset fits in ram, and when it doesn't you have to buy more boxes with more ram rather than just buying more disk, then I think you are making a very uneconomical trade off.
  - fweespeech 9y ago
    > Ha! A few gb ram and a few tb disk is real cheap in the cloud. > If your dataset fits in ram, and when it doesn't you have to buy more boxes with more ram rather than just buying more disk, then I think you are making a very uneconomical trade off. It is all in what lower latency is worth to the bottom line and how well you scale horizontally.
    - willvarfar 9y ago
      My link described performance too. A cluster of innodb boxes is going to be slower than a single tokudb box. (Src: have run each, and stuff like shard-query to parallelise queries and scale them out)
- fgonzag 9y ago
  Why wouldn't 100GB tables be in a single machine? It's not that much data. I'm comfortably running a 3TB table on PostgreSQL, which is growing at a rate of about 75GB a month. The machine isn't even that beefy, 128 GB of RAM, dual mid range Xeons, and prosumer SSDs in RAID 10. I would rather spend money in PCI-E SSDs, 300+ GB of RAM, and high end CPUs if it meant I didn't have to implement sharding at an application level.
- gaius 9y ago
  we are talking 100GB+ tables which probably shouldn't be one a single machine anyway. You can comfortably handle multi-tera tables on single machines these days. Adding more machines isn't "scaling", its admitting "we can't scale in software so make it a hardware problem"
petre 9y ago
TokuDB doesn't come enabled in the MariaDB package that's shipped with most distributions. I wanted to try it out on openSuSE 42.2 but it wasn't there. It ships with MariaDB 10.0.31 I think. I think we are ultimately going to switch to PostgresSQL as it has better tooling and features that we need (json, jsonb, postgis) and performance is also good when properly tuned.

falcolas 9y ago

InnoDB performance drops off if you don't have hotspots in your data access, and is still really good otherwise. This doesn't take away from TokuDB though - which is pretty awesome in any case.

fweespeech 9y ago

> My personal experience is that InnoDB performance drops off a cliff as tables grow and parts of tables stop fitting in RAM. TokuDB just keeps on going. Yes but at least in my experience as things fit in RAM, InnoDB is still the correct choice. TokuDB is great once you run out of RAM-related solutions but given the price point RAM is at these days, we are talking 100GB+ tables which probably shouldn't be one a single machine anyway.

deleted 9y ago
[deleted]
willvarfar 9y ago
Ha! A few gb ram and a few tb disk is real cheap in the cloud. If your dataset fits in ram, and when it doesn't you have to buy more boxes with more ram rather than just buying more disk, then I think you are making a very uneconomical trade off.
- fweespeech 9y ago
  > Ha! A few gb ram and a few tb disk is real cheap in the cloud. > If your dataset fits in ram, and when it doesn't you have to buy more boxes with more ram rather than just buying more disk, then I think you are making a very uneconomical trade off. It is all in what lower latency is worth to the bottom line and how well you scale horizontally.
  - willvarfar 9y ago
    My link described performance too. A cluster of innodb boxes is going to be slower than a single tokudb box. (Src: have run each, and stuff like shard-query to parallelise queries and scale them out)
fgonzag 9y ago
Why wouldn't 100GB tables be in a single machine? It's not that much data. I'm comfortably running a 3TB table on PostgreSQL, which is growing at a rate of about 75GB a month. The machine isn't even that beefy, 128 GB of RAM, dual mid range Xeons, and prosumer SSDs in RAID 10. I would rather spend money in PCI-E SSDs, 300+ GB of RAM, and high end CPUs if it meant I didn't have to implement sharding at an application level.
gaius 9y ago
we are talking 100GB+ tables which probably shouldn't be one a single machine anyway. You can comfortably handle multi-tera tables on single machines these days. Adding more machines isn't "scaling", its admitting "we can't scale in software so make it a hardware problem"

petre 9y ago

TokuDB doesn't come enabled in the MariaDB package that's shipped with most distributions. I wanted to try it out on openSuSE 42.2 but it wasn't there. It ships with MariaDB 10.0.31 I think. I think we are ultimately going to switch to PostgresSQL as it has better tooling and features that we need (json, jsonb, postgis) and performance is also good when properly tuned.