4 ms·

Even MySQL can handle 256gb of rows in memory with ease if you give it enough ram The scan performance of MySQL on that much data is going to be dramatically s

by tmostak 10y ago

Even MySQL can handle 256gb of rows in memory with ease if you give it enough ram The scan performance of MySQL on that much data is going to be dramatically slower. MySQL may be good at indexed accesses but is simply not build for analytics workloads like those tested in the blog post. Not sure exactly how MySQL benches against Postgres, but the latter takes minutes over the same queries. http://tech.marksblogg.com/billion-nyc-taxi-rides-postgresql.html http://tech.marksblogg.com/billion-nyc-taxi-rides-postgresql...

paulasmuth 10y ago
That's not a fair comparison though. I only quickly read the postgres benchmark you linked, but it looks like you stored the data on disk (!) on a machine with only 16GB of ram for the postgres benchmark, while storing the data completely in 96GB of very fast RAM for the mapd benchmark. Unless I misread the post it's just comparing apples and oranges. If anything, the two benchmarks show that one piece of hardware (RAM) is orders of magnitude faster than the other one (SSDs). My previous comment was specifically about MySQL running on a machine where the whole dataset fits into memory. Of course, I'm sure mapd is faster than mysql/postgres for some usecases. But the benchmark doesn't prove that in a fair comparison. EDIT: Maybe I should add a disclaimer: I'm the founder of another open-source database product that could be considered a competitor to MapD.
- tmostak 10y ago
  That's not a fair comparison though. I only quickly read the postgres benchmark you linked, but it looks like you stored the data on disk (!) on a machine with only 16GB of ram for the postgres benchmark, while storing the data completely in 96GB of very fast RAM for the mapd benchmark. Fair enough, but even if the data is in RAM a CPU solution will still be much slower. See this benchmark running the same queries on a 7-node Redshift cluster. http://tech.marksblogg.com/billion-nyc-taxi-rides-redshift-large-cluster.html http://tech.marksblogg.com/billion-nyc-taxi-rides-redshift-l... And MySQL for all its strengths is not in an analytics database and for these types of queries will be much slower than Redshift.

by tmostak 10y ago

paulasmuth 10y ago

That's not a fair comparison though. I only quickly read the postgres benchmark you linked, but it looks like you stored the data on disk (!) on a machine with only 16GB of ram for the postgres benchmark, while storing the data completely in 96GB of very fast RAM for the mapd benchmark. Unless I misread the post it's just comparing apples and oranges. If anything, the two benchmarks show that one piece of hardware (RAM) is orders of magnitude faster than the other one (SSDs). My previous comment was specifically about MySQL running on a machine where the whole dataset fits into memory. Of course, I'm sure mapd is faster than mysql/postgres for some usecases. But the benchmark doesn't prove that in a fair comparison. EDIT: Maybe I should add a disclaimer: I'm the founder of another open-source database product that could be considered a competitor to MapD.

tmostak 10y ago
That's not a fair comparison though. I only quickly read the postgres benchmark you linked, but it looks like you stored the data on disk (!) on a machine with only 16GB of ram for the postgres benchmark, while storing the data completely in 96GB of very fast RAM for the mapd benchmark. Fair enough, but even if the data is in RAM a CPU solution will still be much slower. See this benchmark running the same queries on a 7-node Redshift cluster. http://tech.marksblogg.com/billion-nyc-taxi-rides-redshift-large-cluster.html http://tech.marksblogg.com/billion-nyc-taxi-rides-redshift-l... And MySQL for all its strengths is not in an analytics database and for these types of queries will be much slower than Redshift.