4 ms·

Making MySQL 5,888.6x faster. Yep, you read that right!

4 points by titusblair 12y ago

chenyuanjen 12y ago
Seriously? 5,888x or 5,888%?
- titusblair 12y ago
  5,888x...here is the math I used: MySQL Time: 588.86 seconds BigObject Time: 0.1 second 588.86/0.1 = 5888.6 which is 5888.6x :) Hope this helps!
devhead 12y ago
Do you have plans to support replication from sources? Having to manually load data via a query takes away real time use and takes away the gains of the query in bigobject.
sigmonsays 12y ago
No explanation what so ever on how you did this.....
- titusblair 12y ago
  Let me know what explanation you are looking for and I would be happy to go into more detail.
bandrami 12y ago
Why are you using a locking read on a million+ row table? Why are you doing a naive join between million+ row tables? Is the point here that your product is better than a badly-implemented database on its own?
- titusblair 12y ago
  Q: Why are you using a locking read on a million+ row table? A: BigObject is row based storage as of current implementation. There is a RW lock on the table to coordinate access. Going forward as BigObject adopts advanced storage schemes i.e. column based storage, indexed row group, etc concurrency will naturally improve. For this iteration, we felt that most query are READ/SCAN access, and a simple RW lock is sufficient for the performance gain we observed. The bottom line is, BigObject is a collection of data structures that could operate on structured data sets in an efficient manner. It is the foundation that we build on to provide new ways to address data analytic problems. Q: Why are you doing a naive join between million+ row tables? A: Here we are demonstrating a heavy operation that will scan the entire data set. You could improve the performance if you know first hand what subset of data you will touch, and leave the rest behind, but so would BigObject's performance. Q: Is the point here that your product is better than a badly-implemented database on its own? A: MySQL, PostgreSQL and other database systems are robust and well-maintained solutions. Developers today develop applications with the premise that a database server must be present in their service, but could be using it for the wrong purpose. BigObject is a middle layer for extending the capabilities and performance of databases whose role today are skewed towards general storage and retrieval while the original idea was handling transactions. Arguably, you could tell people to use the right tools for the type of data and inbound frequency they are dealing with, but as a service provider and partner in Big Data Analytics, we have a responsibility beyond RTFM and snide. We hope this helps address your questions. Note: Questions answered via Jeff our product architect here at BigObject. Additional Material if you are interested: https://bitbucket.org/macrodata/benchmark-rdbms https://bitbucket.org/macrodata/benchmark-rdbms

4 points by titusblair 12y ago

chenyuanjen 12y ago

Seriously? 5,888x or 5,888%?

titusblair 12y ago
5,888x...here is the math I used: MySQL Time: 588.86 seconds BigObject Time: 0.1 second 588.86/0.1 = 5888.6 which is 5888.6x :) Hope this helps!

devhead 12y ago

Do you have plans to support replication from sources? Having to manually load data via a query takes away real time use and takes away the gains of the query in bigobject.

sigmonsays 12y ago

No explanation what so ever on how you did this.....

titusblair 12y ago
Let me know what explanation you are looking for and I would be happy to go into more detail.

bandrami 12y ago

Why are you using a locking read on a million+ row table? Why are you doing a naive join between million+ row tables? Is the point here that your product is better than a badly-implemented database on its own?

titusblair 12y ago
Q: Why are you using a locking read on a million+ row table? A: BigObject is row based storage as of current implementation. There is a RW lock on the table to coordinate access. Going forward as BigObject adopts advanced storage schemes i.e. column based storage, indexed row group, etc concurrency will naturally improve. For this iteration, we felt that most query are READ/SCAN access, and a simple RW lock is sufficient for the performance gain we observed. The bottom line is, BigObject is a collection of data structures that could operate on structured data sets in an efficient manner. It is the foundation that we build on to provide new ways to address data analytic problems. Q: Why are you doing a naive join between million+ row tables? A: Here we are demonstrating a heavy operation that will scan the entire data set. You could improve the performance if you know first hand what subset of data you will touch, and leave the rest behind, but so would BigObject's performance. Q: Is the point here that your product is better than a badly-implemented database on its own? A: MySQL, PostgreSQL and other database systems are robust and well-maintained solutions. Developers today develop applications with the premise that a database server must be present in their service, but could be using it for the wrong purpose. BigObject is a middle layer for extending the capabilities and performance of databases whose role today are skewed towards general storage and retrieval while the original idea was handling transactions. Arguably, you could tell people to use the right tools for the type of data and inbound frequency they are dealing with, but as a service provider and partner in Big Data Analytics, we have a responsibility beyond RTFM and snide. We hope this helps address your questions. Note: Questions answered via Jeff our product architect here at BigObject. Additional Material if you are interested: https://bitbucket.org/macrodata/benchmark-rdbms https://bitbucket.org/macrodata/benchmark-rdbms