3 ms·

How do you think it compares to Google cloud data store?

ronnietehnub 9y ago
Datastore is a slightly different beast, they handle all the underlying node work for you , so you are not scaling your cluster. They also provide a SQL like interface for querying your data which in big table would be completely on your implementation. There exists some logical boundaries by using `namespaces` (Inherently tables) and then the `kind` for further selection. When you query bigtable you can provide column / various filters on what type of data you would like to return or truncate. The problem with that is most of the filters at the bigtable level are regex based and can be resource intensive. The better approach with bigtable is to design a table (row key) for each type of query you would want to ask the DB.
funkjunky 9y ago
Former GCP support here. Bigtable and Cloud Datastore (or the newer, shinier Firestore) are very different, and meant for different purposes. Bigtable is meant for wide-column data at high volume. If your data can be organized into simple rows and columns, and you plan on using massive amounts of it at high throughout (think IoT transactions, for example), then bigtable is the right choice. Datastore, on the other hand, is for semi-structured data, with parent/child hierarchies, key value pairs, etc. It isn't run on a cluster of nodes like bigtable, but is managed behind the scenes as part of App Engine. It is slower than bigtable, but is more sophisticated, and offers client libraries for ORMs (ndb), SQL-like queries, and the like. There's a brief comparison chart here: https://cloud.google.com/storage-options/ https://cloud.google.com/storage-options/ I also highly recommend the Google cloud data engineering course at Coursera: https://www.coursera.org/specializations/gcp-data-machine-learning https://www.coursera.org/specializations/gcp-data-machine-le... Or the instructor's book, " Data Science on the Google Cloud Platform: Implementing End-to-End Real-Time Data Pipelines: From Ingest to Machine Learning" https://www.amazon.com/dp/1491974567/ref=cm_sw_r_cp_apa_DILVAbTAV80A7 https://www.amazon.com/dp/1491974567/ref=cm_sw_r_cp_apa_DILV...

Datastore is a slightly different beast, they handle all the underlying node work for you , so you are not scaling your cluster. They also provide a SQL like interface for querying your data which in big table would be completely on your implementation. There exists some logical boundaries by using `namespaces` (Inherently tables) and then the `kind` for further selection. When you query bigtable you can provide column / various filters on what type of data you would like to return or truncate. The problem with that is most of the filters at the bigtable level are regex based and can be resource intensive. The better approach with bigtable is to design a table (row key) for each type of query you would want to ask the DB.

funkjunky 9y ago

Former GCP support here. Bigtable and Cloud Datastore (or the newer, shinier Firestore) are very different, and meant for different purposes. Bigtable is meant for wide-column data at high volume. If your data can be organized into simple rows and columns, and you plan on using massive amounts of it at high throughout (think IoT transactions, for example), then bigtable is the right choice. Datastore, on the other hand, is for semi-structured data, with parent/child hierarchies, key value pairs, etc. It isn't run on a cluster of nodes like bigtable, but is managed behind the scenes as part of App Engine. It is slower than bigtable, but is more sophisticated, and offers client libraries for ORMs (ndb), SQL-like queries, and the like. There's a brief comparison chart here: https://cloud.google.com/storage-options/ https://cloud.google.com/storage-options/ I also highly recommend the Google cloud data engineering course at Coursera: https://www.coursera.org/specializations/gcp-data-machine-learning https://www.coursera.org/specializations/gcp-data-machine-le... Or the instructor's book, " Data Science on the Google Cloud Platform: Implementing End-to-End Real-Time Data Pipelines: From Ingest to Machine Learning" https://www.amazon.com/dp/1491974567/ref=cm_sw_r_cp_apa_DILVAbTAV80A7 https://www.amazon.com/dp/1491974567/ref=cm_sw_r_cp_apa_DILV...