3 ms·

Awesome! Datastore connector, please!! Cloud Datastore is the ideal NoOps and Serverless storage solution for projects from small to large. It would be great if

• rattray 10y ago
  Would it be easy to pump Cloud Datastore into Bigquery and then read from that? (genuine question)
  • bduerst 10y ago
    You can load *.info Datastore backups into BigQuery tables natively. You could probably write up a quick apps script to automate data refreshes it if you wanted. Still not the same as reporting directly from a live datastore though :)
    • obulpathi 10y ago
      That's exactly my issue. I have a crawler that collects data and stores into Datastore. I need to periodically scoop up the snapshots and export them into BigQuery, then run the visualizations. I want my visualization to be live and also don't want the overhead of scooping the data.
      • skybrian 10y ago
        Using App Engine, I schedule task queue tasks to insert rows into a BigQuery table on the fly [1]. It only works for inserts, though. [1] https://cloud.google.com/bigquery/streaming-data-into-bigquery https://cloud.google.com/bigquery/streaming-data-into-bigque...
        • obulpathi 10y ago
          Thanks! And yes, as you mentioned, the problem is that it works only for inserts. I use Datastore as my primary database. So requesting for supporting Datastore from Datastudio.
  • vdm 10y ago
    I would try a Cloud Dataflow streaming job. Both Datastore and Bigquery have "IO" connectors in Dataflow.