3 ms·

> Especially in the sciences. Take a look at how Stanford is leveraging BigQuery for their genomics analysis: http://www.eventbrite.com/e/interactive-cloud-an

by fhoffa 11y ago

> Especially in the sciences. Take a look at how Stanford is leveraging BigQuery for their genomics analysis: http://www.eventbrite.com/e/interactive-cloud-analytics-extra-large-scale-genomics-data-tickets-18717434350 http://www.eventbrite.com/e/interactive-cloud-analytics-extr... > It manages that faster than just the latency to bigquery would be — 18ms over a test dataset of several gigabyte by now. Yes!!! Exactly! That's the whole point. If something is taking you less than a second, don't bring BigQuery into the mix. But when you start hitting processes that take hours or days to run... try BigQuery. Going from hours to seconds changes your life. > I personally use pgsql currently to analyse data from Android crash reports. Cool! Guess what Motorola uses... http://www.slideshare.net/PatrickDeglon/predictive-analytics-business-insights-2014 http://www.slideshare.net/PatrickDeglon/predictive-analytics...

kuschku 11y ago
> If something is taking you less than a second, don't bring BigQuery into the mix. But when you start hitting processes that take hours or days to run... try BigQuery But that's my entire point? Why are all the test datasets small enough to be faster in pgsql than the latency to Google would take? I mean, if I wanted to showcase my software, I'd use a huge dataset with a complex problem — say protein folding, or superconductor molecular analysis — and show it in comparison on bigquery and a standard local database. Although these two examples are bad, as I know from my university that they can be solved in the same time for less money locally than by using bigquery
- fhoffa 11y ago
  Look... if you don't have problems that take hours to solve in your current environment, you haven't found big data problems (yet). And that's OK. Not everyone works with big data. But if one day you do, please ping me, it will be fun to do a follow up.
  - kuschku 11y ago
    Yeah, that's why I had hoped the test datasets would be big data problems — so I could see what that actually looks like ;)

by fhoffa 11y ago

kuschku 11y ago

> If something is taking you less than a second, don't bring BigQuery into the mix. But when you start hitting processes that take hours or days to run... try BigQuery But that's my entire point? Why are all the test datasets small enough to be faster in pgsql than the latency to Google would take? I mean, if I wanted to showcase my software, I'd use a huge dataset with a complex problem — say protein folding, or superconductor molecular analysis — and show it in comparison on bigquery and a standard local database. Although these two examples are bad, as I know from my university that they can be solved in the same time for less money locally than by using bigquery

fhoffa 11y ago
Look... if you don't have problems that take hours to solve in your current environment, you haven't found big data problems (yet). And that's OK. Not everyone works with big data. But if one day you do, please ping me, it will be fun to do a follow up.
- kuschku 11y ago
  Yeah, that's why I had hoped the test datasets would be big data problems — so I could see what that actually looks like ;)