3 ms·
Can you help us understand why one would use a data warehouse from Oracle as opposed to a plethora of arguably far better options like snowflake, BigQuery, and
by vgt 4y ago
Can you help us understand why one would use a data warehouse from Oracle as opposed to a plethora of arguably far better options like snowflake, BigQuery, and so on.
- anshumankmr 4y agoBigQuery can cost a fortune. Our data engineer messed up once by not time partioning the DB and it racked up a ~10K USD bill in a month due to the high number of queries that were running on it.
- vgt 4y agoAs a former PM on BigQuery, I'll offer my perspective: - I agree that, by default, the on-demand pricing model is pure consumption, and folks can and do mess up sometimes. - I had a habit of refunding folks when they asked, and if they made obvious mistakes, even though it took me an average of 3-4 hours to process each refund. - BigQuery has cost controls to prevent runaway costs, sounds like you must check it out ASAP - BigQuery also has a DDL option to require users to include a partition filter predicate in queries - BigQuery also has flat-rate pricing, on which the vast majority of folks above SMB on. There are no runaway costs with this one. I think generally BigQuery is a great example of an extreme serverless consumption-model. You have thousands of cores at your fingertips, and, well, if you do something that overuses, you are allowed, but you do pay for it.
- anshumankmr 4y agoI am a dev, so the pricing part would be in the responsibilities of the client's GCP project we were using along with the architects. The flat pricing was something I did suggest when we on the call discussing the above mentioned situation, but it was ignored. However, I agree its a really cool tool, I have not worked on the ETL side of things, but we use BigQuery in almost all of the projects I have been on, my teams usually needs to figure out the right query to determine key KPIs our clients usually ask for and the SQL is really helpful.
- latchkey 4y ago> even though it took me an average of 3-4 hours to process each refund Interesting. Why?
- dagw 4y agoAren't those both cloud only products that require you to send your data to third party servers (quite possibly in another country)? In some cases that is a hard no.
- vgt 4y agoThat's fair for some (I say niche) use cases. Both BigQuery and Snowflake have a plethora of security and compliance features (including ML-driven automatic PII identification, masking, and categorization), and they certainly do not move data to another country without your explicit direction.