3 ms·
Why wouldn't you mail your co-worker the SQL text of this query? What is the advantage of the binary database file with no data?
by remram 2y ago
Why wouldn't you mail your co-worker the SQL text of this query? What is the advantage of the binary database file with no data?
- wild_egg 2y agoYou then also have to mail them every update to the SQL and hope they're handling versioning well With this approach, next time they attach the DB, they automatically see the latest version
- remram 2y agoOh I missed the part where they put that DB file on S3 too and sent a link. I thought they were attaching the DB to the email. Still, I would more readily send people a link to a Gist or playground than a binary DB on S3.
- ramraj07 2y agoOr if all your data is in a place like snowflake, you just create a view and send them the name..
- banku_brougham 2y agowell yeah when you pay the big bucks you can expect all the bells and whistles.
- MR4D 2y agoThe future is here, it's just not evenly distributed. Wait a decade (or less) and this will be in open source. Heck, it might already be.
- wild_egg 2y agoNot a whole lot of operational difference between sending them a snowflake view name or sending an S3 URL but there's probably two orders of magnitude cost difference
- ramraj07 2y agoDon’t know what you’re talking about. Used correctly snowflake can be as effective cost wise or cheaper for most orgs’ data needs. Compared to spinning up and down clusters of spark (and maintaining g teams to do this work or paying databricks).
- wild_egg 2y agoNot talking about spark clusters anywhere here. I wouldn't be surprised if that's similar in cost to snowflake. What I'm talking about is snowflake vs a $50 EC2 instance running DuckDB reading data from S3. Try it out sometime — the results might surprise you
- ramraj07 2y agoI have, and haven’t come away impressed. I have in fact done this multiple times. perhaps this doesn’t work with actual complex analytics and stuff? DuckDB doesn’t even do proper SIMD for example. To be clear I have used Duckdb in some serverless ideas and it has worked well, but nothing has bet hyper optimized engines like snowflake and redshift.