4 ms·
Not a whole lot of operational difference between sending them a snowflake view name or sending an S3 URL but there's probably two orders of magnitude cost diff
by wild_egg 2y ago
Not a whole lot of operational difference between sending them a snowflake view name or sending an S3 URL but there's probably two orders of magnitude cost difference
- ramraj07 2y agoDon’t know what you’re talking about. Used correctly snowflake can be as effective cost wise or cheaper for most orgs’ data needs. Compared to spinning up and down clusters of spark (and maintaining g teams to do this work or paying databricks).
- wild_egg 2y agoNot talking about spark clusters anywhere here. I wouldn't be surprised if that's similar in cost to snowflake. What I'm talking about is snowflake vs a $50 EC2 instance running DuckDB reading data from S3. Try it out sometime — the results might surprise you
- ramraj07 2y agoI have, and haven’t come away impressed. I have in fact done this multiple times. perhaps this doesn’t work with actual complex analytics and stuff? DuckDB doesn’t even do proper SIMD for example. To be clear I have used Duckdb in some serverless ideas and it has worked well, but nothing has bet hyper optimized engines like snowflake and redshift.