4 ms·
We're hitting some perf issues with Snowflake at work (not necessarily due to Snowflake itself but possibly more what we're trying to do with it: data warehouse
by corford 6y ago
We're hitting some perf issues with Snowflake at work (not necessarily due to Snowflake itself but possibly more what we're trying to do with it: data warehouse storage needs but also a need for close to real-time analytical querying over that data). Has anyone here had any good/bad experiences with MemSQL?
- chrisjc 6y agoHave you considered using change streams to keep a "view" up to date? Do you have a good cluster-key defined? Very curious about your performance issues?
- corford 6y agoWe make heavy use of views within Snowflake, have sensible cluster keying and, further downstream, also leverage things like Looker's PDTs and Elastic Search. The issue is we have billions of rows and very varied analytical requirements, so there are quite a few "pathalogical" queries. Change streams are something we're looking at (as well as MemSQL, Clickhouse etc.)
- cater90 6y agoWhat are your performance requirements? I've found Snowflake to be fast enough for most workloads, but if you're talking in the sub-500ms range, then MemSQL's in-memory capabilities will help. Though I must warn you that the system is a pain to manage and their managed service is unproven.
- AdamProut 6y agoThis use case sounds like a good match for MemSQL at a high level (analytics with an SLA is our bread and butter). We'll also have a lot of elasticity features of snowflake shortly without sacrificing our performance advantages. (https://www.memsql.com/blog/the-future-is-bottomless https://www.memsql.com/blog/the-future-is-bottomless) (disclosure: MemSQL CTO).
- cater90 6y agoWill "bottomless" have the ability to spin up multiple compute clusters on a centralized storage layer?
- AdamProut 6y agoYes
- curiousDog 6y agoOrthogonal question, what happened to you guys? At one point you were the hottest start up on the block every ICPC competitive brogrammer wanted to get into and then you just fell off the radar..
- FridgeSeal 6y agoHaven't used MemSQL, but check out ClickHouse.
- turk- 6y agoSpark/Databricks + Delta lake would be a good solution for combining streaming and batch analytics. MemSQL is good for low latency streaming.