4 ms·

This also depends on why you are choosing this technology - is it part of your existing stack or do you have in-house experts in this tech? While Lucene synax

This also depends on why you are choosing this technology - is it part of your existing stack or do you have in-house experts in this tech? While Lucene synax is very powerful, it is not SQL as pointed out by the OP. If you have a lot of spare developer time and people skilled in this, it will likely work well for a while (potentially a really long while). Going with something like BigQuery or Redshift enables you to utilize other tech to supplement or accelerate skill sets, such as paying a SaaS for visualization/analysis tools (Looker, Mode, etc). In particular, separation of storage and processing helps avoid costs when you are not using the data. With elastic, I believe you need the cluster whether you are using it or not. BigQuery will only charge you for compute you use and pennies for storage. Same thing for Athena/Redshift spectrum. Snowflake is similar, but I believe for enterprise contracts it a bit more complex with minimums and such. Structuring data is another really important consideration - will you need to normalize and will you need to update labels/dimensions? That’s just the start.

karmelapple 8y ago
> is it part of your existing stack or do you have in-house experts in this tech? Nope and nope. That's definitely the scary / unknown part! > If you have a lot of spare developer time and people skilled in this, it will likely work well for a while (potentially a really long while). We don't have a lot of spare developer time and people. However, the Elastic docs make our fairly straightforward use case - dump lots of events in and filter them by date and about 4 different properties - seem not too daunting. The way it's sold on the Elastic site these days seems like a very "batteries included" kind of approach - am I interpreting their marketing a little too positively? Are we kidding ourselves? Would love to hear more about your thoughts! > Structuring data is another really important consideration - will you need to normalize and will you need to update labels/dimensions? That’s just the start. We don't expect much. Thanks for mentioning the other technologies, too! The way we're structuring it allows us to either use Elastic, SQL, or something else entirely, so thankfully we're fairly agnostic on the data store. If Elastic turns out to be a time sink, we'll have no hesitation trying something else out.

karmelapple 8y ago

> is it part of your existing stack or do you have in-house experts in this tech? Nope and nope. That's definitely the scary / unknown part! > If you have a lot of spare developer time and people skilled in this, it will likely work well for a while (potentially a really long while). We don't have a lot of spare developer time and people. However, the Elastic docs make our fairly straightforward use case - dump lots of events in and filter them by date and about 4 different properties - seem not too daunting. The way it's sold on the Elastic site these days seems like a very "batteries included" kind of approach - am I interpreting their marketing a little too positively? Are we kidding ourselves? Would love to hear more about your thoughts! > Structuring data is another really important consideration - will you need to normalize and will you need to update labels/dimensions? That’s just the start. We don't expect much. Thanks for mentioning the other technologies, too! The way we're structuring it allows us to either use Elastic, SQL, or something else entirely, so thankfully we're fairly agnostic on the data store. If Elastic turns out to be a time sink, we'll have no hesitation trying something else out.