> it's not clear why you'd have gone with Elasticsearch in the first place. Having been in a company with a similar issue (Hadoop/Spark, not ES), the issue is:

> it's not clear why you'd have gone with Elasticsearch in the first place. Having been in a company with a similar issue (Hadoop/Spark, not ES), the issue is: You have a bunch of programmers hired to handle the 10TB data/day. Then you need some work done processing some data that is, e.g. 100GB once a week. Rather than evaluate the best way to process that data, the through process basically goes "it doesn't fit in my laptop's memory, so we'll use the cluster."