4 ms·

We have a system based on hbase with similar volume, with my elasticsearch experience, I cannot really see it replace hbase at these volumes. Also, Kafka is no

We have a system based on hbase with similar volume, with my elasticsearch experience, I cannot really see it replace hbase at these volumes. Also, Kafka is not a replacement, but complementary. The only other mature onprem store I can think of is Cassandra. There are other kids on the block, aerospike, cockroach db etc, with different guarantee semantics. I do agree hbase is a monster to manage. When it works though, there aren't many systems which can mach it's insert/scan capabilities T scale.

samspenc 10y ago
Agreed, I've seen reports on ElasticSearch's website that their largest customers are running petabyte-scale data on ElasticSearch clusters, but my rule of thumb has been: HBase for petabyte-scale data, ElasticSearch and others for terabyte-scale data. As for Kafka, I was surprised to see the article lump this together with Hadoop. I've used Kafka exclusively as a messaging bus or to stream large amounts of data. It doesn't exist as a replacement to Hadoop or Big Data technologies, but as a perfect complement, because it doesn't process data, just streams / moves it efficiently.