5 ms·

My company is currently using Mongo, and while it works, I wouldn't recommend it. We're looking at Cassandra and Elasticsearch, which seems to be a lot more pro

by caw 12y ago

My company is currently using Mongo, and while it works, I wouldn't recommend it. We're looking at Cassandra and Elasticsearch, which seems to be a lot more promising.

arenaninja 12y ago
The number of horror stories I've seen about mongo is up to around 10 this month alone. I'm now glad I never made the jump... in the meantime, pgsql is still on my list
- caw 12y ago
  I wouldn't say it's a horror story, it's just not really for time series "big data". The backend guys have had to muck about with the data a lot to get good performance out of it. There's some optimizations we missed on the sysadmin side too, like sharding the cluster after it got to ~250GB, and now it's many times that. Our Mongo clusters have been running production for well over a year.
- brian66 12y ago
  We had a 50gb instance initially and it was no problem, then our app started to get a lot more traffic in a short period of time. We had to start sharding on a reasonably large scale. Mongo is a lot harder to maintain when it's big and unpredictable. I know this sounds like a plug, alas it's the truth, we found Object Rocket and now we don't worry about Mongo.
spiralganglion 12y ago
Beware of using Elasticsearch as a primary DB. Kyle Kingsbury has shown that it loses an awful lot of data during a partition, despite their claims. Example: http://aphyr.com/posts/317-call-me-maybe-elasticsearch http://aphyr.com/posts/317-call-me-maybe-elasticsearch
- capkutay 12y ago
  I think with $70m+ in funding, ES has the resources to fix their split brain issues.
fennecfoxen 12y ago
I've only had about a year and a half of working in that space, but can confirm that Cassandra's quite good for truly huge time-series data. If you want to record an event every time someone makes a purchase at a Wal-Mart, turn to Cassandra or a similar system. It is also much more highly regarded as a primary data store than Elasticsearch.

by caw 12y ago

My company is currently using Mongo, and while it works, I wouldn't recommend it. We're looking at Cassandra and Elasticsearch, which seems to be a lot more promising.

arenaninja 12y ago

The number of horror stories I've seen about mongo is up to around 10 this month alone. I'm now glad I never made the jump... in the meantime, pgsql is still on my list

caw 12y ago
I wouldn't say it's a horror story, it's just not really for time series "big data". The backend guys have had to muck about with the data a lot to get good performance out of it. There's some optimizations we missed on the sysadmin side too, like sharding the cluster after it got to ~250GB, and now it's many times that. Our Mongo clusters have been running production for well over a year.
brian66 12y ago
We had a 50gb instance initially and it was no problem, then our app started to get a lot more traffic in a short period of time. We had to start sharding on a reasonably large scale. Mongo is a lot harder to maintain when it's big and unpredictable. I know this sounds like a plug, alas it's the truth, we found Object Rocket and now we don't worry about Mongo.

spiralganglion 12y ago

Beware of using Elasticsearch as a primary DB. Kyle Kingsbury has shown that it loses an awful lot of data during a partition, despite their claims. Example: http://aphyr.com/posts/317-call-me-maybe-elasticsearch http://aphyr.com/posts/317-call-me-maybe-elasticsearch

capkutay 12y ago
I think with $70m+ in funding, ES has the resources to fix their split brain issues.

fennecfoxen 12y ago

I've only had about a year and a half of working in that space, but can confirm that Cassandra's quite good for truly huge time-series data. If you want to record an event every time someone makes a purchase at a Wal-Mart, turn to Cassandra or a similar system. It is also much more highly regarded as a primary data store than Elasticsearch.