8 ms·

We are first evaluating MongoDB. I believe the main reason behind this is we are already using Mongo in other parts of our application, so there is no additiona

by Ankhers 11y ago

We are first evaluating MongoDB. I believe the main reason behind this is we are already using Mongo in other parts of our application, so there is no additional setup when converting. Note that nothing is set in stone. The decision to begin migrations only happened today. It is possible that we will end up using some other technology altogether, or even we find out the issues we are having with Aerospike and continue using that service.

deleted 11y ago
[deleted]
misframer 11y ago
You should take a look at "Call me maybe: MongoDB" [0] [0] https://aphyr.com/posts/284-call-me-maybe-mongodb https://aphyr.com/posts/284-call-me-maybe-mongodb
- sans-culottes 11y ago
  Not to mention "Call me maybe: Redis" [0]. Rinse and repeat for every NoSQL database out there. Point is, @aphyr skewers everybody. Aerospike is just the flavor of the month. [0] https://aphyr.com/posts/283-call-me-maybe-redis https://aphyr.com/posts/283-call-me-maybe-redis
  - jbergens 11y ago
    I think FoundationDb got great scores from Aphyr, sadly it is now owned by Apple. For the others the big problem is when they promise some kind of ACID and cannot achieve it, if they were explicit with what the supported all customers could make informed decisions. Relations systems are generally bad ad horizontal scaling and can get very slow with full ACID over many servers.
    - masklinn 11y ago
      > I think FoundationDb got great scores from Aphyr, FoundationDB ran Jepsen internally and reported stuff[0], Kyle never worked with it. [0] http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-jepsen http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-j... half broken now, none of the images load for me. @aphyr seems to have taken them at their word wrt testing though: https://twitter.com/aphyr/status/405017101804396546 https://twitter.com/aphyr/status/405017101804396546
      - annnnd 11y ago
        Curious: how difficult is it to replicate aphyr's tests? Can we beleive independent entities posting jepsen reports on some random DB?
  - woozy 11y ago
    Antirez was very receptive http://antirez.com/news/55 http://antirez.com/news/55
    - masklinn 11y ago
      Aphyr wasn't overly impressed by that response though https://aphyr.com/posts/287-asynchronous-replication-with-failover https://aphyr.com/posts/287-asynchronous-replication-with-fa...
  - lmm 11y ago
    I think that's a false equivalence. Aphyr is pretty positive about Riak (with the correct configuration) and Cassandra (when used for appropriate scenarios). If I was choosing a new system those are the two I'd be looking at.
bluef00t 11y ago
Try HBase. At another AdTech company, we have been very happy with it. https://eng.yammer.com/call-me-maybe-hbase https://eng.yammer.com/call-me-maybe-hbase
- annnnd 11y ago
  Nooooooooooooooooooooo! Seriously, no! Use mongoDB, PostgreSQL, even flat files if you must, but HBase? We used it in production about 3-4 years ago and it was a nightmare from both usage and especially maintenance point. Fortunately we had a flat-files based backup system so we were able to rescue data every! Single! Time! the damn thing crashed and took (part of) data with it. Of course, this is anecdotal evidence, and things might have changed from then, but I wouldn't touch it. Life is too short. EDIT: Also, I am curious how the results in the above link would compare to aphyr's if he performed the test on HBase?
  - bluef00t 11y ago
    I see where you are coming from. HBase was unstable 3-4 years ago, but after a great amount of dev effort and battle hardening from Cloudera, Salesforce, etc., it is very stable now. We have ~ 400 nodes running in production for a very critical use case and have seen 0 data loss edge cases in the last 2 years, along with some of our servers running > 6 months without any reboots. We use is in a very real time use case with latency requirements of single digit milliseconds, and if you tweak it the right way, you can the required performance from it, along with easy horizontal scaling. Also, I am curious too for aphyr to take on HBase, but I don't think the result would be different since running Jepsen is straightfoward and not much to a person's interpretation. The results and further experiments are what aphyr does nicely.
    - annnnd 11y ago
      Thanks for the info on HBase stability. I probably won't use it again (once burnt...), but if they really managed to pull their act together - good for them!
ploxiln 11y ago
oh... dude... exact same mistake twice... OK, let me try to be more constructive. Since accounts are independent, shard based on account (in the application, not in some magic shard-distributing layer). Treat each shard as its own cluster. If you want super fast requests but can accept being down for an hour or two a couple times a year, a shard can be a single beefy host with a replicating slave. I'd consider either Redis or Mysql/Postgresql. Really, these old-style sql databases can be the fastest things that have the kind of consistency you need. I've maintained a mongodb cluster configured a couple of different ways. Performance at high load and reasonable consistency is not as great as some older alternatives.

by Ankhers 11y ago

deleted 11y ago

[deleted]

misframer 11y ago

You should take a look at "Call me maybe: MongoDB" [0] [0] https://aphyr.com/posts/284-call-me-maybe-mongodb https://aphyr.com/posts/284-call-me-maybe-mongodb

sans-culottes 11y ago
Not to mention "Call me maybe: Redis" [0]. Rinse and repeat for every NoSQL database out there. Point is, @aphyr skewers everybody. Aerospike is just the flavor of the month. [0] https://aphyr.com/posts/283-call-me-maybe-redis https://aphyr.com/posts/283-call-me-maybe-redis
- jbergens 11y ago
  I think FoundationDb got great scores from Aphyr, sadly it is now owned by Apple. For the others the big problem is when they promise some kind of ACID and cannot achieve it, if they were explicit with what the supported all customers could make informed decisions. Relations systems are generally bad ad horizontal scaling and can get very slow with full ACID over many servers.
  - masklinn 11y ago
    > I think FoundationDb got great scores from Aphyr, FoundationDB ran Jepsen internally and reported stuff[0], Kyle never worked with it. [0] http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-jepsen http://blog.foundationdb.com/call-me-maybe-foundationdb-vs-j... half broken now, none of the images load for me. @aphyr seems to have taken them at their word wrt testing though: https://twitter.com/aphyr/status/405017101804396546 https://twitter.com/aphyr/status/405017101804396546
    - annnnd 11y ago
      Curious: how difficult is it to replicate aphyr's tests? Can we beleive independent entities posting jepsen reports on some random DB?
- woozy 11y ago
  Antirez was very receptive http://antirez.com/news/55 http://antirez.com/news/55
  - masklinn 11y ago
    Aphyr wasn't overly impressed by that response though https://aphyr.com/posts/287-asynchronous-replication-with-failover https://aphyr.com/posts/287-asynchronous-replication-with-fa...
- lmm 11y ago
  I think that's a false equivalence. Aphyr is pretty positive about Riak (with the correct configuration) and Cassandra (when used for appropriate scenarios). If I was choosing a new system those are the two I'd be looking at.

bluef00t 11y ago

Try HBase. At another AdTech company, we have been very happy with it. https://eng.yammer.com/call-me-maybe-hbase https://eng.yammer.com/call-me-maybe-hbase

annnnd 11y ago
Nooooooooooooooooooooo! Seriously, no! Use mongoDB, PostgreSQL, even flat files if you must, but HBase? We used it in production about 3-4 years ago and it was a nightmare from both usage and especially maintenance point. Fortunately we had a flat-files based backup system so we were able to rescue data every! Single! Time! the damn thing crashed and took (part of) data with it. Of course, this is anecdotal evidence, and things might have changed from then, but I wouldn't touch it. Life is too short. EDIT: Also, I am curious how the results in the above link would compare to aphyr's if he performed the test on HBase?
- bluef00t 11y ago
  I see where you are coming from. HBase was unstable 3-4 years ago, but after a great amount of dev effort and battle hardening from Cloudera, Salesforce, etc., it is very stable now. We have ~ 400 nodes running in production for a very critical use case and have seen 0 data loss edge cases in the last 2 years, along with some of our servers running > 6 months without any reboots. We use is in a very real time use case with latency requirements of single digit milliseconds, and if you tweak it the right way, you can the required performance from it, along with easy horizontal scaling. Also, I am curious too for aphyr to take on HBase, but I don't think the result would be different since running Jepsen is straightfoward and not much to a person's interpretation. The results and further experiments are what aphyr does nicely.
  - annnnd 11y ago
    Thanks for the info on HBase stability. I probably won't use it again (once burnt...), but if they really managed to pull their act together - good for them!

ploxiln 11y ago

oh... dude... exact same mistake twice... OK, let me try to be more constructive. Since accounts are independent, shard based on account (in the application, not in some magic shard-distributing layer). Treat each shard as its own cluster. If you want super fast requests but can accept being down for an hour or two a couple times a year, a shard can be a single beefy host with a replicating slave. I'd consider either Redis or Mysql/Postgresql. Really, these old-style sql databases can be the fastest things that have the kind of consistency you need. I've maintained a mongodb cluster configured a couple of different ways. Performance at high load and reasonable consistency is not as great as some older alternatives.