5 ms·

I inherited a MongoDB Ruby (Mongoid) stack that was a terrible choice for the data stored (i.e. relational data). Actually, I'm not really sure why Mongoid exis

I inherited a MongoDB Ruby (Mongoid) stack that was a terrible choice for the data stored (i.e. relational data). Actually, I'm not really sure why Mongoid exists at all. After several years of pain e.g. cursor timeouts (even when supposedly disabled), iterating over the same document multiple times in the one query etc. and of course no transactions (since added). I eventually migrated to PSQL (using some custom scripts) and everything is so much easier to maintain, much more stable and much faster. We're still using MongoDB for our analytics though, as it's inherently unstructured non-relational data, so MongoDB does actually do an okay job. Well, except for the run away memory usage which leads to k8s killing the DB server for violating memory constraints every so often. Yes, MongoDB itself is configured with memory constraints, it seems WiredTiger ignores them. Licensing was considered, however honestly MongoDB itself is (or at least was) so bad that licensing wasn't the primary motivating factor. EDIT: Just to clarify, most of our data is relational, however we do have some unstructured data in PSQL stored in JSONB columns. They work a treat and in some cases have complex indexes on them - they far out-perform anything MongoDB was able to offer us.

SergeAx 7y ago
What is the point of running containerized DB server in production? Isn't it an anti-pattern?
- Benjamin_Dobell 7y ago
  I wouldn't go as far as saying it's an anti-pattern. However, doing it properly is extremely difficult. Why do we do it? Cost saving. It's that simple. For your information, I wouldn't say we do it "properly" either. We do a decent job though. Basically, for us (and many other businesses) DBs are not the bottle-neck. Our business model facilitates a heap of static content that's served (via a CDN) without even hitting the DB. When the DB is used, it's 99% read-only; except for the analytics which aren't business critical. As with all things, it's a trade-off.
  - SergeAx 7y ago
    Sounds not that cost-saving for me when you mention that your not critical DB OOM-crashes here and there. How much do you saving, like $150/year on dedicated host?
    - Benjamin_Dobell 7y ago
      You're assuming that I'm just talking about hosting costs. I'm not. Maintaining VPS infrastructure just to run a DB is a huge amount of overhead. More so, when the DB crashes and/or wildly consumes memory. You need disk/memory monitoring, logging, process monitoring, availability monitoring etc. What if the VPS itself goes down? What automated system is going to spin it back up? A self-managed VPS DIY solution is substantially more expensive to maintain than using well-established tooling like k8s. These are all already solved problems. Not just that, it's a standardised work-flow with the rest of our infrastructure i.e. no manual futzing around with VPC etc. to get our cluster and externally managed VPS talking to each other. For us, a DIY solution was never even on the cards. It was either a fully managed service or what we have now. However, fully managed services don't just cost more per month. There's frequently additional development overhead, as the offerings are "off the shelf" software and not necessarily easily customised. For example, right now our PSQL deployment is running a custom PSQL extension that seamlessly handles BSON (MongoDB) Object IDs, semantically converting them to UUIDs. Thus, post-migration from MongoDB, application code isn't riddled with code handling legacy IDs. Monthly service provider fees are but one consideration when performing costings. EDIT: It's probably also worth noting that our entire infrastructure, including DBs, can be spun up as a k8s cluster on a developer's work machine in one command. We of course also have a staging environment, which mimics production. Development also ought to mimic production as closely as possible (but still be debuggable).
- merb 7y ago
  actually it works quite well when done correctly. basically k8s is less of an container solution and more like a "scheduling" solution. so it's actually way less painfull than setting up your own db.
- alchemism 7y ago
  That’s usually a sign that there is no Ops on staff to prevent the Devs from doing it, yes.
- humble_engineer 7y ago
  You're a bigot.

SergeAx 7y ago

What is the point of running containerized DB server in production? Isn't it an anti-pattern?

Benjamin_Dobell 7y ago
I wouldn't go as far as saying it's an anti-pattern. However, doing it properly is extremely difficult. Why do we do it? Cost saving. It's that simple. For your information, I wouldn't say we do it "properly" either. We do a decent job though. Basically, for us (and many other businesses) DBs are not the bottle-neck. Our business model facilitates a heap of static content that's served (via a CDN) without even hitting the DB. When the DB is used, it's 99% read-only; except for the analytics which aren't business critical. As with all things, it's a trade-off.
- SergeAx 7y ago
  Sounds not that cost-saving for me when you mention that your not critical DB OOM-crashes here and there. How much do you saving, like $150/year on dedicated host?
  - Benjamin_Dobell 7y ago
    You're assuming that I'm just talking about hosting costs. I'm not. Maintaining VPS infrastructure just to run a DB is a huge amount of overhead. More so, when the DB crashes and/or wildly consumes memory. You need disk/memory monitoring, logging, process monitoring, availability monitoring etc. What if the VPS itself goes down? What automated system is going to spin it back up? A self-managed VPS DIY solution is substantially more expensive to maintain than using well-established tooling like k8s. These are all already solved problems. Not just that, it's a standardised work-flow with the rest of our infrastructure i.e. no manual futzing around with VPC etc. to get our cluster and externally managed VPS talking to each other. For us, a DIY solution was never even on the cards. It was either a fully managed service or what we have now. However, fully managed services don't just cost more per month. There's frequently additional development overhead, as the offerings are "off the shelf" software and not necessarily easily customised. For example, right now our PSQL deployment is running a custom PSQL extension that seamlessly handles BSON (MongoDB) Object IDs, semantically converting them to UUIDs. Thus, post-migration from MongoDB, application code isn't riddled with code handling legacy IDs. Monthly service provider fees are but one consideration when performing costings. EDIT: It's probably also worth noting that our entire infrastructure, including DBs, can be spun up as a k8s cluster on a developer's work machine in one command. We of course also have a staging environment, which mimics production. Development also ought to mimic production as closely as possible (but still be debuggable).
merb 7y ago
actually it works quite well when done correctly. basically k8s is less of an container solution and more like a "scheduling" solution. so it's actually way less painfull than setting up your own db.
alchemism 7y ago
That’s usually a sign that there is no Ops on staff to prevent the Devs from doing it, yes.
humble_engineer 7y ago
You're a bigot.