4 ms·

Wow was MongoDB really that bad?

emmanueloga_ 8y ago
Possible alternative headline: "Company weights in trade-offs of having two different database systems, decide to consolidate into a single one." :-) They don't give too many reasons, but say that operationally it is gonna be easier for them to work with a single database than with two (no-brainer?)
- dralley 8y ago
  As someone on the team, that didn't factor into the decision too much, although it's definitely a nice perk. Switching to Postgres made a lot of sense even when looking at Pulp as a standalone project rather than as a part of Satellite.
- taude 8y ago
  Except, I'd argue that most large distributed systems these days are using much more than two types of data stores, especially when you start including name/value keystores (and in memory caching variants), big data pipelines, transactional data, "unstructured" data storage (like Mongo), time series, etc. I work in FinTech and you'll see several backing our apps, and this isn't uncommon.
- zaphirplane 8y ago
  The licensing change surely has to have played a factor
foobiekr 8y ago
A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things like typos, etc. become impossible. Relational structures enforced in the database (and imho sadly lacking in the intervening layers which is something I’d like to get the time to deal with; though the FoubdationDB record layer certainly has an interesting if unexploited take on it) is a natural extension of this. But there’s a whole other world where people just don’t want to be tied to constraints. I suspect it’s a personality thing.
- mlevental 8y ago
  how do you do this? tie it all the way from the DB to the client? incidentally I believe this is the difference between being an engineer and being a product person. I straddle the line. in theory I want a strong type system so I can be confident about the future. in practice I don't want to pay the upfront cost and I'm more interested in shipping now and taking on debt.
  - jacques_chester 8y ago
    What upfront cost?
  - atombender 8y ago
    Look at GraphQL. It lets you enforce a single schema from the front end down to the data layer. In my company's product, we specify the data models as JSON Schema, and then generate the necessary language code -- we generate the GraphQL schema from it as well as Go data types, with some database glue. Our front end code is currently JavaScript, but we hope to migrate to TypeScript, which will make everything statically types all the way through. (gRPC fills a similar role, though the web story is lacking. gRPC is brilliant for APIs between backend services, but browser support is not there yet. GraphQL is more convenient for the app-facing layers.)
    - mlevental 8y ago
      https://github.com/grpc/grpc-web https://github.com/grpc/grpc-web
      - atombender 8y ago
        Yes, you can make it work. But you have to run this special shim or proxy, because browsers still don't all support HTTP/2, header trailers, etc. Hence "lacking".
  - foobiekr 8y ago
    We wrote it. Everything is derived from proto for us. Some of that is hand-written tooling, some is off the shelf. Mostly you have to be aware at design time of what this approach entails. Proto is not elegant and the data model and implementation (especially the very poor performance of maps in the Go implementation) can leave a lot to be desired, but it's a thing that pretty much supports everything. It has good enough performance without headaches. Lowest common denominator can be quite useful and so can constraints.
mehrdadn 8y ago
I'm not sure, but I've seen things like this: https://twitter.com/hackernewsonion/status/382578547069837312 https://twitter.com/hackernewsonion/status/38257854706983731...
HillRat 8y ago
Generally, if you’re going to standardize on a single data repository, a traditional RDBMS will be a better fit than a NoSQL solution, unless you really need polymorphic capabilities and have a document-centric conceptual model (such as a CMS or certain kinds of contact center solutions). So this is really less about MongoDB being bad than relational being more appropriate. MongoDB has definitely come a long way from when they were dropping writes and hitting scaling limits, and their new tools ecosystem holds a lot of promise, but I treat them much like any other non-RDBMS store (Redis, Cassandra, Neo4J...) in that they’re very specific tools for specific needs, not something that can be kit-bashed for any general class of problem.

Possible alternative headline: "Company weights in trade-offs of having two different database systems, decide to consolidate into a single one." :-) They don't give too many reasons, but say that operationally it is gonna be easier for them to work with a single database than with two (no-brainer?)

dralley 8y ago
As someone on the team, that didn't factor into the decision too much, although it's definitely a nice perk. Switching to Postgres made a lot of sense even when looking at Pulp as a standalone project rather than as a part of Satellite.
taude 8y ago
Except, I'd argue that most large distributed systems these days are using much more than two types of data stores, especially when you start including name/value keystores (and in memory caching variants), big data pipelines, transactional data, "unstructured" data storage (like Mongo), time series, etc. I work in FinTech and you'll see several backing our apps, and this isn't uncommon.
zaphirplane 8y ago
The licensing change surely has to have played a factor

foobiekr 8y ago

A way to think about that question is to ask whether typelessness in general is bad because a lot of the schemaless databases come from that side of the divide. I’d personally answer yes, but that’s because I believe that typed, schema-generates structures should be pushed all the way from the database to the typescript code on the front end. Changes become something you can deal with with high confidence and things like typos, etc. become impossible. Relational structures enforced in the database (and imho sadly lacking in the intervening layers which is something I’d like to get the time to deal with; though the FoubdationDB record layer certainly has an interesting if unexploited take on it) is a natural extension of this. But there’s a whole other world where people just don’t want to be tied to constraints. I suspect it’s a personality thing.

mlevental 8y ago
how do you do this? tie it all the way from the DB to the client? incidentally I believe this is the difference between being an engineer and being a product person. I straddle the line. in theory I want a strong type system so I can be confident about the future. in practice I don't want to pay the upfront cost and I'm more interested in shipping now and taking on debt.
- jacques_chester 8y ago
  What upfront cost?
- atombender 8y ago
  Look at GraphQL. It lets you enforce a single schema from the front end down to the data layer. In my company's product, we specify the data models as JSON Schema, and then generate the necessary language code -- we generate the GraphQL schema from it as well as Go data types, with some database glue. Our front end code is currently JavaScript, but we hope to migrate to TypeScript, which will make everything statically types all the way through. (gRPC fills a similar role, though the web story is lacking. gRPC is brilliant for APIs between backend services, but browser support is not there yet. GraphQL is more convenient for the app-facing layers.)
  - mlevental 8y ago
    https://github.com/grpc/grpc-web https://github.com/grpc/grpc-web
    - atombender 8y ago
      Yes, you can make it work. But you have to run this special shim or proxy, because browsers still don't all support HTTP/2, header trailers, etc. Hence "lacking".
- foobiekr 8y ago
  We wrote it. Everything is derived from proto for us. Some of that is hand-written tooling, some is off the shelf. Mostly you have to be aware at design time of what this approach entails. Proto is not elegant and the data model and implementation (especially the very poor performance of maps in the Go implementation) can leave a lot to be desired, but it's a thing that pretty much supports everything. It has good enough performance without headaches. Lowest common denominator can be quite useful and so can constraints.

mehrdadn 8y ago

I'm not sure, but I've seen things like this: https://twitter.com/hackernewsonion/status/382578547069837312 https://twitter.com/hackernewsonion/status/38257854706983731...

HillRat 8y ago

Generally, if you’re going to standardize on a single data repository, a traditional RDBMS will be a better fit than a NoSQL solution, unless you really need polymorphic capabilities and have a document-centric conceptual model (such as a CMS or certain kinds of contact center solutions). So this is really less about MongoDB being bad than relational being more appropriate. MongoDB has definitely come a long way from when they were dropping writes and hitting scaling limits, and their new tools ecosystem holds a lot of promise, but I treat them much like any other non-RDBMS store (Redis, Cassandra, Neo4J...) in that they’re very specific tools for specific needs, not something that can be kit-bashed for any general class of problem.