5 ms·

The biggest thing MongoDB got right was putting developer first, optimize for initial experience and development speed. Most database systems came from relatio

The biggest thing MongoDB got right was putting developer first, optimize for initial experience and development speed. Most database systems came from relation theory background and optimize for performance, operations, data safety, but often developer convenience was sacrificed. Did anybody liked writing data migration? Try explaining value of table migration to some Product Manager of early startup :-). You can argue if that's were the right tradeoffs, but that's how MongoDB get a lot of steam.

matthewmacleod 9y ago
I wouldn’t say this is something it got right - actually something it got wrong! I have never worked with a company that was sunk by the cost of migrating data between schemas. I have, however, worked with companies where the bugs that come with a schemaless database have almost sunk it. YMMV of course but I think it’s a false trade-off. I would say that the developer experience was good in the sense that it was very easy to replicate Mongo databases - but that’s more of a system administration thing. I honestly don’t think that schemaless databases add any real value.
- dfcowell 9y ago
  We're going through hell right now because the system I inherited uses Mongo for order/transaction/inventory data which benefits HUGELY from being stored in an ACID-compliant database, even when you ignore the fact that the data is highly relational and perfectly suited to a "boring" SQL DB. Mongo has its place, but its place isn't in most line-of-business applications for SMEs. "Pick the right tool for the job" has been the biggest casualty in the midst of the schemaless hype we're going through right now.
  - bonesss 9y ago
    > "Pick the right tool for the job" has been the biggest casualty in the midst of the schemaless hype we're going through right now. Even getting to the point of realizing that most domains where NoSQL or schemaless solutions thrive are also domains where you need to be thinking in suites of persistence technology, ie where trying to choose a single data persistence technology is crazy, would be a big step. A beast like Reddit (for example), has clear areas where a document centered schemaless approach would yield big wins, but that doesn't mean it's ideal for handling their advertising billing or their master data. "The job" is quite often many smaller jobs, each of which needs it's own tool. Sometimes it's a lot easier to put on slippers than to carpet the whole world.
camus2 9y ago
> Did anybody liked writing data migration? Try explaining value of table migration to some Product Manager of early startup :-). data integrity isn't good enough for a product manager? It's like saying "statically typed languages aren't worth the effort, do everything in JavaScript". Furthermore, MongoDB or not, an application in development needs DB migrations. I don't believe one second you can change all your classes or structs and pretend like MongoDB will magically update your previous documents and collections to the current state of your application.
- threeseed 9y ago
  Of course not. MongoDB is not magic. But one huge advantage of MongoDB is that it is schemaless. And so you have a lot more options for how you migrate documents. For example you can move all of the data under a particular key e.g. "v1" and allow two schemas to co-exist within the one document. Significantly less risky than running migration scripts or doing a alter schema in production.
  - amarkov 9y ago
    But it's just as easy and just as safe to allow two schemas to co-exist within the same database. If you follow the necessary steps for a risk-free data migration, having both schemas coexist simultaneously and doing gradual switchovers, both models are going to be comparably difficult. What schemaless lets you do is hide your risk better by disguising those steps. "db.table.update({$set: ...}, {multi: true})" looks friendly, but it's the same terrifying operation as running DROP TABLE A the instant you're done generating table B. And it looks easier, but that's because your code's beliefs about which schema it's reading from are hidden and implicit.
koffiezet 9y ago
> The biggest thing MongoDB got right was putting developer first, optimize for initial experience and development speed. That's absolutely true, but this is also what gave it a bad rep. It put security on the back-seat, and since it's "convenient" and just REST, it allows for moronic stuff like talking to your database straight from a client's web-browser. Security as an afterthought promotes bad or inexperienced developers to do silly stuff, and since MongoDB focuses primarily on web applications - that is a really really bad thing. How many leaks have there been simply because the database was exposed to the public internet without any authentication? Too many. Now the problem is that inexperienced devs will flock to something that makes it that easy, and there's no real way of preventing that, but developers should never be put first - it leads to lazy solutions and compromises on too many other levels.
deleted 9y ago
[deleted]
ssijak 9y ago
"Did anybody liked writing data migration? Try explaining value of table migration to some Product Manager of early startup :-)." People are so spoiled this days. What is so hard in writing table migration, it is like a few minutes of your time. And why would some non techy product manager even know that you wrote some migration sql file. It is a minuscule technical detail related to the chosen technology and he will not see any repercussion in the production. But try to explain to him why after some reckless sprints you got many inconsistencies in the user facing app which you will have if you abuse "schemaless" NoSQL but use implicit schemas which change over time without proper checks. And if my product manager will push just for initial speed all the time without 0 time for some basic architecture (and I mean really basic), engineering practices and such, you know really useful stuff, I would leave that place without second thought. It is digging it`s own hole from the beginning. And what is the problem of data migrations? Implicit schemas in MongoDB are much worse of a problem when your data requirements change. With relational databases you at least have integrity checks on the DB level that guard you from much of the possible problems when doing such migrations.
- drinchev 9y ago
  > What is so hard in writing table migration, it is like a few minutes of your time Wow, I wrote migration script for a couple of table rationalisation and it took me a week, to make it error-prone and to provide a revert option. Not to mention that you usually need to work with production data on your local to test if the migration will work without any errors. And If you happen to have a migration error on your production database. It is a terrible experience, which usually includes downtime. Maybe my lack of experience on writing SQL migrations increased that time, but isn't this the whole point of the OP in terms of dev first.
  - ssijak 9y ago
    What you wrote is data migration. Table/database migrations are just DDL scripts which are usually really small on an ongoing project. For example you add 2 columns in a sprint so you have that in the SQL migration file for that sprint.
igor_filippov 9y ago
Nothing comes for free. If your schema doesn't live in your DB, then it lives in your app code. In the end of the day, you have to handle data model changes.
- lmm 9y ago
  True as far as it goes, but I've got much better tools available at the app code level.
  - ssijak 9y ago
    No you don`t. DB should be a single source of truth and there is no better tool that cares for the integrity of the data then the DB itself. If you have highly concurrent web app, and X connections are changing same collection, good luck with keeping the data sane without checks and transactions on the DB level. And let me not even mention if you have several different apps/services accessing the same db.
  - purerandomness 9y ago
    Why use a database then, at all? Just use plain text files.
    - lmm 9y ago
      Managing replication and consensus for me is valuable. Map/reduce infrastructure is valuable. Forcing me to fit my data into a square table model is not helpful. Transactions at the data layer are rarely helpful (transactional behaviour always needs cooperation from the application layer, and that's very difficult to achieve with a traditional RDBMs; almost all database-backed applications I've seen don't actually offer useful transactionality). My tool of choice is Cassandra or similar.
karmakaze 9y ago
> The biggest thing MongoDB got right was putting developer first, optimize for initial experience and development speed Specifically, I would say mobile and node.js developers. The other part that worked well was so sharding, out-of-the-box. Oh, there were major issues like an index updates skipping documents, but it appeared to work and was the easiest to set up.
smnc 9y ago
In a world where developer convenience wins over performance, operations and data safety a significant number of those same developers argue over whether they should be referred to as computer scientists, engineers, architects, or painters. Painters sounds about right.
- ssijak 9y ago
  Agree. If civil engineer chose risky/bad materials for your house because it would be convenient and you would get a house 30 days faster I cant imagine a person not being mad about that.
  - Slartie 9y ago
    So true. Whenever stuff gets "convenient" for me as a developer, I ask myself what the downside of this "convenience" for me is that will have to be endured by others. Often times it's inconveniences for the guys who have to operate the thing that I develop later (the single greatest things about this DevOps hype is that it made quite some developers experience the operational pain that their creations inflict first-hand, thereby making them less likely to take "convenient-for-them" shortcuts that make operations much less convenient). Sometimes it's inconveniences for other developers working on the same project. Other times it's inconveniences for developers maintaining the product later (which might as well include me, too). Whatever the inconvenience is, I need to be able to stand in for them being an acceptable price to pay for my personal convenience. And that is much more likely to be the case if I carry the "inconvenience" part of the equation, while others get the "convenience" part - like the operations people, or the end user, or other developers. That's because I consider myself a professional who's getting paid good money to make other people's life easier, especially if those "other people" vastly outnumber myself or the few other devs on my project team. That is the essence of my job, and it's not intended to be "convenient" for me - developer convenience is therefore one of the least-relevant criteria by which I ought to choose how I'm going to do it. Developers who highly value "convenient" solutions are bloody amateurs in my book, especially if they can't even envision the ways in which their convenience has to be paid for by others.
tjpnz 9y ago
Developer convenience is not a substitute for sitting down and learning SQL - but a good number of people I've met over the years plugging MongoDB et el have at some point thrown their arms in the air at the prospect of doing that. Here's the thing, in practice relational databases aren't hard, nor will most devs ever need to know what relational theory is in order to be productive with databases built on it. Performance optimization does get a bit harder but there are relatively few situations where there's any magic involved, knowing how to read an explain plan and when/where to add an index is what most of it boils down to.
- jamaicahest 9y ago
  As someone who actually bothered to sit down and spend the effort learning SQL, I agree. I've seen many colleagues in my career say "Oh SQL is hard/stupid/broken/whatever. MongoDB is so easy, it just works." and when pressed on whether they actually bothered to try and learn SQL, the answer is always no. Seems developers today are so lazy that we can't be bothered to learn a technology, if it's different from the ones we already know.
pleasecalllater 9y ago
> Did anybody liked writing data migration? Yes, I like it. And you know what? My databases have lots of checks in the schema. This way I know, that I get correct data when I query the database. In MongoDB I'm not sure. I can throw there anything, and get anything. It's just a normal GIGO queue. And the migrations are fine. You just need to know the tools. I observed huge programmers' aversion to learn SQL (however they don't complain when they need to learn HQL).
ivanhoe 9y ago
Mongo or not, you have to migrate your old data at some point, or as the project grows you'll end up adding a layer over layer of complexity in your code to handle all the accumulated inconsistencies.

matthewmacleod 9y ago

I wouldn’t say this is something it got right - actually something it got wrong! I have never worked with a company that was sunk by the cost of migrating data between schemas. I have, however, worked with companies where the bugs that come with a schemaless database have almost sunk it. YMMV of course but I think it’s a false trade-off. I would say that the developer experience was good in the sense that it was very easy to replicate Mongo databases - but that’s more of a system administration thing. I honestly don’t think that schemaless databases add any real value.

dfcowell 9y ago
We're going through hell right now because the system I inherited uses Mongo for order/transaction/inventory data which benefits HUGELY from being stored in an ACID-compliant database, even when you ignore the fact that the data is highly relational and perfectly suited to a "boring" SQL DB. Mongo has its place, but its place isn't in most line-of-business applications for SMEs. "Pick the right tool for the job" has been the biggest casualty in the midst of the schemaless hype we're going through right now.
- bonesss 9y ago
  > "Pick the right tool for the job" has been the biggest casualty in the midst of the schemaless hype we're going through right now. Even getting to the point of realizing that most domains where NoSQL or schemaless solutions thrive are also domains where you need to be thinking in suites of persistence technology, ie where trying to choose a single data persistence technology is crazy, would be a big step. A beast like Reddit (for example), has clear areas where a document centered schemaless approach would yield big wins, but that doesn't mean it's ideal for handling their advertising billing or their master data. "The job" is quite often many smaller jobs, each of which needs it's own tool. Sometimes it's a lot easier to put on slippers than to carpet the whole world.

camus2 9y ago

> Did anybody liked writing data migration? Try explaining value of table migration to some Product Manager of early startup :-). data integrity isn't good enough for a product manager? It's like saying "statically typed languages aren't worth the effort, do everything in JavaScript". Furthermore, MongoDB or not, an application in development needs DB migrations. I don't believe one second you can change all your classes or structs and pretend like MongoDB will magically update your previous documents and collections to the current state of your application.

threeseed 9y ago
Of course not. MongoDB is not magic. But one huge advantage of MongoDB is that it is schemaless. And so you have a lot more options for how you migrate documents. For example you can move all of the data under a particular key e.g. "v1" and allow two schemas to co-exist within the one document. Significantly less risky than running migration scripts or doing a alter schema in production.
- amarkov 9y ago
  But it's just as easy and just as safe to allow two schemas to co-exist within the same database. If you follow the necessary steps for a risk-free data migration, having both schemas coexist simultaneously and doing gradual switchovers, both models are going to be comparably difficult. What schemaless lets you do is hide your risk better by disguising those steps. "db.table.update({$set: ...}, {multi: true})" looks friendly, but it's the same terrifying operation as running DROP TABLE A the instant you're done generating table B. And it looks easier, but that's because your code's beliefs about which schema it's reading from are hidden and implicit.

koffiezet 9y ago

> The biggest thing MongoDB got right was putting developer first, optimize for initial experience and development speed. That's absolutely true, but this is also what gave it a bad rep. It put security on the back-seat, and since it's "convenient" and just REST, it allows for moronic stuff like talking to your database straight from a client's web-browser. Security as an afterthought promotes bad or inexperienced developers to do silly stuff, and since MongoDB focuses primarily on web applications - that is a really really bad thing. How many leaks have there been simply because the database was exposed to the public internet without any authentication? Too many. Now the problem is that inexperienced devs will flock to something that makes it that easy, and there's no real way of preventing that, but developers should never be put first - it leads to lazy solutions and compromises on too many other levels.

deleted 9y ago

[deleted]

ssijak 9y ago

"Did anybody liked writing data migration? Try explaining value of table migration to some Product Manager of early startup :-)." People are so spoiled this days. What is so hard in writing table migration, it is like a few minutes of your time. And why would some non techy product manager even know that you wrote some migration sql file. It is a minuscule technical detail related to the chosen technology and he will not see any repercussion in the production. But try to explain to him why after some reckless sprints you got many inconsistencies in the user facing app which you will have if you abuse "schemaless" NoSQL but use implicit schemas which change over time without proper checks. And if my product manager will push just for initial speed all the time without 0 time for some basic architecture (and I mean really basic), engineering practices and such, you know really useful stuff, I would leave that place without second thought. It is digging it`s own hole from the beginning. And what is the problem of data migrations? Implicit schemas in MongoDB are much worse of a problem when your data requirements change. With relational databases you at least have integrity checks on the DB level that guard you from much of the possible problems when doing such migrations.

drinchev 9y ago
> What is so hard in writing table migration, it is like a few minutes of your time Wow, I wrote migration script for a couple of table rationalisation and it took me a week, to make it error-prone and to provide a revert option. Not to mention that you usually need to work with production data on your local to test if the migration will work without any errors. And If you happen to have a migration error on your production database. It is a terrible experience, which usually includes downtime. Maybe my lack of experience on writing SQL migrations increased that time, but isn't this the whole point of the OP in terms of dev first.
- ssijak 9y ago
  What you wrote is data migration. Table/database migrations are just DDL scripts which are usually really small on an ongoing project. For example you add 2 columns in a sprint so you have that in the SQL migration file for that sprint.

igor_filippov 9y ago

Nothing comes for free. If your schema doesn't live in your DB, then it lives in your app code. In the end of the day, you have to handle data model changes.

lmm 9y ago
True as far as it goes, but I've got much better tools available at the app code level.
- ssijak 9y ago
  No you don`t. DB should be a single source of truth and there is no better tool that cares for the integrity of the data then the DB itself. If you have highly concurrent web app, and X connections are changing same collection, good luck with keeping the data sane without checks and transactions on the DB level. And let me not even mention if you have several different apps/services accessing the same db.
- purerandomness 9y ago
  Why use a database then, at all? Just use plain text files.
  - lmm 9y ago
    Managing replication and consensus for me is valuable. Map/reduce infrastructure is valuable. Forcing me to fit my data into a square table model is not helpful. Transactions at the data layer are rarely helpful (transactional behaviour always needs cooperation from the application layer, and that's very difficult to achieve with a traditional RDBMs; almost all database-backed applications I've seen don't actually offer useful transactionality). My tool of choice is Cassandra or similar.

karmakaze 9y ago

> The biggest thing MongoDB got right was putting developer first, optimize for initial experience and development speed Specifically, I would say mobile and node.js developers. The other part that worked well was so sharding, out-of-the-box. Oh, there were major issues like an index updates skipping documents, but it appeared to work and was the easiest to set up.

smnc 9y ago

In a world where developer convenience wins over performance, operations and data safety a significant number of those same developers argue over whether they should be referred to as computer scientists, engineers, architects, or painters. Painters sounds about right.

ssijak 9y ago
Agree. If civil engineer chose risky/bad materials for your house because it would be convenient and you would get a house 30 days faster I cant imagine a person not being mad about that.
- Slartie 9y ago
  So true. Whenever stuff gets "convenient" for me as a developer, I ask myself what the downside of this "convenience" for me is that will have to be endured by others. Often times it's inconveniences for the guys who have to operate the thing that I develop later (the single greatest things about this DevOps hype is that it made quite some developers experience the operational pain that their creations inflict first-hand, thereby making them less likely to take "convenient-for-them" shortcuts that make operations much less convenient). Sometimes it's inconveniences for other developers working on the same project. Other times it's inconveniences for developers maintaining the product later (which might as well include me, too). Whatever the inconvenience is, I need to be able to stand in for them being an acceptable price to pay for my personal convenience. And that is much more likely to be the case if I carry the "inconvenience" part of the equation, while others get the "convenience" part - like the operations people, or the end user, or other developers. That's because I consider myself a professional who's getting paid good money to make other people's life easier, especially if those "other people" vastly outnumber myself or the few other devs on my project team. That is the essence of my job, and it's not intended to be "convenient" for me - developer convenience is therefore one of the least-relevant criteria by which I ought to choose how I'm going to do it. Developers who highly value "convenient" solutions are bloody amateurs in my book, especially if they can't even envision the ways in which their convenience has to be paid for by others.

tjpnz 9y ago

Developer convenience is not a substitute for sitting down and learning SQL - but a good number of people I've met over the years plugging MongoDB et el have at some point thrown their arms in the air at the prospect of doing that. Here's the thing, in practice relational databases aren't hard, nor will most devs ever need to know what relational theory is in order to be productive with databases built on it. Performance optimization does get a bit harder but there are relatively few situations where there's any magic involved, knowing how to read an explain plan and when/where to add an index is what most of it boils down to.

jamaicahest 9y ago
As someone who actually bothered to sit down and spend the effort learning SQL, I agree. I've seen many colleagues in my career say "Oh SQL is hard/stupid/broken/whatever. MongoDB is so easy, it just works." and when pressed on whether they actually bothered to try and learn SQL, the answer is always no. Seems developers today are so lazy that we can't be bothered to learn a technology, if it's different from the ones we already know.

pleasecalllater 9y ago

> Did anybody liked writing data migration? Yes, I like it. And you know what? My databases have lots of checks in the schema. This way I know, that I get correct data when I query the database. In MongoDB I'm not sure. I can throw there anything, and get anything. It's just a normal GIGO queue. And the migrations are fine. You just need to know the tools. I observed huge programmers' aversion to learn SQL (however they don't complain when they need to learn HQL).

ivanhoe 9y ago

Mongo or not, you have to migrate your old data at some point, or as the project grows you'll end up adding a layer over layer of complexity in your code to handle all the accumulated inconsistencies.