3 ms·

This works only if you use an RDBMS (instead of a NoSQL solution) and you use it as a very dumb data store. For large classes of applications you lose far more

This works only if you use an RDBMS (instead of a NoSQL solution) and you use it as a very dumb data store. For large classes of applications you lose far more than you gain. The reason is that you are FORCED to refactor your data with an RDBMS. As soon as you move outside the RDMBS storage model with agile development something very icky happens. Old data structures stick around until actively cleaned up using processes which may take a long time to complete. If you have half a billion documents in Mongo DB created and used by your agile app, you are going to have to have your application handle those data-structure-wise forever. Every new change then creates an additional corner case that must be perpetually handled.

wpietri 14y ago
I don't get why you think NoSQL solutions are exempt from refactoring and maintaining data quality. I've happily done that with non-SQL stores. No matter what development approach you use, you will eventually have to deal with change. No interesting application hits v 1.0 and sticks there. The only question is whether you embrace change, which is the Agile approach, or whether you resist it. If you resist it, you change in big lurches when circumstances force it. Having tried both, I favor the former regardless of what storage mechanism I'm using.
- einhverfr 14y ago
  Refactoring your data structures is different though, is it not? In an RDBMS, you have to migrate your data when the schema changes, correct? If you have, say, 100 million records in Mongo DB and you decide you need to refactor your data schemas, how long will it take you to do that? The problem is that an RDBMS is very rigid typically on data input but very flexible on data output. if you change data input you MUST migrate data with it. MongoDB and others are very flexible with data input but what you get out is more or less what you have told it to store. This is a very different problem. You lose the ability to continue to refactor your data structure semantics beyond a certain point. This is not really the case with an RDBMS. BTW, I have started to get PostgreSQL object-relational modelling and it is amazing. I suspect that although the barrier to entry (due to knowledge requried) is high, multiple table inheritance ought to allow far more agile development of database schemata than standard relational models allow, and at the same time do full intelligent database stuff.
  - wpietri 14y ago
    Again, I'm not seeing the problem. 100 million records may take a lot of time to update, but so what? It's not like you make major schema changes to the same data every day. If you write your code to read either format v1 or v2 and to output v2, the migration can begin gradually. When you're sure it's working well, you start a low-priority job that reads old records and migrates them. Eventually, all data is in v2, so you can ditch the reader code for v1.
    - einhverfr 14y ago
      When you're sure it's working well, Translation: We cut out the time on managing our schema and sped up our development, and this meant we got to spend it all back and more managing schemaless storage!
      - wpietri 14y ago
        If you don't like NoSQL stores, don't use 'em. Either toolset is fine with an agile approach, though.

wpietri 14y ago

I don't get why you think NoSQL solutions are exempt from refactoring and maintaining data quality. I've happily done that with non-SQL stores. No matter what development approach you use, you will eventually have to deal with change. No interesting application hits v 1.0 and sticks there. The only question is whether you embrace change, which is the Agile approach, or whether you resist it. If you resist it, you change in big lurches when circumstances force it. Having tried both, I favor the former regardless of what storage mechanism I'm using.

einhverfr 14y ago
Refactoring your data structures is different though, is it not? In an RDBMS, you have to migrate your data when the schema changes, correct? If you have, say, 100 million records in Mongo DB and you decide you need to refactor your data schemas, how long will it take you to do that? The problem is that an RDBMS is very rigid typically on data input but very flexible on data output. if you change data input you MUST migrate data with it. MongoDB and others are very flexible with data input but what you get out is more or less what you have told it to store. This is a very different problem. You lose the ability to continue to refactor your data structure semantics beyond a certain point. This is not really the case with an RDBMS. BTW, I have started to get PostgreSQL object-relational modelling and it is amazing. I suspect that although the barrier to entry (due to knowledge requried) is high, multiple table inheritance ought to allow far more agile development of database schemata than standard relational models allow, and at the same time do full intelligent database stuff.
- wpietri 14y ago
  Again, I'm not seeing the problem. 100 million records may take a lot of time to update, but so what? It's not like you make major schema changes to the same data every day. If you write your code to read either format v1 or v2 and to output v2, the migration can begin gradually. When you're sure it's working well, you start a low-priority job that reads old records and migrates them. Eventually, all data is in v2, so you can ditch the reader code for v1.
  - einhverfr 14y ago
    When you're sure it's working well, Translation: We cut out the time on managing our schema and sped up our development, and this meant we got to spend it all back and more managing schemaless storage!
    - wpietri 14y ago
      If you don't like NoSQL stores, don't use 'em. Either toolset is fine with an agile approach, though.