4 ms·

The problem with your argument is that all problem will look like it will have polymorphic and ambigous data until you have mapped it out properly. What the pos

by yxhuvud 14y ago

The problem with your argument is that all problem will look like it will have polymorphic and ambigous data until you have mapped it out properly. What the post you replied to was basically an advice for new projects and developers to fully map out the problem and see where things are going. It is a lot easier to start strict and organized and back down to less strictness with looser rules where you need it, than to do the opposite.

stickfigure 14y ago
Perhaps for "new developers". If the question is "as a neophyte developer, which should I learn first, Postgres or MongoDB?" then I would tend to lean towards SQL as well. But that isn't the proposal. The problem is the old cliche that if all you have is a hammer, everything looks like a nail. "Just start with a hammer" is bad advice because with enough effort and willingness to cause collateral damage, you can almost always get a screw in with a hammer. If you've never seen a screwdriver, you're doomed to posting HN stories about "how I spent the last three weeks rebuilding my application architecture to deal with the fact that hammered-in screws tend to fall out a lot."
- flatline3 14y ago
  You failed to address the parents point entirely: it's a lot easier to start strict and loosen restrictions later, than it is to start loose and add restrictions.
  - stickfigure 14y ago
    I'm not sure what this has to do with SQL vs NoSQL. There are myriad differences between storage systems which have little to do with referential integrity constraints or data validation. Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation. Rapidly changing schemas are a PITA in RDBMSes, especially with a large cluster of servers. It's usually very easy to alter schemas on-the-fly with NoSQL systems. This has nothing to do with data validation. On a related point, the database is a crappy place to put data validation anyways. It can only do the most primitive forms of validation (is this an integer? is the FK reference valid?). Application level-validation gives you the full power of a modern programming language and all the data you could possibly want to fetch. Yeah, "senior discount" is a float... but it should only be given to people over age 65. The fact that the OP managed to insert "111" into a column instead of 111 means he is using crappy tools. My datastore api guarantees that integer fields in my (statically typed) classes are written as integers. I never have this problem, nor have I ever taken down my GAE (NoSQL) applications for maintenance.
    - einhverfr 14y ago
      The basic tradeoff is between flexible input and flexible output. I do think it is sound advice to push the relational database model (this overlaps but is not identical to the relational algebra model) to the breaking point before going elsewhere. If you don't do data validation you can't do ad hoc data transformation later reliably. Strict schemas are always an investment in the future for that reason. > Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation. JSON and XML types in PostgreSQL rock. If you can limit the areas where the data is polymorphic, you can push the rest into those types. Many other RDBMS's support XML types as well. > Rapidly changing schemas are a PITA in RDBMSes, especially with a large cluster of servers. It's usually very easy to alter schemas on-the-fly with NoSQL systems. This has nothing to do with data validation. This again trades one set of problems for another. You are allowing old schema and new schema to co-exist. How many old schemas does your application have to support? It seems to me this would likely ossify things down the road a bit because if your schema is that rapidly changing, you may never get caught up and so backward compatibility is going to be a bitch. In other words, the less guarantees you can make regarding information stored, the fewer guarantees you can make about data output. That's a big deal as your product matures. And don't get me started on BASE.....
      - stickfigure 14y ago
        JSON and XML types give you all the pain of a NoSQL system with none of the benefits of an RDBMS. No thanks. My tools elegantly map database fields to my objects; doing this with serialization formats is incredibly crude and brittle. I do it for some edge cases already; I am looking forward to removing that code. A schemaless datastore does not mean a schemaless app. Furthermore, the application layer is a far better place to do validation simply because it has more knowledge of the real world. An RDBMS can constraint an age field to be integer; the application can constrain it to be positive and within the probable lifespan of a human. How many old schemas does your application have to support? Only as many as I want to; reprocessing removes the old, which I tend to do fairly quickly. I have done a lot of schema migration over the last three years on GAE and the process is vastly easier than it is in RDBMS-land. There's no need to stop the world. Hey, I love Postgres. I spent most of my (almost two-decade-long) career building apps on many different RDBMSes. But there are many applications that are better suited to alternatives, especially with large-scale consumer-facing web apps. These threads annoy me; we should all be looking at new tools saying "wow, I wonder what this is good for" rather than smugly burning all the heretics that turn away from to the One True Idol.

by yxhuvud 14y ago

stickfigure 14y ago

Perhaps for "new developers". If the question is "as a neophyte developer, which should I learn first, Postgres or MongoDB?" then I would tend to lean towards SQL as well. But that isn't the proposal. The problem is the old cliche that if all you have is a hammer, everything looks like a nail. "Just start with a hammer" is bad advice because with enough effort and willingness to cause collateral damage, you can almost always get a screw in with a hammer. If you've never seen a screwdriver, you're doomed to posting HN stories about "how I spent the last three weeks rebuilding my application architecture to deal with the fact that hammered-in screws tend to fall out a lot."

flatline3 14y ago
You failed to address the parents point entirely: it's a lot easier to start strict and loosen restrictions later, than it is to start loose and add restrictions.
- stickfigure 14y ago
  I'm not sure what this has to do with SQL vs NoSQL. There are myriad differences between storage systems which have little to do with referential integrity constraints or data validation. Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation. Rapidly changing schemas are a PITA in RDBMSes, especially with a large cluster of servers. It's usually very easy to alter schemas on-the-fly with NoSQL systems. This has nothing to do with data validation. On a related point, the database is a crappy place to put data validation anyways. It can only do the most primitive forms of validation (is this an integer? is the FK reference valid?). Application level-validation gives you the full power of a modern programming language and all the data you could possibly want to fetch. Yeah, "senior discount" is a float... but it should only be given to people over age 65. The fact that the OP managed to insert "111" into a column instead of 111 means he is using crappy tools. My datastore api guarantees that integer fields in my (statically typed) classes are written as integers. I never have this problem, nor have I ever taken down my GAE (NoSQL) applications for maintenance.
  - einhverfr 14y ago
    The basic tradeoff is between flexible input and flexible output. I do think it is sound advice to push the relational database model (this overlaps but is not identical to the relational algebra model) to the breaking point before going elsewhere. If you don't do data validation you can't do ad hoc data transformation later reliably. Strict schemas are always an investment in the future for that reason. > Polymorphic structures are a PITA to model in relational tables, but they fit very well in schemaless, document-oriented NoSQL systems. This has nothing to do with data validation. JSON and XML types in PostgreSQL rock. If you can limit the areas where the data is polymorphic, you can push the rest into those types. Many other RDBMS's support XML types as well. > Rapidly changing schemas are a PITA in RDBMSes, especially with a large cluster of servers. It's usually very easy to alter schemas on-the-fly with NoSQL systems. This has nothing to do with data validation. This again trades one set of problems for another. You are allowing old schema and new schema to co-exist. How many old schemas does your application have to support? It seems to me this would likely ossify things down the road a bit because if your schema is that rapidly changing, you may never get caught up and so backward compatibility is going to be a bitch. In other words, the less guarantees you can make regarding information stored, the fewer guarantees you can make about data output. That's a big deal as your product matures. And don't get me started on BASE.....
    - stickfigure 14y ago
      JSON and XML types give you all the pain of a NoSQL system with none of the benefits of an RDBMS. No thanks. My tools elegantly map database fields to my objects; doing this with serialization formats is incredibly crude and brittle. I do it for some edge cases already; I am looking forward to removing that code. A schemaless datastore does not mean a schemaless app. Furthermore, the application layer is a far better place to do validation simply because it has more knowledge of the real world. An RDBMS can constraint an age field to be integer; the application can constrain it to be positive and within the probable lifespan of a human. How many old schemas does your application have to support? Only as many as I want to; reprocessing removes the old, which I tend to do fairly quickly. I have done a lot of schema migration over the last three years on GAE and the process is vastly easier than it is in RDBMS-land. There's no need to stop the world. Hey, I love Postgres. I spent most of my (almost two-decade-long) career building apps on many different RDBMSes. But there are many applications that are better suited to alternatives, especially with large-scale consumer-facing web apps. These threads annoy me; we should all be looking at new tools saying "wow, I wonder what this is good for" rather than smugly burning all the heretics that turn away from to the One True Idol.