3 ms·

Sure, but the cost of schema maintenance is grossly overstated by those who simply aren't familiar with the tooling to do so. Furthermore, the cost of dealing

Sure, but the cost of schema maintenance is grossly overstated by those who simply aren't familiar with the tooling to do so. Furthermore, the cost of dealing with the corrupt data down the line will absorb all the gains you had in a matter of months, unless of course you've reinvented strong schema control at the application layer. A lot of inexperienced devs look at NoSQL as being simpler than SQL because the dev documentation is shorter and has a more stylish theme. They then assume that all the touted benefits of SQL are YAGNI and they'll cross that bridge when they come to it. The mistake they make is assuming that the benefits of SQL databases are something that only is needed for a mature application. To the contrary, SQL gives an incredible hedge against the uncertainty of change in the early days of an app when churn is high. A normalized data structure allows much more query flexibility; you should only denormalize for performance once you realize what the workload and logic will look like for the long haul. Of course if you have a known workload that you want to scale up huge then that's a legitimate use case for NoSQL, but the purpose is not flexibility.

hackits 10y ago
Our current manager wants to transition over to NoSQL, as he doesn't see any value having to constantly update JPA Repositories and dependencies when a column is added or removed from the database table. I recommended that JPA Repositories shouldn't be stored at the lowest level of Maven jar package's eg... we have a very deep nested Maven pom dependencies (about 6 layers deep) of which the JPA repositories are defined in the lowest of lowest levels. Granted I gave references to research papers from Google that JPA Repositories and Hibernate ORM should be at the project layer (due to their nature of changing rapidly). Turns out, still going with the NoSQL solution. Although it seems like everyone have convinced themselves that NoSQL is faster development processes than a simple table with constraints.
- hinkley 10y ago
  Leaf node dependencies should be very stable. Otherwise you spend all your time rebuilding or at least rerunning integration tests. I work at a company that has achieved the opposite of that with not one but two libraries, and deployment is a miserable experience.
- path411 10y ago
  In the projects I've worked on that require frequent table modification, I came to the opinion that the correct approach would be to store the data where the columns/values are stored in a table or set of tables based on types and linked together with a relationship to the original table as the parent. The big downside I could see is performance, but maybe I'm falsely assuming that this would still be faster than NoSQL? (I haven't actually used nosql however.) But, for my projects, the performance hit should/would not be a problem. Since my experience is probably just limited, I'm curious if there is an easy to explain example where this is incorrect? (Or I guess maybe most people disagree with me and see my above opinion as always/most of the time incorrect)
  - nickpeterson 10y ago
    This is correct if I understand you. Basically, when people add something new to a schema, they're generally tacitly also admitting it's optional. Optional fields should not exist on the same table as non-optional fields, because then you have to allow everything to be null or a specially designated value.
  - hackits 10y ago
    I will agree with your approach. With any large system or government database schema adding another table and then adding a relation to the necessary reports is 100x better than changing the parent table. I've seen too often jobs being quoted for clients for simply adding a column to table blow out to 4-5 month development tasks. Turns out the table was being used by an old C/C++ program with oracle C bindings. That had to be updated, and that in turn caused another application to stop working.
parasubvert 10y ago
"Sure, but the cost of schema maintenance is grossly overstated by those who simply aren't familiar with the tooling to do so." Scale ruins everything. Most SQL databases are pretty terrible at schema maintenance at scale (i.e. hundreds of MySQL shards). This (among other ways that scale ruins the RDBMS) is what led to NoSQL in the first place at the largest of web properties. Sure, joe average startup probably doesn't have the same problems. But that's marketing.... The relational model and SQL have been beaten up for decades by those who prefer to remain closer to the metal.
- Jweb_Guru 9y ago
  I highly doubt your "close to the metal" solutions outperform state-of-the-art relational databases like EmptyHeaded. The truth is that people prefer to reinvent their own inferior solutions rather than looking for best-in-class stuff.

hackits 10y ago

Our current manager wants to transition over to NoSQL, as he doesn't see any value having to constantly update JPA Repositories and dependencies when a column is added or removed from the database table. I recommended that JPA Repositories shouldn't be stored at the lowest level of Maven jar package's eg... we have a very deep nested Maven pom dependencies (about 6 layers deep) of which the JPA repositories are defined in the lowest of lowest levels. Granted I gave references to research papers from Google that JPA Repositories and Hibernate ORM should be at the project layer (due to their nature of changing rapidly). Turns out, still going with the NoSQL solution. Although it seems like everyone have convinced themselves that NoSQL is faster development processes than a simple table with constraints.

hinkley 10y ago
Leaf node dependencies should be very stable. Otherwise you spend all your time rebuilding or at least rerunning integration tests. I work at a company that has achieved the opposite of that with not one but two libraries, and deployment is a miserable experience.
path411 10y ago
In the projects I've worked on that require frequent table modification, I came to the opinion that the correct approach would be to store the data where the columns/values are stored in a table or set of tables based on types and linked together with a relationship to the original table as the parent. The big downside I could see is performance, but maybe I'm falsely assuming that this would still be faster than NoSQL? (I haven't actually used nosql however.) But, for my projects, the performance hit should/would not be a problem. Since my experience is probably just limited, I'm curious if there is an easy to explain example where this is incorrect? (Or I guess maybe most people disagree with me and see my above opinion as always/most of the time incorrect)
- nickpeterson 10y ago
  This is correct if I understand you. Basically, when people add something new to a schema, they're generally tacitly also admitting it's optional. Optional fields should not exist on the same table as non-optional fields, because then you have to allow everything to be null or a specially designated value.
- hackits 10y ago
  I will agree with your approach. With any large system or government database schema adding another table and then adding a relation to the necessary reports is 100x better than changing the parent table. I've seen too often jobs being quoted for clients for simply adding a column to table blow out to 4-5 month development tasks. Turns out the table was being used by an old C/C++ program with oracle C bindings. That had to be updated, and that in turn caused another application to stop working.

parasubvert 10y ago

"Sure, but the cost of schema maintenance is grossly overstated by those who simply aren't familiar with the tooling to do so." Scale ruins everything. Most SQL databases are pretty terrible at schema maintenance at scale (i.e. hundreds of MySQL shards). This (among other ways that scale ruins the RDBMS) is what led to NoSQL in the first place at the largest of web properties. Sure, joe average startup probably doesn't have the same problems. But that's marketing.... The relational model and SQL have been beaten up for decades by those who prefer to remain closer to the metal.

Jweb_Guru 9y ago
I highly doubt your "close to the metal" solutions outperform state-of-the-art relational databases like EmptyHeaded. The truth is that people prefer to reinvent their own inferior solutions rather than looking for best-in-class stuff.