4 ms·

If your database engine is the bottleneck, you don't have an enterprise architecture. This will sound incredibly mean, but I built a very lucrative career comi

If your database engine is the bottleneck, you don't have an enterprise architecture. This will sound incredibly mean, but I built a very lucrative career coming in behind people who tried to fix everything in their apps and not the DB.

btilly 16y ago
If your database engine is the bottleneck, you don't have an enterprise architecture. What's the busiest website that you have experience with? Do you have any experience with scaling to, say, serve more than a million dynamic pages per hour to consumers? I ask because your opinion seems extremely uninformed when it comes to high performance websites. If you've designed your application right in a traditional CRUD application, everything except the database layer scales fairly easily. The game then is about finding ways to help that remaining bottleneck. Standard techniques include optimizing queries, caching data intelligently, directing queries to read-only slaves, and in extreme cases sharding data across databases. Several of these result in moving more logic into the app, not less.
- jacques_chester 16y ago
  I think the fundamental mismatch is not between OO and relational, or the application layer and the relational layer, but between reads and writes. In the database world this conflict has been studied for decades under the headings of OLAP and OLTP. All you get taught in a university course or a beginner's book is OLTP-focused. Great if you're storing thousands of credit card transactions per second. Less useful if you need to read thousands of records per second. Lots of NoSQL looks like it's actually dimensional modelling minus some of the really clever bits.
  - chopsueyar 16y ago
    Well, with the ORMs the objects generate the relational stuff. I enjoy modeling my entities on paper as an ERD (from the entity relationship perspective) and then writing the object code. However, I have found it is sometimes necessary to denormalize my schema and store entities based on how they are presented, not necessarily the idealistic OO ORM'ed way. At the end of the day, the user doesn't care if your code is abstracted properly (who's definition?) or enough. It just needs to work speedily and correctly and be easily maintainable. I think alot of people know SQL, but may not understand how to model entities correctly, and if they can't model the entities correctly, what ever they denormalize is still coming from a 'less-good' model.
    - jacques_chester 16y ago
      Personally, when it comes to denormalisation, my preference is that it be done in a semi-structured way such as dimensional modelling. And that's exactly what I was talking about: a key purpose for DM is to make the database more accessible for fast bulk queries at the cost of duplicate data. Whereas normalisation drives out duplication in order to maximise transactional performance and space efficiency. In your case, you start with the OLTP/normalised model, then devolve to what is actually a kind of OLAP/denormalised model. Write-oriented vs read-oriented. Maybe the missing link for relational databases in a web world is actually some kind of real-time ETL?
simonw 16y ago
What's your definition of an "enterprise architecture"?

btilly 16y ago

If your database engine is the bottleneck, you don't have an enterprise architecture. What's the busiest website that you have experience with? Do you have any experience with scaling to, say, serve more than a million dynamic pages per hour to consumers? I ask because your opinion seems extremely uninformed when it comes to high performance websites. If you've designed your application right in a traditional CRUD application, everything except the database layer scales fairly easily. The game then is about finding ways to help that remaining bottleneck. Standard techniques include optimizing queries, caching data intelligently, directing queries to read-only slaves, and in extreme cases sharding data across databases. Several of these result in moving more logic into the app, not less.

jacques_chester 16y ago
I think the fundamental mismatch is not between OO and relational, or the application layer and the relational layer, but between reads and writes. In the database world this conflict has been studied for decades under the headings of OLAP and OLTP. All you get taught in a university course or a beginner's book is OLTP-focused. Great if you're storing thousands of credit card transactions per second. Less useful if you need to read thousands of records per second. Lots of NoSQL looks like it's actually dimensional modelling minus some of the really clever bits.
- chopsueyar 16y ago
  Well, with the ORMs the objects generate the relational stuff. I enjoy modeling my entities on paper as an ERD (from the entity relationship perspective) and then writing the object code. However, I have found it is sometimes necessary to denormalize my schema and store entities based on how they are presented, not necessarily the idealistic OO ORM'ed way. At the end of the day, the user doesn't care if your code is abstracted properly (who's definition?) or enough. It just needs to work speedily and correctly and be easily maintainable. I think alot of people know SQL, but may not understand how to model entities correctly, and if they can't model the entities correctly, what ever they denormalize is still coming from a 'less-good' model.
  - jacques_chester 16y ago
    Personally, when it comes to denormalisation, my preference is that it be done in a semi-structured way such as dimensional modelling. And that's exactly what I was talking about: a key purpose for DM is to make the database more accessible for fast bulk queries at the cost of duplicate data. Whereas normalisation drives out duplication in order to maximise transactional performance and space efficiency. In your case, you start with the OLTP/normalised model, then devolve to what is actually a kind of OLAP/denormalised model. Write-oriented vs read-oriented. Maybe the missing link for relational databases in a web world is actually some kind of real-time ETL?

simonw 16y ago

What's your definition of an "enterprise architecture"?