3 ms·

Wrapping both caching logic and database access in an ORM like system is no doubt the right thing to do. Letting front end developers write queries to be conver

by n_are_q 15y ago

Wrapping both caching logic and database access in an ORM like system is no doubt the right thing to do. Letting front end developers write queries to be converted by an orm and reviewed by a DBA later - in my opinion that's not the most efficient method of development. I probably would have invested in an extra DB person or two to help write the data access logic. But hey, I can't argue with results - if it worked for you that's great. But as a general statement I think that sort development methodology is highly conducive to errors and systematic problems that would not become evident until later, and at that point take a great deal of effort to fix.

nettdata 15y ago
The two big systems I architected where I made the decision to go with ORM's were the online EA Sports system (all EA Sports games on all platforms, currently running in a 7 node Oracle cluster), and most recently, the Need For Speed World Online system. We launched the EA Sports system with Madden, and went from 50 to 11 million users hitting the DB in less than an hour. Then we rolled out the other EA Sports games. Needless to say, both systems were slightly bigger than a simple blogging site. In both cases, we had a large number of smart developers who we empowered with the use of an ORM; they understood the domain model, and they didn't have to worry about waiting for a "DB type" to write stored procedures, or develop a data model, etc. As a matter of fact, in both cases, I was the only DBA on the project, and it was a predominately part-time role. We'd meet, ensure we were all on the same page with the object/data model, and then they'd go and build it. The developers were able to immediately build and run and test and integrate something that was functional and operational, when they needed it. This was HUGE, and something that most people don't properly appreciate. Timelines were already insane enough as it was, the last thing we needed to do was artificially constrain ourselves by waiting for other (db) devs before work could go on. Especially when requirements had the potential to change from one day to the next. In both situations, we took advantage of very, very sophisticated testing procedures that would happen nightly, both functional and stress/load, and it pointed us at the bottlenecks of each nightly build that would require tuning and investigation. We intentionally set up our testing to be able to monitor and test the effectiveness of the ORM, and to point it out when it didn't work efficiently. The devs would do the majority of the heavy lifting with the initial data model, and the results would be tested, reviewed, and then modified if required. The performance modifications were not a lot of effort to fix, either. Usually it was a very slight data model change, or using a named query to take advantage of a database-specific features. And CLOBS. Every database seems to handle them differently, so we had to hack some solutions. Having done large scale database development for almost 25 years, using the classic stored procedure approach and the ORM approach, I'll say again that ORM's are a great solution for certain projects with the right staff, and aren't a crutch or some lazy choice if used properly.
- spudlyo 15y ago
  Thanks for sharing your experiences. This is why I read HN.
- fendale 15y ago
  As another 'Oracle guy' this is an interesting post. I have said before on here, if you pay for Oracle, and also pay for decent storage arrays, Oracle can shift a serious amount of data before it reaches its limit. In my opinion, it really seems to be an order of magnitude better than its closest open source competitor.
  - jacques_chester 15y ago
    I have to admit that when I hear "relational doesn't scale", I usually rewrite it in my head as "MySQL doesn't scale". Because people have been running massive systems on Oracle, DB2, Sybase, TeraData etc for years now.
    - mkjones 15y ago
      I think when people say "relational doesn't scale," what they mean is "MySQL often requires application-level changes to scale out." I assume (among this crowd, anyway) that scaling out is more desirable vs. scaling up because the majority of the hardware costs are variable, whereas scaling up requires a step function of large cash investments that startups often can't afford. Do those massive systems on Oracle etc scale out, or simply scale up with expensive hardware?
      - jacques_chester 15y ago
        > Do those massive systems on Oracle etc scale out, or simply scale up with expensive hardware? Both. If you still need ACID guarantees and want hundreds of thousands (or even millions, if necessary) of TPM, you will need to pay the piper.
      - fendale 15y ago
        You can certainly scale out with Oracle RAC. At some point the bottleneck will likely be disk however, so buying a high end storage system would probably become priority.

by n_are_q 15y ago

nettdata 15y ago

The two big systems I architected where I made the decision to go with ORM's were the online EA Sports system (all EA Sports games on all platforms, currently running in a 7 node Oracle cluster), and most recently, the Need For Speed World Online system. We launched the EA Sports system with Madden, and went from 50 to 11 million users hitting the DB in less than an hour. Then we rolled out the other EA Sports games. Needless to say, both systems were slightly bigger than a simple blogging site. In both cases, we had a large number of smart developers who we empowered with the use of an ORM; they understood the domain model, and they didn't have to worry about waiting for a "DB type" to write stored procedures, or develop a data model, etc. As a matter of fact, in both cases, I was the only DBA on the project, and it was a predominately part-time role. We'd meet, ensure we were all on the same page with the object/data model, and then they'd go and build it. The developers were able to immediately build and run and test and integrate something that was functional and operational, when they needed it. This was HUGE, and something that most people don't properly appreciate. Timelines were already insane enough as it was, the last thing we needed to do was artificially constrain ourselves by waiting for other (db) devs before work could go on. Especially when requirements had the potential to change from one day to the next. In both situations, we took advantage of very, very sophisticated testing procedures that would happen nightly, both functional and stress/load, and it pointed us at the bottlenecks of each nightly build that would require tuning and investigation. We intentionally set up our testing to be able to monitor and test the effectiveness of the ORM, and to point it out when it didn't work efficiently. The devs would do the majority of the heavy lifting with the initial data model, and the results would be tested, reviewed, and then modified if required. The performance modifications were not a lot of effort to fix, either. Usually it was a very slight data model change, or using a named query to take advantage of a database-specific features. And CLOBS. Every database seems to handle them differently, so we had to hack some solutions. Having done large scale database development for almost 25 years, using the classic stored procedure approach and the ORM approach, I'll say again that ORM's are a great solution for certain projects with the right staff, and aren't a crutch or some lazy choice if used properly.

spudlyo 15y ago
Thanks for sharing your experiences. This is why I read HN.
fendale 15y ago
As another 'Oracle guy' this is an interesting post. I have said before on here, if you pay for Oracle, and also pay for decent storage arrays, Oracle can shift a serious amount of data before it reaches its limit. In my opinion, it really seems to be an order of magnitude better than its closest open source competitor.
- jacques_chester 15y ago
  I have to admit that when I hear "relational doesn't scale", I usually rewrite it in my head as "MySQL doesn't scale". Because people have been running massive systems on Oracle, DB2, Sybase, TeraData etc for years now.
  - mkjones 15y ago
    I think when people say "relational doesn't scale," what they mean is "MySQL often requires application-level changes to scale out." I assume (among this crowd, anyway) that scaling out is more desirable vs. scaling up because the majority of the hardware costs are variable, whereas scaling up requires a step function of large cash investments that startups often can't afford. Do those massive systems on Oracle etc scale out, or simply scale up with expensive hardware?
    - jacques_chester 15y ago
      > Do those massive systems on Oracle etc scale out, or simply scale up with expensive hardware? Both. If you still need ACID guarantees and want hundreds of thousands (or even millions, if necessary) of TPM, you will need to pay the piper.
    - fendale 15y ago
      You can certainly scale out with Oracle RAC. At some point the bottleneck will likely be disk however, so buying a high end storage system would probably become priority.