5 ms·

Never seen a large enough project that relies on an ORM be anything other than a giant mess. I mean never. The conclusion is correct. From an application perspe

Never seen a large enough project that relies on an ORM be anything other than a giant mess. I mean never. The conclusion is correct. From an application perspective the db is just another API and should be treated that way and the ORM should just be thought of as a convenient DSL for creating queries on top of that API.

kennell 10y ago
What do you define as "large enough"? There have been a ton of fairly big Ruby on Rails apps written over the last few years, the majority of them using ActiveRecord to handle everything.
- dkarapetyan 10y ago
  Really? How many of those backends have you seen and worked on? Heroic effort does not qualify as success in my book. You can maintain any nightmare by just throwing enough bodies at it. What counts is the maintenance burden and the cost of change.
rhizome 10y ago
It strikes me as a thick/thin client tug of war. You can supplant ORMs by e.g. moving your logic into stored procedures on the db side, until you find an inconvenience there, then back and forth until the end of time. As with many dyadic architectural choices, there are good arguments on both sides.
- dkarapetyan 10y ago
  I agree. The problem is that striking that balance is hard. If you go all-in with an ORM then trying to backpedal and figure out at which point you went wrong becomes almost impossible and so now you have a tangled web of ORM queries and no clear path to untangling things.
  - rhizome 10y ago
    Sure, but the balance is entirely context-dependent. But hey, I guess this is where all polemics fall down. That said, where is the academic problem in refactoring into the db and decomposing ORM queries into straight SQL where necessary/desired? I'm not a Fortune 500 Systems Architect, but I'm under the impression that this is a fairly straightforward process. Now, management of the systems will necessarily incur extra complexity, but that's a second order effect that should be assessed in the decision process as part of the cost estimations. Extra technical debt, for sure. So, choosing one is technically cheaper and more simple and maintainable. Choosing multiple may be faster (or, in many cases, "academically satisfying"), but more complex with a steeper learning curve. Be sure to hire the best of the best!
    - dkarapetyan 10y ago
      Not so simple. Every big enough place will be a disjointed mess of cross-cutting concerns. The ORM (usually another custom mess on top of some open source thing) ends up being owned by everyone and in effect ends up being owned by no one. So then you end up going around asking each team in turn to fix stuff but each team consists of domain experts and not ORM/SQL experts so no one does anything. So then you bring in ORM/SQL experts but they don't have any domain knowledge so they just chip away at the edges and end up making no improvements and you still are stuck with an ORM nightmare. The bottleneck in fixing technical things is never the technical stuff. After all, most of this stuff is 70s tech to begin with and the science has been worked out by much smarter people already. The social stuff on the other hand is a different matter entirely and once you're in an ORM quagmire it is impossible to get out.
nostrademons 10y ago
To be fair, I've never seen a large project be anything other than a giant mess. This is the nature of large projects. If it's not a giant mess, you are probably leaving user happiness (= $$$) on the table.
- dkarapetyan 10y ago
  And it comes at the expense of craftsmanship and programmer happiness. Lately I've been thinking how much initial architecture affects the mental well-being of all subsequent programmers that join the project. This is one of the reasons there is such a high turn-over and burnout rate in programming. If you're not consistently paying back technical debt and improving the architecture of the project then you're paying for it in other ways with the revolving door of programmers. Then again if you're a big enough business it probably doesn't matter as long as customers are paying you. All you have to do is plug-in another cog/programmer into the machine when the previous one wears out.
  - nostrademons 10y ago
    All this is true, but it's also the reason why programmers are paid as much as they are. In general, in a market economy, you are paid to do things that other people don't want to do. The whole point of a market is to create an incentive to do things that are unpleasant. Beautiful code does exist in the software world. It's usually found in hobby, open-source, and research projects without profit motives, userbases, deadlines, and all those other complications that result in shitty hacks. In the meantime, you could look at the crap that is most large-scale codebases as a barrier to entry that keeps demand for programmers high. Enjoy the money you get for cleaning up other peoples' messes, and then use that to buy time to make the beautiful, useless stuff.
lmm 10y ago
I've worked on a 400kloc Java system that was beautiful in the half that used Hibernate, and horrific in the half that didn't.
- dkarapetyan 10y ago
  400kloc Java. That's cute. Not saying that's not impressive. The scales I've seen you're off by a few orders of magnitude.
  - lmm 10y ago
    Oh sure, it's not the largest system I've worked on by any means, it just seemed relevant because at the time I was working on it it was almost exactly a 50/50 split between the hibernate parts and the not. And honestly I find beyond a certain size the techniques you have to use are pretty similar - there's very little practical difference between working on a 400kloc and a 4mloc project.

kennell 10y ago

What do you define as "large enough"? There have been a ton of fairly big Ruby on Rails apps written over the last few years, the majority of them using ActiveRecord to handle everything.

dkarapetyan 10y ago
Really? How many of those backends have you seen and worked on? Heroic effort does not qualify as success in my book. You can maintain any nightmare by just throwing enough bodies at it. What counts is the maintenance burden and the cost of change.

rhizome 10y ago

It strikes me as a thick/thin client tug of war. You can supplant ORMs by e.g. moving your logic into stored procedures on the db side, until you find an inconvenience there, then back and forth until the end of time. As with many dyadic architectural choices, there are good arguments on both sides.

dkarapetyan 10y ago
I agree. The problem is that striking that balance is hard. If you go all-in with an ORM then trying to backpedal and figure out at which point you went wrong becomes almost impossible and so now you have a tangled web of ORM queries and no clear path to untangling things.
- rhizome 10y ago
  Sure, but the balance is entirely context-dependent. But hey, I guess this is where all polemics fall down. That said, where is the academic problem in refactoring into the db and decomposing ORM queries into straight SQL where necessary/desired? I'm not a Fortune 500 Systems Architect, but I'm under the impression that this is a fairly straightforward process. Now, management of the systems will necessarily incur extra complexity, but that's a second order effect that should be assessed in the decision process as part of the cost estimations. Extra technical debt, for sure. So, choosing one is technically cheaper and more simple and maintainable. Choosing multiple may be faster (or, in many cases, "academically satisfying"), but more complex with a steeper learning curve. Be sure to hire the best of the best!
  - dkarapetyan 10y ago
    Not so simple. Every big enough place will be a disjointed mess of cross-cutting concerns. The ORM (usually another custom mess on top of some open source thing) ends up being owned by everyone and in effect ends up being owned by no one. So then you end up going around asking each team in turn to fix stuff but each team consists of domain experts and not ORM/SQL experts so no one does anything. So then you bring in ORM/SQL experts but they don't have any domain knowledge so they just chip away at the edges and end up making no improvements and you still are stuck with an ORM nightmare. The bottleneck in fixing technical things is never the technical stuff. After all, most of this stuff is 70s tech to begin with and the science has been worked out by much smarter people already. The social stuff on the other hand is a different matter entirely and once you're in an ORM quagmire it is impossible to get out.

nostrademons 10y ago

To be fair, I've never seen a large project be anything other than a giant mess. This is the nature of large projects. If it's not a giant mess, you are probably leaving user happiness (= $$$) on the table.

dkarapetyan 10y ago
And it comes at the expense of craftsmanship and programmer happiness. Lately I've been thinking how much initial architecture affects the mental well-being of all subsequent programmers that join the project. This is one of the reasons there is such a high turn-over and burnout rate in programming. If you're not consistently paying back technical debt and improving the architecture of the project then you're paying for it in other ways with the revolving door of programmers. Then again if you're a big enough business it probably doesn't matter as long as customers are paying you. All you have to do is plug-in another cog/programmer into the machine when the previous one wears out.
- nostrademons 10y ago
  All this is true, but it's also the reason why programmers are paid as much as they are. In general, in a market economy, you are paid to do things that other people don't want to do. The whole point of a market is to create an incentive to do things that are unpleasant. Beautiful code does exist in the software world. It's usually found in hobby, open-source, and research projects without profit motives, userbases, deadlines, and all those other complications that result in shitty hacks. In the meantime, you could look at the crap that is most large-scale codebases as a barrier to entry that keeps demand for programmers high. Enjoy the money you get for cleaning up other peoples' messes, and then use that to buy time to make the beautiful, useless stuff.

lmm 10y ago

I've worked on a 400kloc Java system that was beautiful in the half that used Hibernate, and horrific in the half that didn't.

dkarapetyan 10y ago
400kloc Java. That's cute. Not saying that's not impressive. The scales I've seen you're off by a few orders of magnitude.
- lmm 10y ago
  Oh sure, it's not the largest system I've worked on by any means, it just seemed relevant because at the time I was working on it it was almost exactly a 50/50 split between the hibernate parts and the not. And honestly I find beyond a certain size the techniques you have to use are pretty similar - there's very little practical difference between working on a 400kloc and a 4mloc project.