4 ms·

>> "...Had they not bothered introducing LINQ in the first place, they'd have had their 100x performance boost from the get go..." Except that when StackOverfl

by rayvega 15y ago

>> "...Had they not bothered introducing LINQ in the first place, they'd have had their 100x performance boost from the get go..." Except that when StackOverflow started it was a small startup with just three developers and a small user base. They probably made the decision early on to tradeoff any premature performance optimization with the ability to quickly iterate and ship the product by using Linq-2-Sql instead of hand-writing every single CRUD operation. Now that their business (i.e. web traffic) has grown exponentially and have a larger group of developers they can now go back and optimize and performance tune the code including doing complete re-writes as necessary. This is an important takeaway for anyone building a startup.

eftpotrm 15y ago
But.... every single CRUD is normally not a huge amount of code and relatively simple to generate from the information schema if you know a bit of SQL Server. Do it that way and you've got your sprocs just sitting there for later use, primarily OK to be used straight out of the box but available for modification as soon as you need something different.
- MartinCron 15y ago
  You say that like having CRUD sprocs sitting around is a good thing.
  - noblethrasher 15y ago
    Just curious, why isn't it?
    - MartinCron 15y ago
      All code, auto-generated or not, is a sort of liability in that you have to maintain it. You have to re-generate it each time you make changes, and there's a possibility that someone can mess with it and cause it to break. My take on auto-generated code is this: If code is so simple/deterministic/boilerplate that it can be auto-generated, then the generation of that code should be completely automated (at runtime, if possible) and the generated code should be invisible and untouchable. I've never had a pleasant experience tweaking auto-generated code.
      - eftpotrm 15y ago
        But, if the code is being generated in that way, you're also doing a lot of identical string concatenation at runtime, when identical results with higher performance can be achieved by doing it at an earlier point - compile, design, whatever. You're burning CPU cycles to avoid the existence of these objects and separetly make it harder to optimise your application later when you find you need it in places. Depending on your application architecture, you're also increasing the chances of your needing multiple round-trips to the database to query various bits of information for building your query, increasing the load on the DBMS (and possibly the network, depending on your architecture) and increasing the query latency. Properly auto-generated code should be the cleanest, best laid out, most consistent code you work on, quite possibly well commented as well. It's obviously not a panacaea that'll work for everything and I'm not saying ORM tools never have their benefits, but certainly I'm not convinced that third-party ORM tools are right for what I'm doing at present.
        MartinCron 15y ago
        Sure, ORM tools can be a tradeoff between what's fast at dev time and what's fast at runtime. What has been working for me is relying on the ORM and then hand-tuning the small subset of things that need to be hand-tuned. And, of course, I don't know anything about what you're doing at present, so I won't even try to speak to that. But I can't imagine a weaker argument against ORM than the first one you made, though. When you're doing data-intensive work, are you really worried about the CPU overhead of string concatenation on your application layer? :)
        eftpotrm 15y ago
        For a decent subset of the procedures I've worked on, yes, I am, from experience. I've had app performance killed by excessive string concatenation before, so my inclination to repeat identical jobs at runtime which I know from experience cause performance issues is limited. Precompile. I should stress in this though, I'm not a complete 'it must be a compiled sproc stored in the DBMS' guy; I have worked with systems before which effectively provided their own stored procedure implementation within the application source code because the underlying DMBS didn't support anything else; it's not ideal for a number of reasons but it does work, is an order of magnitude superior to ad-hoc SQL scattered throughout the app, still permits some of the benefits of genuine sprocs but also still provides the dev with proper control over what they're doing.
    - gnaritas 15y ago
      Your generated procs's are your home-brew ORM, what exactly do you think an ORM does other than dynamically generate CRUD and map data into objects to speed development? The ORM is better, generates it's code at runtime so it doesn't leave a mess behind when things change and removes any need for you to regenerate. That ORM can generate update statements that only update changed fields... can your proc do that or does it blindly update all fields? That ORM effortlessly maps data from the db into objects so you don't have to pass record-sets around; I don't want record-sets and data readers littered all over my programs, I want business objects. Programs that keep all SQL in procs are horrible to work on and a bear to maintain, I'll take an ORM any day over a mess like that.
Stormbringer 15y ago
They cover the "why Linq?" question in one of their podcasts... and it basically boils down to if you're drinking the Microsoft cool-aid anyway, might as well drink the whole cup. I think there was an implicit assumption that everything Microsoft does is great, and Microsoft would never lie to you. Oh, and the next version will be even betterer! Honest! Basically the fix boils down to replacing a slow ORM with a faster ORM, and using indexes. Well of course indexes are faster - Duh! HOWEVER this is where if I was them, I would start to get worried, because every SQL Server horror story I've been told basically starts with "And then the indexes became randomly corrupted" and goes rapidly downhill from there.

by rayvega 15y ago

eftpotrm 15y ago

But.... every single CRUD is normally not a huge amount of code and relatively simple to generate from the information schema if you know a bit of SQL Server. Do it that way and you've got your sprocs just sitting there for later use, primarily OK to be used straight out of the box but available for modification as soon as you need something different.

MartinCron 15y ago
You say that like having CRUD sprocs sitting around is a good thing.
- noblethrasher 15y ago
  Just curious, why isn't it?
  - MartinCron 15y ago
    All code, auto-generated or not, is a sort of liability in that you have to maintain it. You have to re-generate it each time you make changes, and there's a possibility that someone can mess with it and cause it to break. My take on auto-generated code is this: If code is so simple/deterministic/boilerplate that it can be auto-generated, then the generation of that code should be completely automated (at runtime, if possible) and the generated code should be invisible and untouchable. I've never had a pleasant experience tweaking auto-generated code.
    - eftpotrm 15y ago
      But, if the code is being generated in that way, you're also doing a lot of identical string concatenation at runtime, when identical results with higher performance can be achieved by doing it at an earlier point - compile, design, whatever. You're burning CPU cycles to avoid the existence of these objects and separetly make it harder to optimise your application later when you find you need it in places. Depending on your application architecture, you're also increasing the chances of your needing multiple round-trips to the database to query various bits of information for building your query, increasing the load on the DBMS (and possibly the network, depending on your architecture) and increasing the query latency. Properly auto-generated code should be the cleanest, best laid out, most consistent code you work on, quite possibly well commented as well. It's obviously not a panacaea that'll work for everything and I'm not saying ORM tools never have their benefits, but certainly I'm not convinced that third-party ORM tools are right for what I'm doing at present.
      - MartinCron 15y ago
        Sure, ORM tools can be a tradeoff between what's fast at dev time and what's fast at runtime. What has been working for me is relying on the ORM and then hand-tuning the small subset of things that need to be hand-tuned. And, of course, I don't know anything about what you're doing at present, so I won't even try to speak to that. But I can't imagine a weaker argument against ORM than the first one you made, though. When you're doing data-intensive work, are you really worried about the CPU overhead of string concatenation on your application layer? :)
        eftpotrm 15y ago
        For a decent subset of the procedures I've worked on, yes, I am, from experience. I've had app performance killed by excessive string concatenation before, so my inclination to repeat identical jobs at runtime which I know from experience cause performance issues is limited. Precompile. I should stress in this though, I'm not a complete 'it must be a compiled sproc stored in the DBMS' guy; I have worked with systems before which effectively provided their own stored procedure implementation within the application source code because the underlying DMBS didn't support anything else; it's not ideal for a number of reasons but it does work, is an order of magnitude superior to ad-hoc SQL scattered throughout the app, still permits some of the benefits of genuine sprocs but also still provides the dev with proper control over what they're doing.
  - gnaritas 15y ago
    Your generated procs's are your home-brew ORM, what exactly do you think an ORM does other than dynamically generate CRUD and map data into objects to speed development? The ORM is better, generates it's code at runtime so it doesn't leave a mess behind when things change and removes any need for you to regenerate. That ORM can generate update statements that only update changed fields... can your proc do that or does it blindly update all fields? That ORM effortlessly maps data from the db into objects so you don't have to pass record-sets around; I don't want record-sets and data readers littered all over my programs, I want business objects. Programs that keep all SQL in procs are horrible to work on and a bear to maintain, I'll take an ORM any day over a mess like that.

Stormbringer 15y ago

They cover the "why Linq?" question in one of their podcasts... and it basically boils down to if you're drinking the Microsoft cool-aid anyway, might as well drink the whole cup. I think there was an implicit assumption that everything Microsoft does is great, and Microsoft would never lie to you. Oh, and the next version will be even betterer! Honest! Basically the fix boils down to replacing a slow ORM with a faster ORM, and using indexes. Well of course indexes are faster - Duh! HOWEVER this is where if I was them, I would start to get worried, because every SQL Server horror story I've been told basically starts with "And then the indexes became randomly corrupted" and goes rapidly downhill from there.