3 ms·

I feel like most skilled engineers would end up re-writing a crappy version of a framework in the process of writing an app--so why not start with one? I think

by notJim 14y ago

I feel like most skilled engineers would end up re-writing a crappy version of a framework in the process of writing an app--so why not start with one? I think it depends partly on whether you're really writing an app, or hacking together an MVP. The reason I feel this way is that most apps have a great deal of simple CRUD (SELECT * FROM [table] WHERE id=? and UPDATE [table] SET [...] WHERE id=?), which is exactly the problem ORMs optimize for. Any skilled engineer is going to notice they're writing the same boilerplate SQL -> data structure of some kind (maybe it's a dictionary type structure) code over and over again and write a framework around that. Congratulations, you now have an ORM, except that it's one hacked together by a busy engineer, far less tested, and missing features that would save you writing a lot of boilerplate code. Meanwhile, your engineer realizes that one should separate view logic from business logic, and maybe have some glue in between, so you've got a request-handling layer, a view (+view model?) layer, and something that handles business logic and database access (or maybe those two things are separate, depending on your engineer.) Congratulations, you now have an MVC stack! Except again it's one that's far less tested, has no documentation, and has been hacked together by your engineer, who I should mention is very busy trying to write all this boilerplate code. So now you've got all that taken care of, it's time to allow people to login. Your engineer is now spending their time writing an authentication system. Your engineer is certainly skilled, but there are a lot of nuances to get right with this sort of thing, so your authentication system has a lot of little security flaws and subtle bugs that you will spend the next several months fixing. You see where this is going...

PommeDeTerre 14y ago
I think you're only looking at one side of the coin. While there may be benefits to using ORMs and web app frameworks, there are also many significant drawbacks. A lot of the time, these drawbacks can wipe out many, if not all, of the gains. You usually end up losing a lot of flexibility when using ORMs or frameworks. If you don't do things exactly the way the ORM or framework forces you to, you'll be in a world of pain. Of course, there will always be situations where you need to customize things to your particular case. ORMs and frameworks are notorious for making this far more difficult than it should be. I've seen teams waste more time twisting a framework or ORM to handle a unique situation than it may have saved them in the first place. It's naive to think that ORMs and frameworks necessarily have fewer bugs, or better quality, or better documentation, or more tests than custom-written code. I've worked on enough teams that have had a hell of a time due to ORM or framework bugs. It's worse when you're using a closed-source ORM or framework that you can't easily fix directly. Somewhat related to that, for any sizable or complex application, it's just not possible to completely avoid understanding the inner working of the ORM or other frameworks that you're using. Eventually you'll need to dig into them, whether it's to fix a bug, or to figure out how to do something that's poorly documented, or perhaps to figure out performance problems. This can be very time-consuming, often exceeding the effort needed to write, debug and test custom code. If you do need to make additions or fixes to an externally-developed ORM or framework, this can very easily cause headaches down the road when it comes to upgrading to a new version of the framework or ORM. You end up having to choose between using an old, patched version of the framework that lacks critical features or bug fixes, or you can upgrade and try to re-apply your fixes/additions, or maybe even moving to a whole new framework altogether. Having to deal with issues like these can make custom code look very compelling.
- fallenhitokiri 14y ago
  I believe ORMs are great for something notJim mentions in his first sentence: "why not start with one?". You lose some flexibility, you can encounter some bugs. But till you reach this point you do not have to write SQL and (hopefully) gain some development speed, at least if you find an ORM that fits your needs. For some people who do not mess with databases every days this is IMHO great. I sadly do not remember who said it but I liked the idea "machines should be able to generate SQL much faster and more reliable than me". I think if you really hit the limits of an ORM (one of the bigger ones) and are not experienced with databases (and SQL) you reached the point where you should look for someone who is. But if you never do and only need CRUD and some relations you can live a happy life.
- notJim 14y ago
  I'd say that the main thrust of my argument could be summarized as "frameworks are good, you should use them to write apps." My observation is that often when people say they don't want to use a framework, they end up trying to write code that does similar things the framework would do, but because they're not experts in designing frameworks, they do a bad job of it. An example would be rather than using an ORM, people write some one-off database service, and then copy-and-paste and find-replace to write another one. This leads to a great deal of code duplication, and makes it difficult to fix bugs and fix performance issues (e.g., to implement a read-through cache, you'd have to go through each copy-pasted variant and implement the cache, rather than implementing it once in a base class.) It's not that these people are writing a better framework that adapts more to their needs, it's that they read on the internet "don't use frameworks" and decided they were smarter than all those other startups that use Rails. The result is generally a pile of half-complete, poorly-conceived ideas centered around decisions that no longer make a lot of sense. I say all this having used, seen and maintained many a homegrown framework in my time as a contract developer. I'm sure they're out there, but I've never seen a codebase that doesn't use a pre-existing framework that's better than one that does. I think if you work on a small team, it is very difficult to take the time to write a well-designed and well-tested framework in addition to your main responsibility, which is presumably to make features that users are dying to give you money to use. You are definitely right that you can paint yourself into a corner with frameworks. I suppose my theory is that as long as you're using an open-source framework, you will always be able to code your way out of whatever situation you end up in. I also believe (but obviously cannot prove) that in nearly all of these situations, the net effect of using a framework will be a positive in terms of both your code quality and your productivity. It goes without saying that if you choose to use a crappy framework, then you're going to have problems.
tonecluster 14y ago
I disagree, respectfully, that a skilled engineer would create a framework in the process of writing an app. Instead: they'd write the app using the microframeworks chosen. There's a big difference -- creating framework that solves a general set of problems is a different task than building an app. This argument assumes that an engineer will create a framework to solve their problem ("build an app"), and create an ORM, and will recreate an auth process, etc. This isn't necessarily true. Also - why assume that a skilled engineer will "hack" together anything? How hard is it to write a bunch of SQL? Not particularly hard at all - no need to hack (in the negative context of the work I infer you're using here). In fact, a skilled engineer will re-use the micro-components he/she's created. An unskilled engineer will "hack" together a new "framework", "ORM", and so on - creating their own megaframework - to defeat the purpose of using microframeworks in the first place? Sorry, that kind of engineer needs more experience or skills. Also, I think we need to clearly separate two things: writing an app, and writing a framework. I'm recommending using tools (microframeworks) to create apps, websites, etc -- not to ask an engineer to devolve into using these tools to create yet another megaframework in attempt to abstract away problems (that they probably don't fully understand, but should, as in your example of authentication) and failing. In fact, I'll go ahead and posit that a skilled engineer knows how to create an auth process (using existing crypto and hash libraries instead of rolling their own). An unskilled engineer -- not so much.

by notJim 14y ago

PommeDeTerre 14y ago

I think you're only looking at one side of the coin. While there may be benefits to using ORMs and web app frameworks, there are also many significant drawbacks. A lot of the time, these drawbacks can wipe out many, if not all, of the gains. You usually end up losing a lot of flexibility when using ORMs or frameworks. If you don't do things exactly the way the ORM or framework forces you to, you'll be in a world of pain. Of course, there will always be situations where you need to customize things to your particular case. ORMs and frameworks are notorious for making this far more difficult than it should be. I've seen teams waste more time twisting a framework or ORM to handle a unique situation than it may have saved them in the first place. It's naive to think that ORMs and frameworks necessarily have fewer bugs, or better quality, or better documentation, or more tests than custom-written code. I've worked on enough teams that have had a hell of a time due to ORM or framework bugs. It's worse when you're using a closed-source ORM or framework that you can't easily fix directly. Somewhat related to that, for any sizable or complex application, it's just not possible to completely avoid understanding the inner working of the ORM or other frameworks that you're using. Eventually you'll need to dig into them, whether it's to fix a bug, or to figure out how to do something that's poorly documented, or perhaps to figure out performance problems. This can be very time-consuming, often exceeding the effort needed to write, debug and test custom code. If you do need to make additions or fixes to an externally-developed ORM or framework, this can very easily cause headaches down the road when it comes to upgrading to a new version of the framework or ORM. You end up having to choose between using an old, patched version of the framework that lacks critical features or bug fixes, or you can upgrade and try to re-apply your fixes/additions, or maybe even moving to a whole new framework altogether. Having to deal with issues like these can make custom code look very compelling.

fallenhitokiri 14y ago
I believe ORMs are great for something notJim mentions in his first sentence: "why not start with one?". You lose some flexibility, you can encounter some bugs. But till you reach this point you do not have to write SQL and (hopefully) gain some development speed, at least if you find an ORM that fits your needs. For some people who do not mess with databases every days this is IMHO great. I sadly do not remember who said it but I liked the idea "machines should be able to generate SQL much faster and more reliable than me". I think if you really hit the limits of an ORM (one of the bigger ones) and are not experienced with databases (and SQL) you reached the point where you should look for someone who is. But if you never do and only need CRUD and some relations you can live a happy life.
notJim 14y ago
I'd say that the main thrust of my argument could be summarized as "frameworks are good, you should use them to write apps." My observation is that often when people say they don't want to use a framework, they end up trying to write code that does similar things the framework would do, but because they're not experts in designing frameworks, they do a bad job of it. An example would be rather than using an ORM, people write some one-off database service, and then copy-and-paste and find-replace to write another one. This leads to a great deal of code duplication, and makes it difficult to fix bugs and fix performance issues (e.g., to implement a read-through cache, you'd have to go through each copy-pasted variant and implement the cache, rather than implementing it once in a base class.) It's not that these people are writing a better framework that adapts more to their needs, it's that they read on the internet "don't use frameworks" and decided they were smarter than all those other startups that use Rails. The result is generally a pile of half-complete, poorly-conceived ideas centered around decisions that no longer make a lot of sense. I say all this having used, seen and maintained many a homegrown framework in my time as a contract developer. I'm sure they're out there, but I've never seen a codebase that doesn't use a pre-existing framework that's better than one that does. I think if you work on a small team, it is very difficult to take the time to write a well-designed and well-tested framework in addition to your main responsibility, which is presumably to make features that users are dying to give you money to use. You are definitely right that you can paint yourself into a corner with frameworks. I suppose my theory is that as long as you're using an open-source framework, you will always be able to code your way out of whatever situation you end up in. I also believe (but obviously cannot prove) that in nearly all of these situations, the net effect of using a framework will be a positive in terms of both your code quality and your productivity. It goes without saying that if you choose to use a crappy framework, then you're going to have problems.

tonecluster 14y ago

I disagree, respectfully, that a skilled engineer would create a framework in the process of writing an app. Instead: they'd write the app using the microframeworks chosen. There's a big difference -- creating framework that solves a general set of problems is a different task than building an app. This argument assumes that an engineer will create a framework to solve their problem ("build an app"), and create an ORM, and will recreate an auth process, etc. This isn't necessarily true. Also - why assume that a skilled engineer will "hack" together anything? How hard is it to write a bunch of SQL? Not particularly hard at all - no need to hack (in the negative context of the work I infer you're using here). In fact, a skilled engineer will re-use the micro-components he/she's created. An unskilled engineer will "hack" together a new "framework", "ORM", and so on - creating their own megaframework - to defeat the purpose of using microframeworks in the first place? Sorry, that kind of engineer needs more experience or skills. Also, I think we need to clearly separate two things: writing an app, and writing a framework. I'm recommending using tools (microframeworks) to create apps, websites, etc -- not to ask an engineer to devolve into using these tools to create yet another megaframework in attempt to abstract away problems (that they probably don't fully understand, but should, as in your example of authentication) and failing. In fact, I'll go ahead and posit that a skilled engineer knows how to create an auth process (using existing crypto and hash libraries instead of rolling their own). An unskilled engineer -- not so much.