7 ms·

I would add this to the complaints: - NOT AMENABLE TO SOURCE CONTROL Half the code lives in the database, configuration is all mixed up with content, and ther

by tedsuo 14y ago

I would add this to the complaints: - NOT AMENABLE TO SOURCE CONTROL Half the code lives in the database, configuration is all mixed up with content, and there's no reliable programmatic way to extract it. It makes doing deployment a nightmare. And the worst part? Drupal developers by and large have such a narrow range of experience that they don't even know what they are missing. They have no idea what good coding standards look like, a lot of them don't even have a development/production separation: they just hack away on the live site with no source control or backup. And no one writes tests, or sees why they would be useful. Some of the big professional drupal shops appear to have basically written their own cms on top of drupal, with a bunch of terrible hacks to try to make source control kind of work, and just use the drupal name basically as a marketing technique. So glad I don't have to deal with any of that insanity any more.

oinksoft 14y ago
This is the main reason I chucked my Drupal 'cuffs in 2008, no longer taking on new Drupal projects and helping my clients move over to other Drupal service providers: It was very, very difficult to scale up my business because bringing on extra developers, and having consistent local/dev/staging/production environments was at least an order of magnitude more difficult that it would be with other tools. In Drupal 6, the only way to reproduce an environment was with a massive install profile for the project, and the average Drupal developer was entirely clueless to matters of testing and version control when compared to the average Django/Rails/CakePHP developer. This made for many painful projects and unhappy clients, usually beginning 6-8 weeks into the project when the issues of database drift became inevitable. And like you said, everything lives in the database. So the chances that some production issue (or new issue only seen in dev) was caused by an errant user action in the Drupal UI was pretty high, and this made debugging a nightmare -- where is my issue really coming from? I'd just about forgotten about my Drupal days of yore, and between the "click monkey" analogy and the "clear the cache" nostalgia, the author sure conjured up some bad memories!
jackbravo 14y ago
This is why http://drupal.org/project/features http://drupal.org/project/features module was born. To allow you to export your functionality to code and have it under version control. Features and things like http://drupal.org/project/drush_make http://drupal.org/project/drush_make is what makes a lot of the drupal distributions (http://drupal.org/project/distributions http://drupal.org/project/distributions) that keep popping up because sourced controlled, feature driven development is now possible in drupal. Also the testing part has been improving, and although very few of the contrib modules have tests most of core has automatic testing in place for when you submit a patch as you can see in the issue queues (http://drupal.org/node/466576 http://drupal.org/node/466576).
- w4y2 14y ago
  Features is good as a concept, but in implementation it fails considerably. Here are a few issues: - How to deprecate a feature (I created http://drupal.org/node/1400346 http://drupal.org/node/1400346 to attempt to solve this, but admitedly, it's a very dangerous module) - Ever created a feature with tons of things in it? Hello, 5 minute load times for admin pages! - How do you merge two features together? How do you commit just one feature and not another (say, one you're still working on?) The CMI initiative will hopefully take care of a lot of the messiness and bugs in Features.
  - vaporstun 14y ago
    The other downside of Features in Drupal is that if we export something as a Feature (say a View) and then our site builders tweak and change that View for our clients and then we update some core element of that View and re-export the Feature, that re-export will overwrite any custom changes that were made to that View, thereby trashing any of the custom changes. This is a real problem with Features to which I'm surprised there has not yet been a good solution.
    - jackbravo 14y ago
      Same problem as other frameworks. You need to know the rules. If you use rails, and the designer changes the CSS without committing the change to git, then next code push will erase his change.
  - jackbravo 14y ago
    Right, features is not perfect. Upgrading a feature from D6 to D7 has been a pain to me. I've merged features together and it was not as hard, at the end it is all code, you just need to be very careful, but at the same time you have git to revert if something goes wrong. I don't understand the part about committing just one feature. You can commit just one =P.
  - exratione 14y ago
    Features is indeed the answer, and all of the potential issues are manageable. I finished up a project earlier this year that has more than 300 modules active in a custom install profile, with 10MB of features modules stuck in there. We committed/contributed to a few fixes in core and Features to speed it up greatly. That's out in the wild on major sites and doing fine. We didn't have any terrible admin screen issues, or indeed any terrible issues beyond occasional painful version control collisions.
emmelaich 14y ago
> [ ..in the database ..] Similar comments for Moodle, another popular PHP app. You cannot even change the url for your site without rewriting content in the database.
- smacktoward 14y ago
  WordPress too. In fact in WordPress the site root URL is stored in two separate configuration fields, so woe betide you if you rewrite one field and not the other. Thankfully the WP core devs seem to have been slowly cleaning stuff like this up over the last couple years. So even though this wart persists, I have hope it'll eventually get zapped.
  - jasey 14y ago
    In my wp-config.php file I have the following logic https://gist.github.com/3831423 https://gist.github.com/3831423
hippich 14y ago
On one hand there is Features modules and tons of modules built on top of it. On the other hand - if this problem could be solved - Drupal become holy grail, which should not exist! :)
knieveltech 14y ago
I work for a company that provides site implementation, custom feature coding and ongoing support for ~30 enterprise websites and all of them have this in common: 100% of the source code for each site is in revision control (SVN or Git, depending on client preferences). If you want to complain about Drupal be my guest. The ground's very fertile, but what you've posted here is FUD. Code in the database? No, just no. While it is technically possible to stuff code into the database community best practices (not to mention common sense) strongly advise against this practice. What you are describing is the system handing you enough rope to hang yourself, not it's default behavior. Configuration lives in configuration tables. Content lives in content tables. "they just hack away on the live site with no source control or backup" It would be a mistake to take the fumblings of a few scrub freelancers or interns and extrapolate that as being the norm for all (or even most) Drupal developers. For all core modules and all quality* contrib modules there are $N_load() functions to extract these (reliably) from the database. * With ~8000 contributed modules to choose from, quality varies wildly. While unfortunate, it's to be expected when you've got that many chefs in the kitchen.
- apendleton 14y ago
  This is quibbling over semantics. Drupal's custom content types, custom fields, and Views all allow developers to express complex logic that would, in any other development environment, be expressed as code and committed to version control -- creating a new content type in Drupal and adding fields to it is exactly analogous to creating a new model in Django. You can call it "configuration" if that makes it easier to justify its being in a table, but it doesn't change the fact that Drupal developers spend lots of time building shit that can't go in a VCS, can't be easily tested, can't be branched/merged, and can't be easily deployed without a convoluted export/import dance. Just because there isn't literal code in the database, it doesn't mean there's a clean logic/content separation.
  - fauigerzigerk 14y ago
    Not that I know much about Drupal, but isn't it inevitable that stuff that can be changed via the UI (sometimes even per user) has to live in the database? This is simply a consequence of blurring the lines between developers and users, which is exactly the point of CMSs. Actually, I think what we should be asking is this: Do CMS databases and VCS really have to be two completely seperate universes? I would say no. There is room for a new type of CMS database design that includes all the functions of a VCS.
lotyrin 14y ago
I work on big Drupal sites and this simply is not true. Drupal lets you get a ton of shit accomplished without making you understand a single line of code, have a reasonable Dev-Stage-Prod workflow, QA, even version control or backups, etc., but it's completely possible to do all of that. That means that it can scale from anywhere from something like a 16 year-old's shared-host fanfiction site to say Whitehouse.gov. Of course that means that there will be tons of fuckups live-coding (or live-configuring) on production, or people that don't export and version things in environments when they should, and that people will construct fragile combinations of modules and configuration where a tiny bit of code would have sufficed, or write tons of fragile custom code with no tests or security review when there was already a module for that. But allowing people to be fuckups doesn't force them to be, and they can be trained.
- mtts 14y ago
  "That means that it can scale from anywhere from something like a 16 year-old's shared-host fanfiction site to say Whitehouse.gov." Scaling to Whitehouse.gov is not impressive at all. Like a 16 year-old's shared-host fanfiction site whitehouse.gov really is nothing more than a content site, albeit a very large one. The problem is Drupal isn't content to simply be a CMS. It has to pretend to be a framework for developing web apps as well, and for that it's just plain terrible.
  - lotyrin 14y ago
    If you compare it to a framework it loses, obviously. It's massive and complicated. And while PHP's is trying hard to modernize, the tools and ecosystem still don't compare to Ruby or Python (see below). Drupal is only for when what it (or modules) can provide is a vast majority of the requirements of the project. Anything that couldn't be described as "nothing more than a content site" probably would not be a good fit. The only thing it has to offer is when what you need is one of the myriad things it can be configured to be, because then you get a bunch of community eyes and tests on a bunch of code you didn't have to write. Edit: Discussion on recent improvements to PHP's and its ecosystem (which are exciting progress in a direction I believe there is more room to go) http://news.ycombinator.com/item?id=4605715 http://news.ycombinator.com/item?id=4605715
modarts 14y ago
Wow. This sounds exactly like a complaint that would be lobbed against SharePoint (rightfully so, it's a special kind of hellish nightmare for people responsible for developing on it.)
mmcnickle 14y ago
Such is the inner platform effect. I would happily bet there is someone working on a drupal-specific, poor substitute for version control.
- lotyrin 14y ago
  Nope. They're busting the inner platform and having config live in the filesystem to get versioned like anything else. Good luck with those prejudices, though.
  - mmcnickle 14y ago
    Does it address config drift in production where users make changes that aren't reflected in the versioned codebase? The question is not intended as a retort, I'm interested in the answer. The prejudices I have against Drupal are the result of my experience of it starting as far back as version 4. I don't believe they are unfounded.
    - TylerE 14y ago
      You're not wrong at all. I'm currently investing moving our sites (a network with 1M+ visits/and 12M+ pageviews) from Drupal 6 to Rails. We have a large amount of custom code (I've written 20+ modules for this platform), and yet we've gotten to a point where we really want Drupal (and PHP) out of our lives, because it isn't worth the headaches anymore.
ZoFreX 14y ago
Absolutely not true. When I was working with Drupal 6, CCK and Views 2, I would use the GUI to try out new content types and views, but I would then export their definitions and put the code for them into a custom module, which was under version control. When I first started out I actually wasn't using CCK or Views 2, I was creating new node types with a custom module, and if I were to go back to Drupal now I would be sorely tempted to do this again. Views was a bit too fragile and the queries it generated were horrific, and I think if you can code and know PHP and MySQL well they really give you very little value, or possibly even work out slower overall.
Daniel_b 14y ago
I completely disagree with your views about Drupal development. We've recently migrated bmj.com (a big site with gets 1.6 million visitors and 6.8 million page views a month) from old Java Servlet based platform to Drupal. We've used continuous development process where up to 4 developers were working on the same code base in dev, staging/testing and prod environments. It is now a standard Drupal development practice to export all configuration data to the source code. BMJ.com is not a simple site, it has an article page with quite complex data structure with up to 60 meta data elements (author details, publication date, vol/issue number, section, series, category, taxonomy, relations to other articles, open access flag, etc...) being rendered at the relevant blocks. The business and editorial departments have been happy with the new opportunities Drupal is offering them. For more detail, please access Drupal success case study: http://drupal.org/node/1557636 http://drupal.org/node/1557636
- berkes 14y ago
  > from old Java Servlet based platform to Drupal. This does not say much about Drupal being good in absolute terms. It only tells us the obvious point that Drupal is a lot better then your old application. What you state would have been just as trough had you developed in Rails, Django, and maybe even in PHPnuke. What you don't state, is why Drupal was the better fit compared to modern alternatives. Is it?
dougvann 14y ago
Right you are @tedsuo There are many ppl not writing tests and not using staging and best deployment practices. This is true in every CMS and WCF. I don't believe it it any more prevalent in Drupal than anywhere else. Drupal invites a lot of BEGINNER types to the arena of web solutions. As such you will certainly find the things you mention. I don't agree with your suspicion that big shops are hacking core and building custom CMSs upon Drupal in a vain effort to try and gain source control features. Rather... I'm aware of big shops sticking to Drupal bets practices and using Features, Strongarm, Plumber and other modules to get configuration into code and keep it source controlled. If Drupal deployment were the nightmare you s=describe, then the phenomenal adoption of Drupal that we see today would make no sense. Why would the Georgia Tech Authority decide to move SIXTY-FIVE sites off of a Oracle, Java, Vignette solution and onto Drupal? Well.. It is isn't because they like nightmares during deployment. Drupal is growing up. The issues that plagued us 5 years ago when I got involved are being solved with varying degrees of success and effort. We're not where we want to be but the market is eating us up. We are bullish about our future and know that out continued improvement will only serve to better our position as the leader that we are in the web space. Doug Vann [Drupal Trainer, Consultant, Developer] http://dougvann.com http://dougvann.com

by tedsuo 14y ago

oinksoft 14y ago

This is the main reason I chucked my Drupal 'cuffs in 2008, no longer taking on new Drupal projects and helping my clients move over to other Drupal service providers: It was very, very difficult to scale up my business because bringing on extra developers, and having consistent local/dev/staging/production environments was at least an order of magnitude more difficult that it would be with other tools. In Drupal 6, the only way to reproduce an environment was with a massive install profile for the project, and the average Drupal developer was entirely clueless to matters of testing and version control when compared to the average Django/Rails/CakePHP developer. This made for many painful projects and unhappy clients, usually beginning 6-8 weeks into the project when the issues of database drift became inevitable. And like you said, everything lives in the database. So the chances that some production issue (or new issue only seen in dev) was caused by an errant user action in the Drupal UI was pretty high, and this made debugging a nightmare -- where is my issue really coming from? I'd just about forgotten about my Drupal days of yore, and between the "click monkey" analogy and the "clear the cache" nostalgia, the author sure conjured up some bad memories!

jackbravo 14y ago

This is why http://drupal.org/project/features http://drupal.org/project/features module was born. To allow you to export your functionality to code and have it under version control. Features and things like http://drupal.org/project/drush_make http://drupal.org/project/drush_make is what makes a lot of the drupal distributions (http://drupal.org/project/distributions http://drupal.org/project/distributions) that keep popping up because sourced controlled, feature driven development is now possible in drupal. Also the testing part has been improving, and although very few of the contrib modules have tests most of core has automatic testing in place for when you submit a patch as you can see in the issue queues (http://drupal.org/node/466576 http://drupal.org/node/466576).

w4y2 14y ago
Features is good as a concept, but in implementation it fails considerably. Here are a few issues: - How to deprecate a feature (I created http://drupal.org/node/1400346 http://drupal.org/node/1400346 to attempt to solve this, but admitedly, it's a very dangerous module) - Ever created a feature with tons of things in it? Hello, 5 minute load times for admin pages! - How do you merge two features together? How do you commit just one feature and not another (say, one you're still working on?) The CMI initiative will hopefully take care of a lot of the messiness and bugs in Features.
- vaporstun 14y ago
  The other downside of Features in Drupal is that if we export something as a Feature (say a View) and then our site builders tweak and change that View for our clients and then we update some core element of that View and re-export the Feature, that re-export will overwrite any custom changes that were made to that View, thereby trashing any of the custom changes. This is a real problem with Features to which I'm surprised there has not yet been a good solution.
  - jackbravo 14y ago
    Same problem as other frameworks. You need to know the rules. If you use rails, and the designer changes the CSS without committing the change to git, then next code push will erase his change.
- jackbravo 14y ago
  Right, features is not perfect. Upgrading a feature from D6 to D7 has been a pain to me. I've merged features together and it was not as hard, at the end it is all code, you just need to be very careful, but at the same time you have git to revert if something goes wrong. I don't understand the part about committing just one feature. You can commit just one =P.
- exratione 14y ago
  Features is indeed the answer, and all of the potential issues are manageable. I finished up a project earlier this year that has more than 300 modules active in a custom install profile, with 10MB of features modules stuck in there. We committed/contributed to a few fixes in core and Features to speed it up greatly. That's out in the wild on major sites and doing fine. We didn't have any terrible admin screen issues, or indeed any terrible issues beyond occasional painful version control collisions.

emmelaich 14y ago

> [ ..in the database ..] Similar comments for Moodle, another popular PHP app. You cannot even change the url for your site without rewriting content in the database.

smacktoward 14y ago
WordPress too. In fact in WordPress the site root URL is stored in two separate configuration fields, so woe betide you if you rewrite one field and not the other. Thankfully the WP core devs seem to have been slowly cleaning stuff like this up over the last couple years. So even though this wart persists, I have hope it'll eventually get zapped.
- jasey 14y ago
  In my wp-config.php file I have the following logic https://gist.github.com/3831423 https://gist.github.com/3831423

hippich 14y ago

On one hand there is Features modules and tons of modules built on top of it. On the other hand - if this problem could be solved - Drupal become holy grail, which should not exist! :)

knieveltech 14y ago

I work for a company that provides site implementation, custom feature coding and ongoing support for ~30 enterprise websites and all of them have this in common: 100% of the source code for each site is in revision control (SVN or Git, depending on client preferences). If you want to complain about Drupal be my guest. The ground's very fertile, but what you've posted here is FUD. Code in the database? No, just no. While it is technically possible to stuff code into the database community best practices (not to mention common sense) strongly advise against this practice. What you are describing is the system handing you enough rope to hang yourself, not it's default behavior. Configuration lives in configuration tables. Content lives in content tables. "they just hack away on the live site with no source control or backup" It would be a mistake to take the fumblings of a few scrub freelancers or interns and extrapolate that as being the norm for all (or even most) Drupal developers. For all core modules and all quality* contrib modules there are $N_load() functions to extract these (reliably) from the database. * With ~8000 contributed modules to choose from, quality varies wildly. While unfortunate, it's to be expected when you've got that many chefs in the kitchen.

apendleton 14y ago
This is quibbling over semantics. Drupal's custom content types, custom fields, and Views all allow developers to express complex logic that would, in any other development environment, be expressed as code and committed to version control -- creating a new content type in Drupal and adding fields to it is exactly analogous to creating a new model in Django. You can call it "configuration" if that makes it easier to justify its being in a table, but it doesn't change the fact that Drupal developers spend lots of time building shit that can't go in a VCS, can't be easily tested, can't be branched/merged, and can't be easily deployed without a convoluted export/import dance. Just because there isn't literal code in the database, it doesn't mean there's a clean logic/content separation.
- fauigerzigerk 14y ago
  Not that I know much about Drupal, but isn't it inevitable that stuff that can be changed via the UI (sometimes even per user) has to live in the database? This is simply a consequence of blurring the lines between developers and users, which is exactly the point of CMSs. Actually, I think what we should be asking is this: Do CMS databases and VCS really have to be two completely seperate universes? I would say no. There is room for a new type of CMS database design that includes all the functions of a VCS.

lotyrin 14y ago

I work on big Drupal sites and this simply is not true. Drupal lets you get a ton of shit accomplished without making you understand a single line of code, have a reasonable Dev-Stage-Prod workflow, QA, even version control or backups, etc., but it's completely possible to do all of that. That means that it can scale from anywhere from something like a 16 year-old's shared-host fanfiction site to say Whitehouse.gov. Of course that means that there will be tons of fuckups live-coding (or live-configuring) on production, or people that don't export and version things in environments when they should, and that people will construct fragile combinations of modules and configuration where a tiny bit of code would have sufficed, or write tons of fragile custom code with no tests or security review when there was already a module for that. But allowing people to be fuckups doesn't force them to be, and they can be trained.

mtts 14y ago
"That means that it can scale from anywhere from something like a 16 year-old's shared-host fanfiction site to say Whitehouse.gov." Scaling to Whitehouse.gov is not impressive at all. Like a 16 year-old's shared-host fanfiction site whitehouse.gov really is nothing more than a content site, albeit a very large one. The problem is Drupal isn't content to simply be a CMS. It has to pretend to be a framework for developing web apps as well, and for that it's just plain terrible.
- lotyrin 14y ago
  If you compare it to a framework it loses, obviously. It's massive and complicated. And while PHP's is trying hard to modernize, the tools and ecosystem still don't compare to Ruby or Python (see below). Drupal is only for when what it (or modules) can provide is a vast majority of the requirements of the project. Anything that couldn't be described as "nothing more than a content site" probably would not be a good fit. The only thing it has to offer is when what you need is one of the myriad things it can be configured to be, because then you get a bunch of community eyes and tests on a bunch of code you didn't have to write. Edit: Discussion on recent improvements to PHP's and its ecosystem (which are exciting progress in a direction I believe there is more room to go) http://news.ycombinator.com/item?id=4605715 http://news.ycombinator.com/item?id=4605715

modarts 14y ago

Wow. This sounds exactly like a complaint that would be lobbed against SharePoint (rightfully so, it's a special kind of hellish nightmare for people responsible for developing on it.)

mmcnickle 14y ago

Such is the inner platform effect. I would happily bet there is someone working on a drupal-specific, poor substitute for version control.

lotyrin 14y ago
Nope. They're busting the inner platform and having config live in the filesystem to get versioned like anything else. Good luck with those prejudices, though.
- mmcnickle 14y ago
  Does it address config drift in production where users make changes that aren't reflected in the versioned codebase? The question is not intended as a retort, I'm interested in the answer. The prejudices I have against Drupal are the result of my experience of it starting as far back as version 4. I don't believe they are unfounded.
  - TylerE 14y ago
    You're not wrong at all. I'm currently investing moving our sites (a network with 1M+ visits/and 12M+ pageviews) from Drupal 6 to Rails. We have a large amount of custom code (I've written 20+ modules for this platform), and yet we've gotten to a point where we really want Drupal (and PHP) out of our lives, because it isn't worth the headaches anymore.

ZoFreX 14y ago

Absolutely not true. When I was working with Drupal 6, CCK and Views 2, I would use the GUI to try out new content types and views, but I would then export their definitions and put the code for them into a custom module, which was under version control. When I first started out I actually wasn't using CCK or Views 2, I was creating new node types with a custom module, and if I were to go back to Drupal now I would be sorely tempted to do this again. Views was a bit too fragile and the queries it generated were horrific, and I think if you can code and know PHP and MySQL well they really give you very little value, or possibly even work out slower overall.

Daniel_b 14y ago

I completely disagree with your views about Drupal development. We've recently migrated bmj.com (a big site with gets 1.6 million visitors and 6.8 million page views a month) from old Java Servlet based platform to Drupal. We've used continuous development process where up to 4 developers were working on the same code base in dev, staging/testing and prod environments. It is now a standard Drupal development practice to export all configuration data to the source code. BMJ.com is not a simple site, it has an article page with quite complex data structure with up to 60 meta data elements (author details, publication date, vol/issue number, section, series, category, taxonomy, relations to other articles, open access flag, etc...) being rendered at the relevant blocks. The business and editorial departments have been happy with the new opportunities Drupal is offering them. For more detail, please access Drupal success case study: http://drupal.org/node/1557636 http://drupal.org/node/1557636

berkes 14y ago
> from old Java Servlet based platform to Drupal. This does not say much about Drupal being good in absolute terms. It only tells us the obvious point that Drupal is a lot better then your old application. What you state would have been just as trough had you developed in Rails, Django, and maybe even in PHPnuke. What you don't state, is why Drupal was the better fit compared to modern alternatives. Is it?

dougvann 14y ago

Right you are @tedsuo There are many ppl not writing tests and not using staging and best deployment practices. This is true in every CMS and WCF. I don't believe it it any more prevalent in Drupal than anywhere else. Drupal invites a lot of BEGINNER types to the arena of web solutions. As such you will certainly find the things you mention. I don't agree with your suspicion that big shops are hacking core and building custom CMSs upon Drupal in a vain effort to try and gain source control features. Rather... I'm aware of big shops sticking to Drupal bets practices and using Features, Strongarm, Plumber and other modules to get configuration into code and keep it source controlled. If Drupal deployment were the nightmare you s=describe, then the phenomenal adoption of Drupal that we see today would make no sense. Why would the Georgia Tech Authority decide to move SIXTY-FIVE sites off of a Oracle, Java, Vignette solution and onto Drupal? Well.. It is isn't because they like nightmares during deployment. Drupal is growing up. The issues that plagued us 5 years ago when I got involved are being solved with varying degrees of success and effort. We're not where we want to be but the market is eating us up. We are bullish about our future and know that out continued improvement will only serve to better our position as the leader that we are in the web space. Doug Vann [Drupal Trainer, Consultant, Developer] http://dougvann.com http://dougvann.com