5 ms·

I’m building a hostingplatform for fully managed Wordpress sites. By managed I mean both the WP install, but also plugins, themes and other customizations. I h

I’m building a hostingplatform for fully managed Wordpress sites. By managed I mean both the WP install, but also plugins, themes and other customizations. I have a customer who does marketing, SoMe, SEO and all that (as a small business for other small to medium businesses) but it turns out she spends a lot of time dealing with tech issues from WP sites (plugin update errors, other tech issues the hosting provider doesnt handle etc). I’ve been a SW dev for 10+ years now and know very little about WP. But it has always seemed crazy to me how everyone in the “WP space” seems to be making changes directly in prod o_O. I know WP is mostly configuration management, but Im going to build a platform for hosting based on methods and best practices from “my world” (docker, blue/green deploy, multiple environments (eg for test), automated tests, etc). WP wasnt exactly intended to be run this way but I want to build it as both a personal challenge, and to prove - at least to myself, that the current Leeroy Jenkins in Prod approach has a better alternative, and ofc because I have a customer willing to pay for the service. And to be able to provide GDPR compliant by design (WP) hosting (EU based). Im using Hetzner Cloud for the servers. Today I got a blue/green deploy setup working with docker-compose, nginx and wordpress images, in which WP updates are installed just by updating the docker image version, and the switch betweem active instance (blue/green) is a simple cmd.

gremlinsinc 4y ago
I've been wanting to pull the ripcord on a marketplace platform, think a shopify for WP Maintenance services and freelancers, or an unbundling of fiverr's WP gigs marketplace. Basically some sort of service that merges tools like MainWP with dashboards, issues, kanban boards, feedback for managing clients, logins, hosting issues, etc. It just always seems to broad and scope to get started, and I personally kinda hate WP and would probably do it all in Laravel. I'd love to figure out how to do something like WP Maintenance but for legacy Laravel Apps, but I'm not sure there's a market fit for that.
unity1001 4y ago
> I’ve been a SW dev for 10+ years now and know very little about WP WP has become a gigantic ecosystem with its own sub-ecosystems and specializations that pertain to those ecosystems at this point. Seeing it as a 'Linux ecosystem, but as a web app' would be a better way to understand the landscape. Therefore, you should definitely try to learn more about WP because: > But it has always seemed crazy to me how everyone in the “WP space” seems to be making changes directly in prod o_O ...even starting from the need to understand that, there are a lot of things to understand. People make changes directly to prod because making WP high available is very difficult due to the HA-enabled infra that is out today not supporting stateful workloads well, hence its difficult to have dev, staging, prod deployments etc. Especially because a change in plugin code or setting (an update, changing a setting etc) may trigger database changes and one wont work without the other. And when you end up setting up one, it comes out way too expensive for any individual or small business to afford paying for it. As opposed to a managed, non-HA WP hosting provider allowing you to host a WP site that handles million or more visitors every month for $45 as opposed to having to pay $200-300 for a HA setup so that they wont make changes in production and they will have copies of their site running. Which, they can also do by buying another $45/month hosting and duplicating their site behind an IP-gating etc by the way. WP site owners use plugins to make happen what they want to make happen, and a lot of these plugins need to be able to modify files in the filesystem to be able to work their magic. This ranges from being able to resize thumbnails to creating sitemaps to more complicated ecommerce application needs. Some need to modify .htaccess in order for a major feature to work. Others may need to modify the config file to add something etc. And at that moment the whole containerization, docker, deployments, multiple environments thing hit the wall: If all the plugins are built into the container, they wont be able to modify the filesystem, especially their own files when needed. Which is a game breaker for any average WP site owner. You could use an NFS to mount the wp-content folder, and that would fix a LOT of your issues. However, cheap, reliable managed NFS offerings are extremely scarce. And what's there is expensive. So you will end up having to rack up the price to a point where most people wont want to pay. Even when you solve that, you still have problems - you are now building the WP core into the containers, and therefore are able to deploy automatic WP version updates and also having improved performance, great. But what will happen when you deploy a WP version update that breaks various important plugins because they are not compatible with that version yet? Boom. Upset users. A broken plugin and feature is most of the time is as bad as your entire site being broken. You could prevent users from installing plugins and you could offer a selection of plugins that the users can use. Assuming that you solved the NFS problem. Then you could avoid version conflicts, compatibility issues. But then many users wont use your service because they cant install the plugins they want. You could have a market like that and a lot of web agencies do that apparently, but they are for their clients with specific needs who wouldnt need anything more freeform and complicated like most of the WP users do. ... The best fit on the market for solving all of these is Google Kubernetes Engine + Google Filestore + Google Cloud SQL. (people complain about AWS nfs offering having spotty performance - bad for WP). But Google egress charges literally kill any chances of Google having any foothold in the WP ecosystem because with their outrageous egress pricing, a small WP site owner that hosts a middling WP site for $25/month would have to pay another $15/month for egress traffic costs on average, and that makes google double or triple more expensive for out-of-the-box WP compatible web hosts right from the start. And that's a variable price too - no WP site owner would want to pay a surprise $100 in a random month because their site has had more traffic suddenly. Standard WP hosts can provide everything from cpu/db to traffic costs even for a million visitors for $50/month fixed. So anyone who builds any WP SaaS or consumer grade hosting at Google infra would have to start with 3-4 times more expensive compared to all other offerings when they add their own profit margin. If you end up choosing to not allow users install plugins and themes, using WP multisite could be much better for your setup - it acts as a singular WP installation that can host an infinite amount of WP sites inside itself, with various plugin bundles activated/deactivated for different packages. You can build this into a container, and if you don't have any plugin that needs to use the filesystem, you can even deploy it in K8 in front of a managed DB and with the media uploads sent to some S3 compatible storage or something. (That requires custom code). Also - nginx is not a good choice for WP. A large swath of WP plugins require .htaccess to be present and accessible for adding custom rules for their features, and this makes Apache the better candidate for such a situation. Apache with event mpm and php-fpm behind it is as performant as nginx these days, so you can just slap Apache and not ever have to deal with the rewrite rules that your customers will be asking you to add to nginx rewrites.
- Msurrow 4y ago
  At a first read through there seems to be a of usefull knowledge and experience in your post. So I’ll surely go through it in detail and take away everything I can. That being said, unfortunately you have a few assumptions, that makes some of what you write less relevant. > But then many users wont use your service because they cant install the plugins they want. This is sort of the core misunderstanding: This setup is defined for one customer (marketing agency) so I dont really care how the masses would like to use WP hosting. And in extension of this users will not be allowed to freely install plugins, customizations etc. That is want it means for this to be “fully managed” (I wrote about what and how in another comment above). The business need for this thing came about in the first place because the customer didnt not want to be the one to fiddle with webservers, plugins and updates. So the goal is to build something that strikes a balance between enabling the customer to deliver their (marketing stuff) to their clients and doing it in a controlled manner, for an acceptable price. Restrictions and processes are going to drive prices down, while the need for flexibility and time-to-deliver will drive price up. The question is if there exists a balance thats acceptable to all parties. On a more overall level, I’ll add that I get how WP has become a huge ecosystem like you say. But part of why I fint this little project interesting is that, in my eyes, its an ecosystem that has evolved through thousands of low-tech users (certainly not experienced devs) hacking together bad, but somewhat functioning, solutions and making so many bad tech decisions along the way, resulting in a mountain of tech debt which leads to all of the stacked problems you describe in your comment. I think you are completely correct in all the issues you describe, but Im interested in whats possible to do if all the usual ways of doing things in this ecosystem is thrown out the window, and we look at how things should be done. Is it possible that a better solution doesnt exist? Sure! But wont know still we try. There is just so many ingrown “bad things” that seems to be the norm in this WP world, that would not even be considered in a more regular sw development situation: Imagine giving the somewhat “tech savy” end user access to the C# codebase, installing nuget packages (WP: plugins), making code changes (WP: functions.php) and messing with appsettings (WP: configs).. and directly in prod. Its just too far out. I wont even let the senior devs on my team deploy code changes directly to prod. We have git, PRs, reviews, tests, build pipelines, test environments and so on.. We’re not savages. Or, a less dramatic example: a few comments/people have asked now, what I’ll do about “that broken plugin which breaks the site”. Well, I’m not going to use it - its broken!? Just like I’m not using broken python packages.
BilalBudhani 4y ago
hey I've been tinkering around to build something on the same lines using similar stack. Let's connect? I'm hey[at]bilal[dot]dev
jhoelzel 4y ago
i have been beating around this bush for eons and so far my best release strategy is custom build wordpress container where plugins and themes are built in. The problem i was facing was mostly that almost all of the plugins used in "general", do not come with a testing architecture in any way. How are you going to handle a "rouge" plugin not closing its <div> tag correctly? I used to have "hidden" control pages where i would render the most used elements with demo content which i could check against. But then you throw things like composer into the game and most of that goes out of the window. After all these years, my best approach is still making a backup, hitting the upgrade button, and praying :/
- Msurrow 4y ago
  Great question! Its one of the things where Im very curious to see if my plans hold up when they meet reality. I plan to solve it my a mix of process and multiple environments: 0) Base assumption is that Im starting from a working site and everything that may break it is going to be a change to the working site. 1) Every live/prod site will have a test environment. Like test.example.com 2) (process) The customer that wants to change something will work in Test. Or me, if Im the one indtaling the plugins. Plugin(s) are installed/php changed and then verified to be working as intended in Test (manuals testing basically) 3) (process) Customer never works in prod. 4) (process) When the changes made in Test are working as expected, Ill then update the docker images used for prod with the (verified) working changes. 5) Deploy new images to prod in-actice site. 6) Switch active / in-active. 7) Verify updated live site works as intended. If not, switch back to previous active bringing the live site back online without errors and go back to step 2. Its not going to be absolutely error free, nothing ever is, but the intention is to (a) minimize the number of errors by enforcing some process designed to promote quality[1], and (b) when something breaks it should be very fast to switch back to a working site, so error fixing dont need to be done in a rush because the (live) site is down. 1: Yes the process is old school compared to todays CI/CD pipelines and multiple deployments to prod each day, but that is not the goal nor the business needs. A stable working site is. Edit: not sure I actually answered your question. A broken plugin (missing </div>) would never make it past step 2.
  - 4y ago
ivyirwin 4y ago
Interesting, I have a similar goal but in the Drupal space. My partner and I have been building Drupal sites for clients for almost 15 years and we keep running into hosting issues for our clients at scale. Most of the "managed" options get extremely expensive very quickly, but they're also not managed from a content/modules perspective. We're trying to do the best of both worlds and provide a managed (albeit opinionated) install that leverages our experience on the development side with performance and ease of use on the hosting side. Would be happy to compare notes as it comes together. Our website is in my profile and our email is hello@ the domain listed there.
- Msurrow 4y ago
  Interesting. I have zero experience with Drupal, but I assume its similar to WP in its architecture since you bring it up :) I think any good solution to working with these 50/50 code and configuration mgmt technologies have to start with a solid analysis and understanding of what the actual business needs are(?). And then (re)evaluate how things are done. And then perhaps also pricing models when it comes to the ‘“managed”’ part you mention. I think, for what Im building, there will end up needing to be a “managed hosting” offering which is more restricted and a custom development/T&M work which will be hourly. Could be interesting to compare notes when get more of the setup up and running

gremlinsinc 4y ago

I've been wanting to pull the ripcord on a marketplace platform, think a shopify for WP Maintenance services and freelancers, or an unbundling of fiverr's WP gigs marketplace. Basically some sort of service that merges tools like MainWP with dashboards, issues, kanban boards, feedback for managing clients, logins, hosting issues, etc. It just always seems to broad and scope to get started, and I personally kinda hate WP and would probably do it all in Laravel. I'd love to figure out how to do something like WP Maintenance but for legacy Laravel Apps, but I'm not sure there's a market fit for that.

unity1001 4y ago

> I’ve been a SW dev for 10+ years now and know very little about WP WP has become a gigantic ecosystem with its own sub-ecosystems and specializations that pertain to those ecosystems at this point. Seeing it as a 'Linux ecosystem, but as a web app' would be a better way to understand the landscape. Therefore, you should definitely try to learn more about WP because: > But it has always seemed crazy to me how everyone in the “WP space” seems to be making changes directly in prod o_O ...even starting from the need to understand that, there are a lot of things to understand. People make changes directly to prod because making WP high available is very difficult due to the HA-enabled infra that is out today not supporting stateful workloads well, hence its difficult to have dev, staging, prod deployments etc. Especially because a change in plugin code or setting (an update, changing a setting etc) may trigger database changes and one wont work without the other. And when you end up setting up one, it comes out way too expensive for any individual or small business to afford paying for it. As opposed to a managed, non-HA WP hosting provider allowing you to host a WP site that handles million or more visitors every month for $45 as opposed to having to pay $200-300 for a HA setup so that they wont make changes in production and they will have copies of their site running. Which, they can also do by buying another $45/month hosting and duplicating their site behind an IP-gating etc by the way. WP site owners use plugins to make happen what they want to make happen, and a lot of these plugins need to be able to modify files in the filesystem to be able to work their magic. This ranges from being able to resize thumbnails to creating sitemaps to more complicated ecommerce application needs. Some need to modify .htaccess in order for a major feature to work. Others may need to modify the config file to add something etc. And at that moment the whole containerization, docker, deployments, multiple environments thing hit the wall: If all the plugins are built into the container, they wont be able to modify the filesystem, especially their own files when needed. Which is a game breaker for any average WP site owner. You could use an NFS to mount the wp-content folder, and that would fix a LOT of your issues. However, cheap, reliable managed NFS offerings are extremely scarce. And what's there is expensive. So you will end up having to rack up the price to a point where most people wont want to pay. Even when you solve that, you still have problems - you are now building the WP core into the containers, and therefore are able to deploy automatic WP version updates and also having improved performance, great. But what will happen when you deploy a WP version update that breaks various important plugins because they are not compatible with that version yet? Boom. Upset users. A broken plugin and feature is most of the time is as bad as your entire site being broken. You could prevent users from installing plugins and you could offer a selection of plugins that the users can use. Assuming that you solved the NFS problem. Then you could avoid version conflicts, compatibility issues. But then many users wont use your service because they cant install the plugins they want. You could have a market like that and a lot of web agencies do that apparently, but they are for their clients with specific needs who wouldnt need anything more freeform and complicated like most of the WP users do. ... The best fit on the market for solving all of these is Google Kubernetes Engine + Google Filestore + Google Cloud SQL. (people complain about AWS nfs offering having spotty performance - bad for WP). But Google egress charges literally kill any chances of Google having any foothold in the WP ecosystem because with their outrageous egress pricing, a small WP site owner that hosts a middling WP site for $25/month would have to pay another $15/month for egress traffic costs on average, and that makes google double or triple more expensive for out-of-the-box WP compatible web hosts right from the start. And that's a variable price too - no WP site owner would want to pay a surprise $100 in a random month because their site has had more traffic suddenly. Standard WP hosts can provide everything from cpu/db to traffic costs even for a million visitors for $50/month fixed. So anyone who builds any WP SaaS or consumer grade hosting at Google infra would have to start with 3-4 times more expensive compared to all other offerings when they add their own profit margin. If you end up choosing to not allow users install plugins and themes, using WP multisite could be much better for your setup - it acts as a singular WP installation that can host an infinite amount of WP sites inside itself, with various plugin bundles activated/deactivated for different packages. You can build this into a container, and if you don't have any plugin that needs to use the filesystem, you can even deploy it in K8 in front of a managed DB and with the media uploads sent to some S3 compatible storage or something. (That requires custom code). Also - nginx is not a good choice for WP. A large swath of WP plugins require .htaccess to be present and accessible for adding custom rules for their features, and this makes Apache the better candidate for such a situation. Apache with event mpm and php-fpm behind it is as performant as nginx these days, so you can just slap Apache and not ever have to deal with the rewrite rules that your customers will be asking you to add to nginx rewrites.

Msurrow 4y ago
At a first read through there seems to be a of usefull knowledge and experience in your post. So I’ll surely go through it in detail and take away everything I can. That being said, unfortunately you have a few assumptions, that makes some of what you write less relevant. > But then many users wont use your service because they cant install the plugins they want. This is sort of the core misunderstanding: This setup is defined for one customer (marketing agency) so I dont really care how the masses would like to use WP hosting. And in extension of this users will not be allowed to freely install plugins, customizations etc. That is want it means for this to be “fully managed” (I wrote about what and how in another comment above). The business need for this thing came about in the first place because the customer didnt not want to be the one to fiddle with webservers, plugins and updates. So the goal is to build something that strikes a balance between enabling the customer to deliver their (marketing stuff) to their clients and doing it in a controlled manner, for an acceptable price. Restrictions and processes are going to drive prices down, while the need for flexibility and time-to-deliver will drive price up. The question is if there exists a balance thats acceptable to all parties. On a more overall level, I’ll add that I get how WP has become a huge ecosystem like you say. But part of why I fint this little project interesting is that, in my eyes, its an ecosystem that has evolved through thousands of low-tech users (certainly not experienced devs) hacking together bad, but somewhat functioning, solutions and making so many bad tech decisions along the way, resulting in a mountain of tech debt which leads to all of the stacked problems you describe in your comment. I think you are completely correct in all the issues you describe, but Im interested in whats possible to do if all the usual ways of doing things in this ecosystem is thrown out the window, and we look at how things should be done. Is it possible that a better solution doesnt exist? Sure! But wont know still we try. There is just so many ingrown “bad things” that seems to be the norm in this WP world, that would not even be considered in a more regular sw development situation: Imagine giving the somewhat “tech savy” end user access to the C# codebase, installing nuget packages (WP: plugins), making code changes (WP: functions.php) and messing with appsettings (WP: configs).. and directly in prod. Its just too far out. I wont even let the senior devs on my team deploy code changes directly to prod. We have git, PRs, reviews, tests, build pipelines, test environments and so on.. We’re not savages. Or, a less dramatic example: a few comments/people have asked now, what I’ll do about “that broken plugin which breaks the site”. Well, I’m not going to use it - its broken!? Just like I’m not using broken python packages.

BilalBudhani 4y ago

hey I've been tinkering around to build something on the same lines using similar stack. Let's connect? I'm hey[at]bilal[dot]dev

jhoelzel 4y ago

i have been beating around this bush for eons and so far my best release strategy is custom build wordpress container where plugins and themes are built in. The problem i was facing was mostly that almost all of the plugins used in "general", do not come with a testing architecture in any way. How are you going to handle a "rouge" plugin not closing its <div> tag correctly? I used to have "hidden" control pages where i would render the most used elements with demo content which i could check against. But then you throw things like composer into the game and most of that goes out of the window. After all these years, my best approach is still making a backup, hitting the upgrade button, and praying :/

Msurrow 4y ago
Great question! Its one of the things where Im very curious to see if my plans hold up when they meet reality. I plan to solve it my a mix of process and multiple environments: 0) Base assumption is that Im starting from a working site and everything that may break it is going to be a change to the working site. 1) Every live/prod site will have a test environment. Like test.example.com 2) (process) The customer that wants to change something will work in Test. Or me, if Im the one indtaling the plugins. Plugin(s) are installed/php changed and then verified to be working as intended in Test (manuals testing basically) 3) (process) Customer never works in prod. 4) (process) When the changes made in Test are working as expected, Ill then update the docker images used for prod with the (verified) working changes. 5) Deploy new images to prod in-actice site. 6) Switch active / in-active. 7) Verify updated live site works as intended. If not, switch back to previous active bringing the live site back online without errors and go back to step 2. Its not going to be absolutely error free, nothing ever is, but the intention is to (a) minimize the number of errors by enforcing some process designed to promote quality[1], and (b) when something breaks it should be very fast to switch back to a working site, so error fixing dont need to be done in a rush because the (live) site is down. 1: Yes the process is old school compared to todays CI/CD pipelines and multiple deployments to prod each day, but that is not the goal nor the business needs. A stable working site is. Edit: not sure I actually answered your question. A broken plugin (missing </div>) would never make it past step 2.
- 4y ago

ivyirwin 4y ago

Interesting, I have a similar goal but in the Drupal space. My partner and I have been building Drupal sites for clients for almost 15 years and we keep running into hosting issues for our clients at scale. Most of the "managed" options get extremely expensive very quickly, but they're also not managed from a content/modules perspective. We're trying to do the best of both worlds and provide a managed (albeit opinionated) install that leverages our experience on the development side with performance and ease of use on the hosting side. Would be happy to compare notes as it comes together. Our website is in my profile and our email is hello@ the domain listed there.

Msurrow 4y ago
Interesting. I have zero experience with Drupal, but I assume its similar to WP in its architecture since you bring it up :) I think any good solution to working with these 50/50 code and configuration mgmt technologies have to start with a solid analysis and understanding of what the actual business needs are(?). And then (re)evaluate how things are done. And then perhaps also pricing models when it comes to the ‘“managed”’ part you mention. I think, for what Im building, there will end up needing to be a “managed hosting” offering which is more restricted and a custom development/T&M work which will be hourly. Could be interesting to compare notes when get more of the setup up and running