15 ms·

Ask HN: Junior dev in charge of rewriting 500k line PHP app. Looking for advice

13 points by poncho_romero 2y ago

We are a 4 person company, that makes a web app I'll call Star. Today, Star is 11 years old and showing its age. It was developed entirely by one of the founders, who has no formal training, and the niche industry we serve has changed greatly in the last decade, rendering many of our core abstractions obsolete. In theory, Star follows MVC, but in practice there is no coherent architecture. Most of the functionality is tied up in 10k+ line controllers, which use an outdated framework full of magic. Due to a trend of centralization in the industry, our database design (focused on local markets) is entirely orthogonal to our customer's needs. We are also stuck on outdated technologies that no longer receive security updates. Although our existing platform is in bad shape, I would have been wary to rewrite (due to stories of Netscape et al), but the decision has been made. Because most of our competitors are also operating outdated/inflexible software platforms, if we can make this transition happen, we will have a huge leg up on them in development velocity. Our founders have deep knowledge of the industry, and we have our biggest client's support. We are primarily a CRUD app, but we also integrate with external APIs (e.g. to handle texting) and utilize Amazon S3 and SQS for generating reports and other long running tasks. Tentatively, I want to propose a Model-View-Controller-Services architecture, where each model is a thin database abstraction layer that knows nothing else; views are pure and idempotent (à la React components, but server side); each controller is responsible for one endpoint, receives a Request object, delegates work to services, and returns a Response object; and most work happens in services, which can call each other and models. We delegate authentication to Google, but will need to implement a very fine grained permissions system. I want to keep things simple, and avoid bringing in too many dependencies, so I am leaning towards a minimal set of Symfony components, rather than something like Laravel. One of the primary complaints we get is that our current system is too slow. In part, this is because most actions trigger a full page reload. I want to use HTMX to increase responsiveness while still keeping most of the functionality in the backend. We will also be using Docker. We may use Redis for session management, or we may again store session data in the database. Our application will be deployed on DigitalOcean and use Cloudflare for caching. We've agreed to have one large component of our app ready by Apr 1st, which our largest client will use while we finish the rest of the work. Our development strategy will be based around delivering this key component, plus the minimum number of features required to support it (accounts, permissions, ...). I am curious if anyone could share their experience building a new system from scratch, and what pitfalls we might try to avoid while focusing on delivering this component. Our data is relational in nature, although there are some areas where we will need to store JSON/markup (we allow users to create custom forms, workflows, and templates). We will likely use MySQL 8. One of our founders wants to have a separate database and deployment per client, because he is concerned about accidentally showing Company 1's data to Company 2. I think this may be overkill, as some of our clients are very small, but also because it adds developer overhead needing to make changes to X databases and X deployments, rather than 1 shared database and 1 shared deployment. (One problem we currently face is that updating/deploying Star is manual and time consuming.) Right now, we average on the order of 100s of concurrent users, while hoping to grow to 1000s of concurrent users soon. We have users across the US & Canada, but are unlikely to expand beyond those markets. Is a shared database and single deployment reasonable for us, or should each client have their own database?

solardev 2y ago
It sounds like you're replacing an old monolith with a new faux-monolith that's actually just a bunch of microservices wrapped together under one umbrella? I don't know what your underlying team makeup is like, but it's a truism that your app architecture often reflects your team/company organization. What works well for one dev is often hard to maintain for a team of devs, or many teams. How has your company changed or grown over time, i.e. do all 4 of you work the same things together now, whatever comes up, or are some of you more specialized into frontend/backend etc.? I second everyone else's opinion of "this is too big to rewrite all at once". Break it up piecemeal and replace one feature/area/service/whatever at a time. Your app sounds way too big to be able to successfully rewrite all at once without breaking edge cases and pissing off customers. Now to venture into IMHO territory... this is my biased opinion as someone who grew up in PHP (including Laravel/Symfony/Drupal/Wordpress/Twig/etc) and eventually moved to fully Javascript clientside apps. I think the MVC paradigm works well for environments where your app can inherently be a full-stack monolith, like a desktop app. But it can slow down work in mixed environments like the web where the frontend world moves much much faster than the backend, and is usually made by entirely different vendors (Google, Apple, Meta, Vercel) than the backend (all the FOSS stuff, plus Redis and VMs, etc.). It makes it so that it's really hard to change any one part of the stack without changing or destroying the rest, which means your frontend can only move as fast as your backend. This might be a situation where decoupling the frontend and backend more might help make it all more modular, i.e. the web UI just calls APIs, as opposed to PHP controlling everything from the server side and rendering changes there. That separation can help you with both a modular rewrite and also future changes. Your backend won't have to be as volatile as the frontend. Basic CRUD stuff hasn't changed all that much and PHP can handle that just fine, but I think it's a bad idea in 2024 to limit yourself upfront to Symfony and HTMX, both because you're tightly coupling your presentation to your data/API layer and because probably by the time you finish the rewrite, the web frontend world will have moved on even more. Both of those are already quite niche compared to the big JS systems, so if your goal is increased velocity, using those systems would make it harder to hire and your codebase harder to maintain. It's also harder to scale with # of users if all your services and rendering for all your customers live in the same VM. And it makes it really hard to shard your services across customers (if you ever did want to move to isolated databases, which is a big if). On the UI performance side, really it all boils down to using more AJAX, which HTMX and PHP can both do, or you can do it manually. But that works better if your frontend is decoupled from your backend, so the server isn't tied up trying to render new HTML chunks while it's also performing database lookups and mutations. The decoupling allows you to more easily manage complex frontend loading/error/caching/optimistic update/network error/etc states, and more cleanly dictate which UI updates can happen entirely in the browser and which to wait for async server data. It also saves server resources since you're not rendering any HTML for the client. Phones and browsers are plenty fast for that these days, and it makes the end-user UI much snappier, usually. Even if the network request is a fixed value (i.e. the network roundtrip time is the same regardless of your frontend framework), being able to do dynamic clientside DOM mutations like that will make it feel faster for your visitors, even if your backend stays exactly the same. It also makes clientside component reuse easier, as well as managing global state and inheritance. I think HTMX is fine for simpler sites but I'm not how far it will do for a more complex app... hard to say without even know what "Star" is (a CMS? a CRM? a spreadsheet? ecommerce? etc.) This isn't an argument to use the big JS frameworks & latest patterns (like Next.js with React Server Components), because in some ways that would just emulate how it already works in PHP (yeah, we've gone full circle lol). It might be a situation, instead, to use something more like the traditional single-page-apps that takes care of the UI clientside but relies on the server APIs for data lookups and mutations. In that way you can more truly decouple the frontend from the data and give your users fast navigation & interaction updates without having to wait for server renders all the time, like you do in PHP components. I think HTMX can do some light componentization with HTML Web Components, but those are way less powerful than the JS equivalents, especially when you have complex state and inheritance. If I were in your shoes -- again, strictly IMHO -- I'd do the rewrite piecemeal, but instead of using a true MVC architecture, just have a bunch of REST endpoints that do simple idempotent operations and then tie them together with something like a backend-for-frontend if needed. That way you don't have to change your backend unless the actual schema changes, you can easily and quickly iterate your frontend to seek the latest optimizations in whatever framework or language is popular this year, and in between, your BFF (or just serverless request proxies) can help smooth over some of the roughness inherent in any such decoupled system. Any one part of the system can be swapped in and out as long as they respect the pureness of their predecessors' inputs and outputs. I think rebuilding the whole thing on a more traditional MVC PHP setup would just result in you needing to rewrite the whole thing yet again in a few more years when it grows a bit more (both users and maybe your team size). The TLDR is that a cleaner separation at each layer helps makes your app more maintainable & modular, because each layer can be properly abstracted to only handle its own concerns and not have to worry about anything above or below it. I don't think a MVC architecture can really do that because there is no simple correspondence between MVC and the actual Web boundaries (VMs, backend logic, caching, hosting, CDN, network retries, frontend logic, frontend UI). But again, just IMHO. Others here would probably disagree entirely.

13 points by poncho_romero 2y ago

solardev 2y ago

It sounds like you're replacing an old monolith with a new faux-monolith that's actually just a bunch of microservices wrapped together under one umbrella? I don't know what your underlying team makeup is like, but it's a truism that your app architecture often reflects your team/company organization. What works well for one dev is often hard to maintain for a team of devs, or many teams. How has your company changed or grown over time, i.e. do all 4 of you work the same things together now, whatever comes up, or are some of you more specialized into frontend/backend etc.? I second everyone else's opinion of "this is too big to rewrite all at once". Break it up piecemeal and replace one feature/area/service/whatever at a time. Your app sounds way too big to be able to successfully rewrite all at once without breaking edge cases and pissing off customers. Now to venture into IMHO territory... this is my biased opinion as someone who grew up in PHP (including Laravel/Symfony/Drupal/Wordpress/Twig/etc) and eventually moved to fully Javascript clientside apps. I think the MVC paradigm works well for environments where your app can inherently be a full-stack monolith, like a desktop app. But it can slow down work in mixed environments like the web where the frontend world moves much much faster than the backend, and is usually made by entirely different vendors (Google, Apple, Meta, Vercel) than the backend (all the FOSS stuff, plus Redis and VMs, etc.). It makes it so that it's really hard to change any one part of the stack without changing or destroying the rest, which means your frontend can only move as fast as your backend. This might be a situation where decoupling the frontend and backend more might help make it all more modular, i.e. the web UI just calls APIs, as opposed to PHP controlling everything from the server side and rendering changes there. That separation can help you with both a modular rewrite and also future changes. Your backend won't have to be as volatile as the frontend. Basic CRUD stuff hasn't changed all that much and PHP can handle that just fine, but I think it's a bad idea in 2024 to limit yourself upfront to Symfony and HTMX, both because you're tightly coupling your presentation to your data/API layer and because probably by the time you finish the rewrite, the web frontend world will have moved on even more. Both of those are already quite niche compared to the big JS systems, so if your goal is increased velocity, using those systems would make it harder to hire and your codebase harder to maintain. It's also harder to scale with # of users if all your services and rendering for all your customers live in the same VM. And it makes it really hard to shard your services across customers (if you ever did want to move to isolated databases, which is a big if). On the UI performance side, really it all boils down to using more AJAX, which HTMX and PHP can both do, or you can do it manually. But that works better if your frontend is decoupled from your backend, so the server isn't tied up trying to render new HTML chunks while it's also performing database lookups and mutations. The decoupling allows you to more easily manage complex frontend loading/error/caching/optimistic update/network error/etc states, and more cleanly dictate which UI updates can happen entirely in the browser and which to wait for async server data. It also saves server resources since you're not rendering any HTML for the client. Phones and browsers are plenty fast for that these days, and it makes the end-user UI much snappier, usually. Even if the network request is a fixed value (i.e. the network roundtrip time is the same regardless of your frontend framework), being able to do dynamic clientside DOM mutations like that will make it feel faster for your visitors, even if your backend stays exactly the same. It also makes clientside component reuse easier, as well as managing global state and inheritance. I think HTMX is fine for simpler sites but I'm not how far it will do for a more complex app... hard to say without even know what "Star" is (a CMS? a CRM? a spreadsheet? ecommerce? etc.) This isn't an argument to use the big JS frameworks & latest patterns (like Next.js with React Server Components), because in some ways that would just emulate how it already works in PHP (yeah, we've gone full circle lol). It might be a situation, instead, to use something more like the traditional single-page-apps that takes care of the UI clientside but relies on the server APIs for data lookups and mutations. In that way you can more truly decouple the frontend from the data and give your users fast navigation & interaction updates without having to wait for server renders all the time, like you do in PHP components. I think HTMX can do some light componentization with HTML Web Components, but those are way less powerful than the JS equivalents, especially when you have complex state and inheritance. If I were in your shoes -- again, strictly IMHO -- I'd do the rewrite piecemeal, but instead of using a true MVC architecture, just have a bunch of REST endpoints that do simple idempotent operations and then tie them together with something like a backend-for-frontend if needed. That way you don't have to change your backend unless the actual schema changes, you can easily and quickly iterate your frontend to seek the latest optimizations in whatever framework or language is popular this year, and in between, your BFF (or just serverless request proxies) can help smooth over some of the roughness inherent in any such decoupled system. Any one part of the system can be swapped in and out as long as they respect the pureness of their predecessors' inputs and outputs. I think rebuilding the whole thing on a more traditional MVC PHP setup would just result in you needing to rewrite the whole thing yet again in a few more years when it grows a bit more (both users and maybe your team size). The TLDR is that a cleaner separation at each layer helps makes your app more maintainable & modular, because each layer can be properly abstracted to only handle its own concerns and not have to worry about anything above or below it. I don't think a MVC architecture can really do that because there is no simple correspondence between MVC and the actual Web boundaries (VMs, backend logic, caching, hosting, CDN, network retries, frontend logic, frontend UI). But again, just IMHO. Others here would probably disagree entirely.