20 ms·

Don’t build a general purpose API to power your own front end (2021)

266 points by aabbcc1241 3y ago

aabbcc1241 3y ago
previous discussion: https://news.ycombinator.com/item?id=28511570 https://news.ycombinator.com/item?id=28511570
- preommr 3y ago
  Quite surprised at how much people agreed with this post. I personally don't think this is such a good idea - it's not that hard to write a general purpose API that covers 80% of obvious use cases, and have app-specific grouping of endpoints for the remainder. And I include things like rollilng multiple requests into one as part of the obvious use case. Because it's usually very easy to implement - either copy/paste, or 5 lines for more complex solutions like an orm, graphql, etc. It seems like a terrible idea to couple the backend with the front-end so strongly.
  - padjo 3y ago
    Seems like an over correction. Yes trying to make your front end API be your public API is probably a bad idea, but making your front end API completely coupled to the structure of your app structure is probably a bad idea too. Best approach is probably in the middle somewhere.
  - isaacremuant 3y ago
    It probably comes from misusing the API route and feeling they want to keep it simple, but instead heavily coupling things. A motivated BE that wants to be productive avoiding unnecessary work would probably hate being "support for frontend" whenever they decide to change something and a motivated frontend would probably want to turn themselves to full stack powers to not depend on someone else. It feels really bad as a pattern unless that webapp is your only concern and is constantly moving or something, or if there's no actually FE or BE but just few people maintaining stuff as a side concern (backend for frontend works there).
tanepiper 3y ago
I read this and got to the end, and laughed because that's what we're working on (more a composition tool from different data sources using a headless CMS and a Knowledge Graph) - so in the end we do need a general purpose API since that's part of the USP.
d_watt 3y ago
I'd caveat that it can be a good idea to make it a general api if you're building a B2B app. These days, purchasing checklists often include "do you have a public api?" Saying no to that can be a sign of product immaturity that is worth trying to get ahead of, and having to maintain internal and external apis is a lot of overhead. It also can be good as you scale to have an API interface you can have a services team learn how to work with to enable "special" flows for customers.
- runeks 3y ago
  There are clearly reasons for a backend to expose an API; no one is disputing that. The article simply points out that it should not be the default.
Y-bar 3y ago
> Imagine if you could just send it the whole “page” worth of JSON. Make an endpoint for /page/a and render the whole JSON for /page/a there. Do this for every page. Don’t force your front-end developers to send a bunch of individual requests to render a complex page. Stop annoying them with contrived limitations. Align yourselves. Why not just send HTML as a single response at this stage? Sometimes it feels like we are doing web development more complex than it needs to be.
- tylercrompton 3y ago
  Something about injecting HTML from an API, even from a controlled server, feels wrong.
  - prmph 3y ago
    But that’s the point: its not an API, it’s just an app web server. There’s nothing wrong with a web server serving HTML; that’s their whole purpose.
- hakunin 3y ago
  The first 3 sentences of the article are implying that the whole thing is written with "if you must" attitude. That said, a lot of companies have front-end teams, and those front-end teams commit years to building React components. So even if you render server-side, this is how you'd put data into those components. I'm a backend dev. I don't like some of what frontend is doing, but you gotta work with people, right?
- shaftoe444 3y ago
  > Imagine if you could just send it the whole “page” Could have stopped here
  - aatd86 3y ago
    Maybe for separation of concerns between data and UI? The browser is not going to be the only client?
sergioisidoro 3y ago
My personal take: Don't put it all in the same box. Just have a api/v1/rest/ base path for modular well defined resources that can be used across your application, without breaking changes, or even shared to third parties. Then have a api/v1/features for coupled endpoints that return non standard objects, and with a more flexible lifecycle. Chances are you're better off having both instead of trying to cram all the use cases into one approach
- antonhag 3y ago
  Most times you won't need a well-formed (REST) API to start with, and many times you will never need it. Most small projects should start without one and create one when need arises.
  - sergioisidoro 3y ago
    Sure, you won't "need" them. But it's very likely that you'll have resources that behave very well for rest like operations - user comes to mind, for things like user profile page, user settings, user detail, etc A rest endpoint is likely handy to use across your application without having to replicate the same serialization over and over again. Also many frameworks remove all the boilerplate for those operations (eg. Django class views), so creating one is a really good starting point.
- indymike 3y ago
  I can't agree more: having a separate API for functionality and a separate one for non-standard (i.e. UI) is a great way to do this.
- nicodjimenez 3y ago
  That’s a nice approach.
lll-o-lll 3y ago
This is good general advice. Sometimes you are building a general library or API, and sometimes there just happens to be a network partition between two components that make one module. My general rule of thumb is that it takes 3-5 times as long to build a library/generic API over regular code. It also requires a very different mindset. Should you do that sometimes? Absolutely! In big companies you probably have entire teams doing this for API’s that are entirely internal. Just recognise that it is a huge cost and only do it when the benefits are likely to outweigh the costs. That’s not to say that you should never think about generalisation when writing normal code, (one of my pet peeves is interfaces taking a single “something” when a collection could just as easily have been used), there are many ways code can evolve and some up-front thought can save a good deal of refactoring later. Just don’t write a generic library when domain specific code will do.
unconed 3y ago
This is entirely backwards. Don't build a general purpose API so your front end can have half the the logic outside. Just build it the way you would as if it were local, and then hook it up to a back end with a git-like synchronization mechanism. You know, the same way you do all your own development. If you don't know how to do that, maybe you could learn that instead of arguing over REST vs GraphQL, or whether your line format should care what boxes are on the page. All I know is, it's fantastic to be able to add features to a front end, persisted in a back end, without having to write code on both sides. "What about validation??" Guess what, if users each have their own workspace, isolated and data driven, you don't need to worry about one bad request ruining your whole service.
- urbandw311er 3y ago
  Could you point me to a sample project/source that does this “git-like synchronisation” you describe? At the moment this proposal feels vague.
rblatz 3y ago
This pattern has a name Backend For Frontend (BFF) https://aws.amazon.com/blogs/mobile/backends-for-frontends-pattern/#:~:text=According%20to%20Sam%20Newman%2C%20the,one%20general%2Dpurpose%20API%20backend https://aws.amazon.com/blogs/mobile/backends-for-frontends-p....
jaza 3y ago
Gotta say I disagree with this idea. RESTful APIs have become the norm for a reason, and in my opinion it's dangerous and damaging to suggest abandoning them for a whole swathe of back-ends. Re: YAGNI. I worked on a product where, about 10 years ago, they decided to build proper APIs (for the first time), mainly to power their own front-end, but they also told the paying clients that they were welcome to use them. Didn't take some clients long to start using and loving those APIs. (Granted, they were enterprise clients, with their own significant dev resources, that's not the case for all software.) Re: front-end doesn't really have freedom. Yes it does! If one day page A needs a list of X, and there's already a nice generic API endpoint to list X, then only a front-end code change is needed.
- weego 3y ago
  No, what we have is everything trending towards a monoculture where every solution is a minor iteration or flavour change to systems that solved problems for companies at scale such as meta/google. We regularly see developers talking about optimising js loops, dom updates etc in products that are little more than blogs with a few interactive elements. Most of us work on things that will get ripped out and redone within a couple of years, yet dev culture tells us we should be focusing on developing for every possible future with reuse and scale in mind, and that's unfortunately false.
  - szundi 3y ago
    At my company most code we did in 20 years ago is there and working. Providing b2b service for 5k customers. Thanks god it’s Java. Our competitors are lost trying to tinker with the old PHP or redoing everything with cutting edge node/js dead end techs.
    - whynotminot 3y ago
      You think node js is dead end tech?
      - halfcat 3y ago
Spiwux 3y ago
Don't think I could disagree any harder. Requirements change. Business needs change. You want your FE to be able to evolve independently without having to wait for the BE to implement the precise logic you need.
- another-dave 3y ago
  As well as your FE team forced to wait on BE changes for new features, you've also created busywork for the BE team anytime there's a FE refactor/restructure to boot.
  - bramblerose 3y ago
    An organisation that separates frontend and backend teams will naturally converge to an architecture where the collaboration between frontend and backend is minimized (i.e., a generic REST API) -- Conway's law. This doesn't mean that an actual cross-functional team that contains both frontend and backend developers can't use the proposed single-purpose API to run circles around the split team.
    - another-dave 3y ago
      Regardless of cross-functional teams, I think having to make changes at an API level at the backend for UI changes will negate a lot of the benefit. For clarity, I don't think you need to treat your own front-end like an external consumer & think it's fine to have bespoke endpoints as needed. What I'm skeptical of is the author's suggestion that a backend endpoint brings back presentational info like "leftBoxTitle".
TekMol 3y ago
The way HN is build is the best way to build web applications. There is a concept of a "page". A page is loaded as an html page. This thread for example is a page. And then there is JavaScript which does all kinds of dynamic stuff on a page. Like when you update this commment or when you collapse/expand comments.
- osrec 3y ago
  I disagree. I understand a lot of people enjoy the simplicity of HN, but when you find a developer who really knows how to take advantage of the additional "bells and whistles" that come with modern browsers, without making them annoying, the experience can be significantly better than HN.
  - laurels-marts 3y ago
    By "bells and whistles" you mean the Web APIs? https://developer.mozilla.org/en-US/docs/Web/API https://developer.mozilla.org/en-US/docs/Web/API
  - TekMol 3y ago
    Sites like Reddit, Twitter, Facebook would be 100x better if they would be built the HN way. If these companies don't have the developers to show that "Bloated React Single Page General API sites" make a better UI then who has?
  - throwsubsun 3y ago
    Can you elaborate on this? I'm wondering what those bells and whistles are, or how the experience could be "significantly better."
    - robertoandred 3y ago
      Like not having to go through two page loads just to add a comment.
- MrPatan 3y ago
  I don't think this applies to bigger apps like an excel or Photoshop. I don't think HN is very "appy", to be honest.
gabrielmrts 3y ago
is the first time that i see that idea, sounds crazy and out of the box imagine a real project using it..
- slrainka 3y ago
  Not so crazy. I can name at least 2 of the top 5 North American airlines running this in production for Mobile and Web supporting hundreds of millions of user requests, daily.
projektfu 3y ago
As a user, this approach often leaves things in a difficult state. The front end works, but automating data access is a hard slog through inconsistent API that's not fully revealed. Reporting is dependent on what is planned for the front end, not what is needed by the user. Because the front and back are tightly coupled, everything is fragile unless it's baked in. I think many business apps should focus on having an API be a first-class citizen, and there needs to be better standardization in how they're organized and used. GraphQL could be part of that solution. It's also nice when the API lets you do the things you can do from the front end as well as more data-oriented tasks. For example, in Zoho Books, reporting isn't automated by the API, you'd have to roll your own or script the UI. The API should cover both reading and writing data items like transactions and downloading a PDF or Excel report using the reporting interface.
caleblloyd 3y ago
The past 2 years have seen some advancements in full-stack frameworks with streaming components. Such as NextJS with React Server Components or .NET Blazor. Those would be my preference for an app that doesn't need a public API. If each partial layout and page component does it's own data loading server side, you can skip GraphQL or making a Full Page API and partial update APIs. Part of the problem with this is organizational though. For the past decade teams have largely been split between frontend/backend who typically work in 2 completely different languages/frameworks. The steaming components approach is a switch back to full stack and a single language/framework. Also every language doesn't have one of these streaming component frameworks, so it's not really possible to do an app completely in Go for example.
smarkov 3y ago
Based on my personal experience general purpose APIs lead to backends for frontends, which is a colossal footgun unless orchestrated by a mastermind. Even if you did have that kind of mastermind, if that person decides to leave you're left with a very expensive puzzle for everyone else to solve. Here's my take: don't build general purpose APIs, build general purpose domain actions. A user making an account is a domain action. Sending a notification to the user is a domain action. Make doing each separately easy. Make doing both at the same time easy. Do this with all of your domain actions and now you have an easy way of building exactly the kind of endpoints all your front ends need.
- tootie 3y ago
  Curious about this take, since I've done it successfully many times. Orchestrating a backend for frontend is usually a pretty trivial task. It's usually just mapping one json to another.
  - smarkov 3y ago
    Here's just some of the things that become non-trivial in such a scenario: * Validation - you either replicate the same rules on the generalized API/BFF/FE and risk inconsistencies or figure out a way to extract that validation to a shared package * Making API calls inside the BFF - usually generalized APIs have a lot of features in individual endpoints, some of which aren't always documented so you'd need a really good documentation that gets constantly updated or some kind of a connector package that makes it easy to discover those features * Testing - most of your endpoints are just calls to another API, all you can do is mock those calls in your tests based on what you expect the API to return but this gets hairy if you're just copy pasting the same 200 line JSON body across multiple tests so you'll need some helpers/a package for that too. It also requires that you always have a person on your team who's very familiar with how the API you're making calls to works so you don't make false assumptions You either need someone who's aware of every little detail that goes on in both the generalized API and the BFF or you need all of these constant abstractions to be able to scale properly in any shape or form without introducing bugs with every new feature.
jgilias 3y ago
The problem is not building a general purpose API, the problem is over thinking it. There’s a sweet spot where you build the API just slightly more flexible than your FE needs at the time, yet without wasting time on making it fit ‘all possible future use cases’. Then you can later on whip up a mobile app if needed, let the clients use it, whatever. And evolve it accordingly.
- runeks 3y ago
  I disagree. Trying to generalize a single implementation into an interface is always problematic. And that's exactly what inserting an API between the FE and BE is if your only UI is a web page. If, later on, you want a mobile app, then an API can easily be deduced from your current, working UI, instead of trying to imagine what the UIs of your web FE and mobile app have in common.
indymike 3y ago
This article is good advice to get a working prototype or a decent solo-dev MVP website that will be re-written later. I've spent the last two months cleaning up the work of a developer who thought this way. Here's what you run into: * Front end and back end become conflated. Often times code that should run on the backend runs on the front end and vice versa. Often shortcuts like saving ui state using the same api endpoint you use to save data become complex, and slow down improvements to both the ui and the public API. Likewise data manipulation and in some cases even validation gets done on the wrong end, too. * You confuse UI code with application functionality. Code used to save the order of columns, stuff you entered the last time a dialog was displayed, and the convenient data structures (i.e. easy, not best) for the front end leak into the back end. So you end up spending a lot of time building back end functionality that has literally no utility to anyone else other than the front end developer. * You lose focus on your other users: developers who integrate and build on your product. Eventually, those developers become in-house people who are doing integrations and implementations for customers, and a messy API will really slow them down. Better advice: separate the concerns and have a separate API UI backend. Build exactly what you need to deal with UI state, and component specific UI settings. Your front end team is a perfect proxy for a third party developer for your public API. Take advantage of that. If they can't use it, or they get paper cuts, then paying customers will have problems, too. You'll also be able to on-board developers faster (easier training) and have better documentation.
- zelphirkalt 3y ago
  Do I understand your advice/recommendation correctly?: frontend <--> UI API, only has what frontend needs <--> backend service with its own API In that case: What design do you suggest for the backend's actual API?
- ozim 3y ago
  For me back end and front end being conflated is a good thing. Because most of the time it ends up one application anyway. If you need generic API you can make separate application that will be API. So if you make domain driven design your front end is separate concern. API for third parties is separate concern. Still in database you might have stuff not used by external parties but then you don’t expose it in your external API. But you can make models on your front end API. Making “generic” solution might seem like saving time and money but as times go by and it turns out you have to fight with generic API to get things done because one is stuck in his ways - it definitely pays off.
anymouse123456 3y ago
I've built both models (and more too) over decades. There are lots of questions one might ask before deciding between these two (or other) approaches: * Where does this team want the boundary between client and server in terms of user experience, server load and substantial UX interaction latency? * Can this frontend team independently plumb a changes all the way through the database? * Are backend teams always available for (or interested in) every minor update? * Are both halves of the application deployed on the same schedule? These are a handful of the first questions I'd ask before trying to push either solution at this point in time, the answers would likely lead to more questions.
deleted 3y ago
[deleted]
davidy123 3y ago
When are we going to get away from all the YAGNI engineering that has sold short the promise of the Web for more rented silos, and get to something more like HATEOAS?
FrustratedMonky 3y ago
Think this is describing other frameworks that do this. Isn't this how ELM works? Or ELM architecture in general? This is kind of how some 'functional' web api's work.
kakapo88 3y ago
I'm finishing a project now, using exactly this design pattern. Sort of backed into it, motivated pretty much by the issues outlined. In addition, I wanted something that is clear and sustainable far into the future. The simplicity of this approach supports that. Quite liberating. Complexity is the enemy. True, one might reasonably propose a number of what-if objections. But for many (most?) projects), imo this is a good approach.
xg15 3y ago
And the htmx-ification trend continues. I guess this is sort of the counter movement now to all the API-First/Data-First/SPA-First evangelism of the last few decades. I don't quite understand why though. Or rather, why those posts always seem to have an intense emotional dislike to anything data-driven. What exactly is wrong with just using React? Or, heaven forbid, plain Javascript, which browsers have spent the last decades optimizing it make working with REST APIs and JSON as easy as possible.
- neon_electro 3y ago
  Can you expand on "just" using React? Are you saying "what is wrong with using React (the framework including React the library and all of its friends)?", or are you saying "what is wrong with just using React (the library) only?"
  - xg15 3y ago
    I was more thinking of React, the framework, with as little addons as possible. React's data model always seemed the most reasonable to me, if you have to use a framework or library at all.
over190bpm 3y ago
My experience working on "medium sized" B2B apps in 10-20 people teams is that the cleanest way to design API-s is to have a regular REST API for the basic CRUD operations on the entities, and then have more specialized endpoints for specific pages, tables, dashboards etc. on top of that. Having specialized endpoints is kind of required, as the data often needs to be searchable, sortable, filterable and paginated, which would be pretty wasteful, or sometimes even impossible to do on the frontend. I find these kind of REST API-s pretty straight forward to design and implement for the most part, and as others pointed out, onboarding is extremely easy if the new developer on the team has worked in software dev at least for a year or two. This also makes it pretty easy and quick to transition to a fully public API if needed (although I have rarely seen this as a requirement). Edit: fixed a typo.
tootie 3y ago
This is BFF pattern described many years ago. Here's the standard way I build this kind of thing: Develop your APIs per functional or really per system. One for content, one for identity, one for ads or whatever. Then you build an orchestration tier that does pretty much what OP says. It stitches the various services together and returns a page model. The ideal is to reduce round-trips and cognitive load on FE dev. If you need the top 3 pieces of relevant content, plus an ad plus the user's first name, return all of that and nothing else. The last piece is just SSR. Use next or nuxt or whatever and you can output HTML on the server or stich it on the client with one codebase seamlessly. https://samnewman.io/patterns/architectural/bff/ https://samnewman.io/patterns/architectural/bff/
hakunin 3y ago
One extra thing I have confirmed after writing this article: it's usually a bad idea to reuse back-end data in multiple places on the front-end. If you think about functions or object constructors in general, it kind of makes sense. One of the most important architectural practices I've ever discovered is: don't pass in the parameters that you have, pass in the parameters that the function needs. So if you have companyName, and you are constructing a page that has a title, you shouldn't pass it companyName: companyName, you should pass it pageTitle: companyName (parameter name: parameter value). It's crucial that the parameter is named after what is needed, not what you have. This is kind of the essence of what the article is suggesting. If your front-end is reusing a lot of fields from back-end, it's likely that you're passing it what you have in your database/resources, and not what it needs — fields for constructing the UI, configuring the views. Moreover, I rarely see front-end going through the exercise of defining exactly what they need. The article encourages more of that. Once the front-end starts reusing the generic back-end fields everywhere, you lose track of your use-cases. Everything you ship from the back-end becomes potentially important for mysterious reasons that not even front-end can easily understand anymore. Front-end already built complex dependency trees based on these generic fields. If you actually untangle these trees, it might turn out that your front-end can only exist in 5 different "modes", but you can no longer see that in the tangled mess. You can no longer streamline and optimize these 5 configurations. So to summarize: pass what is needed, not what you have is a good general architectural rule that's also applicable between back-end and front-end.
- appplication 3y ago
  This is really timely for me, thank you. I’m ignorant when it comes to web dev best practices (data engineering background mostly) but find myself suddenly having created a couple small backends for some of our internal (non-customer facing) tooling, and running into basically all of your questions. One of these I struggled with right away was “how do I ensure the front end and back end have consistent data sourcing and validation”, e.g. for various forms, dropdown selectors, etc. I have found surprisingly few resources for general patterns here. So I did your anti-pattern of having an endpoint for each little thing (e.g. /api/v1/country-codes or something like that). This has felt… ok at best. One challenge, perhaps entirely imagined, is coherent namespacing. Part of the backend is also a public (internally) API. We auto-generate all our docs with swagger, and I was finding that the few meaningful endpoints I had were diluted by the mess of individual endpoints for the frontend. So I threw these under /api/v1/config. Feels like a junk drawer but at least it cleaned things up. I like the idea you’re proposing here for having one API per page, I think it makes a lot of sense. It also eliminates the “I guess I’ll add another endpoint” and changed it to “let me add another field”. Feels more granular and scoped. I will be giving this a try!
ricardo81 3y ago
It's also an ideal vector for scraping your site-specific data if it's an XHR request, typically an enumerating id or something a sitemap will spell out for more sparse IDs.
deleted 3y ago
[deleted]
solatic 3y ago
Hard agree with the YAGNI conclusion. There's a certain minimal amount of complexity needed to build your system, you need a good reason to build out another layer of indirection / abstraction here, something like needing web + mobile for the MVP or offering users direct access to the API as part of the MVP. Remember: if you think there's value in a REST-style layer, you don't need to build a separate API to support that. You can just write a getUsers() method on the server instead. What do you get when you throw out the API layer? Easier refactoring (no wondering who all the consumers of GET /users are). No implicit contract with users who have reverse-engineered your API (and complain if you break it). Easier security (who needs API tokens?). No organizational footgun that leads to a frontend/backend team split and then requires a group manager to get new features shipped, because of course your API doesn't actually expose feature-specific data for features that Product hasn't dreamed up yet. Sure, if you're FAANG scale, it'll be easier to work with APIs. You'll have some Staff+ Engineer architecting everything and coordinating between Product and a half-dozen teams that'll take months if not years to ship, and everyone will be happy with that. Strong fences make good neighbors, and contracts keep teams independent of each other. But in the early days? Feel free to ship things that don't scale.
- kstrauser 3y ago
  I beat the drum of complexity vs complication. Solutions to a given problem have an inherent complexity. That’s the minimum amount of sophistication and cleverness required to implement them. And then there’s complications (think of the watchmaker’s term) which extra features that don’t directly solve the problem but, well, they look kinda neat! Strive to find that inherent, irreducible complexity and solve at that level. Resist the urge to say “for just 20% more effort…” I want to get a YAGNI tattoo.
  - garethrowlands 3y ago
    No, you won’t need that.
- rewmie 3y ago
  I think your comment doesn't really make sense. Unless you're somehow assuming that the one and only way to do web apps is to go the dynamic html path, I don't see how your suggestion can even apply. Once you have a frontend that needs to retrieve data from a backend, how do you even address that? Your comment reads as if someone complains people don't actually need cars because they can just show up somewhere.
hot_gril 3y ago
The complaint about general-purpose APIs is valid. I've seen teammates, including senior ones, jump onto the bandwagon of some monstrous "data service" at work where the entire API to the frontend was an ORM. Absolute trainwreck. This article advocates for another extreme that I've also seen at work. They invented some UI language in JSON and built some default JS components to render it. It works for very simple cases, but it's too rigid for anything slightly advanced, so the end result is some very clunky UIs and unhappy devs. You said your experience was bored frontend devs; that's usually a sign that their value is going to waste, and probably not because the backend devs can do their job better. There's a good middle ground that's pretty common. You don't define the API to end all APIs, you don't pretend FE and BE are totally decoupled, you just have the frontend team ask for what they need and let the backend team serve it. Maybe you get an endpoint per page this way, but it's just sending back the data to populate the page rather than the actual layout. Maybe a few things are separate endpoints, but a reasonably low number.
- hakunin 3y ago
  > This article advocates for another extreme that I've also seen at work. They invented some UI language in JSON This is called server-driven UI, and not what the article is advocating. I try to make that distinction by calling the whole practice server-informed UI, rather than server-driven UI. The article is meant to be advocating almost exactly what you wrote in the 3rd paragraph.
  - hot_gril 3y ago
    Your note says, "A Server Driven UI is when the API tells the client which components to display and with which content." Your description of server-informed UI is, "Render concrete boxes, sections, paragraphs, lists. Render the visual page structure." What's the difference? More concretely, in your example: { "section1": { "topBoxTitle": "Foo", "leftBoxTitle": "Bar", "linkToClose": "https://…" }, "section2": { … } } I was thinking these keys are special and a different page would also have "section1" with "topBoxTitle," but maybe I misunderstood. If these keys aren't special and the frontend dev is just humanly interpreting this differently on each page then defining the actual layout to display it, that's probably fine. But I wouldn't describe this as rendering the visual page structure in the backend.
ChrisMarshallNY 3y ago
I've done this, and agree. But it was a good exercise for me, and I ended up using the general-purpose backend for a specific frontend. It's just that it's way overkill. Much more complex than it needs to be. But it works really, really well, and is secure as hell.
ChicagoDave 3y ago
I’m going to get zapped once again, but if you’re building complex applications, custom read models automatically updated based on events from operational writes is the optimal way to connect a front-end to APIs. Each read model should have a distinct business purpose. Generic models will only confuse maintainers and they’re very likely going to see such code as brittle and tech debt.
qudat 3y ago
Very similar sentiment to this article https://bower.sh/dogma-of-restful-api https://bower.sh/dogma-of-restful-api
ahuth 3y ago
Nice article This is one thing I’m loving about Remix. It’s how the framework works, so there’s no need to enforce this pattern or educate folks about it. AND Remix’s nested routing still has you break down the data loading into a couple different parts (1 per route segment). So it feels like you’re not loading everything in one place, and may be less likely to make people feel like they’re doing something “different” or “wrong”. It’s
agentultra 3y ago
Better, just don’t write an HTTP API for your front end. Use a server side framework. Done. It’s a pain coming into a project that was built the way the article recommends. Endpoints become RPC calls instead of resources serving entities. The hodgepodge of “API” calls becomes difficult to scale horizontally due to implied (and never documented) data dependencies. So you miss out on transport layer caching, entity caching, and a host of other goodies. All because your front end “framework” wants to live in a separate codebase from the server.
- deleted 3y ago
  [deleted]
- hakunin 3y ago
  The "page" becomes the resource for reads. You get great scalability, because you can use your database in the most efficient way possible to provide entire data for the page. You can maintain 100% back-end clarity on what each page depends on (instead of letting front-end request anything they want in any part of the view). I personally prefer server-side frameworks. I also understand when a front-end team prefers to work a certain way, has a lot of knowledge and tooling built-up around a certain way, and is worth accommodating for that reason.
backendanon 3y ago
> This is similar but not quite the same as Server Driven UI1. Perhaps we could call it Server Informed UI. I think we should just move back to server side rendering, JSP and even ASP were great, plain HTML, some JavaScript where desired since we can't seem to get away from JS.
- cpursley 3y ago
  The sweet-spot seems to be LiveView (Phoenix) and it's derivatives.
- 9dev 3y ago
  Oh, right. Tell me how well that would work for Figma, Photoshop, or VS Code?
kburman 3y ago
Have to disagree with this. API is not always for FE. It can be consumed by other service, or a script. Making generic API helps reuse of common API across web pages. For example notification, user profile.
ricardobayes 3y ago
This only works for small projects. Data-heavy frontend usually sends multiple requests to allow async rendering of different parts of a page with different loading times. This also doesn't work if your design changes a lot. AKA a design change will now mean a backend re-design too.
- rrradical 3y ago
  Your first point is addressed in the article. For the second point, my experience is that the backend probably needs some work in that case anyway. If the backend is well designed, then only the presentation API endpoint needs to change. So a BE engineer is doing the tweaking instead of a FE engineer; not a big deal. Less of a big deal if there's no line drawn between FE and BE engineers.
jensneuse 3y ago
An alternative to what's suggested in this post, how about going API first or even API only? Don't build a frontend at all, just build an API. E.g. you can deploy things on fly.io just by using their GraphQL and REST APIs. The advantage of focussing on the apis is that other developers can much easier integrate with your service and embed it into their own applications.
ggregoire 3y ago
I don't know when we started splitting people into "frontend" and "backend" teams (FAANG influence?) and for what reasons, but 10 years ago we hired people with a general background, smart and curious enough to learn SQL, HTML/CSS/JS and whatever language you want to use on the server, able to build some end-to-end CRUD features on their own. And that's like what 99% of the companies need. My point being: when you have the same person building the UI and the API, you avoid a lot of the problems described in this article.
- hakunin 3y ago
  In a way, this article tries to bring the alignment between the frontend and backend closer to the full stack days, while also acknowledging the increased complexity of the front-end, that may be deserving of its own team focused on great UX.
- Phreaker00 3y ago
  For someone who has been in this field for 20 years I remember starting out doing every aspect of webdevelopment myself. Then the bigger and more complex the projects became, the less that made sense. The requirements of the distinct fields have gotten to a point where it's very impractical and hardly worthwhile for one person to master them all. You can't expect someone to translate a design into modern components that support all types of devices, resolutions and contexts to also come up with optimized SQL queries and knowing how to choose between Docker and Ansible for their distributed serverless functions that handle a variety of external endpoints. Ofcourse not every project that uses this approach needs this approach, and there's still a market for the Do-It-All developer. My point is that the webdevelopment world of today isn't that of a decade ago.
  - ggregoire 3y ago
    > You can't expect someone to translate a design into modern components that support all types of devices, resolutions and contexts to also come up with optimized SQL queries and knowing how to choose between Docker and Ansible for their distributed serverless functions that handle a variety of external endpoints. Why not? Obviously you can't expect this from someone fresh out of college/bootcamp, but that's what I do on a daily basis and I'd expect this from anyone with 10+ years of exp.
moomoo11 3y ago
I write all my api code using a business logic “service” layer and a data ops “repository” layer. The endpoints are by feature not entity, although most of the time they overlap. This lets me move fast while keeping it simple to build the UI whether it’s mobile or web.
willsmith72 3y ago
I totally agree with the general idea, and this is generally how I build backends these days. If there are multiple clients, it's usually even ok for each to have a backend which talks to the "master" backend. But I will never get on board with sending things like content or UI related stuff from the backend. I've seen too many times, engineers think "all our forms are the same, let's just have the backend send a list of input field names and types like 'address'". It's almost always a bad abstraction, causes pain on the frontend, and restricts the creativity of engineers and designers.
- robcohen 3y ago
  Do you have the same opinion of headless CMS systems like Strapi?
- hakunin 3y ago
  The article doesn't disagree with your second paragraph. It encourages to treat each "page" as a unique snowflake, including forms in it. I should probably do a better job to discourage consistency and standardization of these json structures. This is addressed in another one of my comments if you search for "my bad".
Toine 3y ago
The "Controller" part of an API conceptually belongs to the "presentation" layer.
algem 3y ago
It sounds almost like the author is advocating for a really dumbed down BFF layer (backend for frontend) but what they are missing is behind the bff layer you would have general purpose apis. And if I interpret the article to the extreme what is the difference between this pattern and how html is currently served. Salesforce works sort of similar to what is described and it’s a nightmare, these restrictive ideals in my experience just aren’t good in practice because they tend to slow down development.
- SlickStef11 3y ago
  For those who are new to BFF's or backend for frontends, This website offers a detailed explanation on the common pattern. https://bff-patterns.com/ https://bff-patterns.com/
jokethrowaway 3y ago
Great article. We used to call this common sense. It's insane we drifted so far from being sensible. I genuinely think 90% of frontend developers should not exist: the web would be better and companies would have better products. I include myself in the frontenders, I do terrible apps which should just be simple static pages with a sprinkle of js - in my defense, I need the money and nobody listens to me, they say this is modern frontend development. For my tech not-savy clients I can deliver awesome and simple solutions without using the dreaded R framework.
tuckerconnelly 3y ago
I see the point of the article, and even on a principled level I might agree with it (e.g., don't overly abstract). A few problems I see though: 1. If you make backend-only abstractions to keep your code even a little bit DRY, you'll have a duplication of code (both the abstraction and the BFF endpoint). I think it'd be better to go all in on the abstraction up-front. 2. This will impair the conceptual integrity of the system and arguably lead to development slowdowns in the long run. To quote Mythical Man Month: “I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.” 3. Your ad-hoc logic will likely be duplicated anyways across pages. 4. A general purpose API makes security features like RBAC much simpler. I think a general-purpose API is a well established abstraction you can rely on for pretty much any project.
bofaGuy 3y ago
This is how you lock yourself into a single front end design. And when you get ready for the next generation of your app or site, now you’re locked in and changing that front end design becomes exponentially harder.
- hakunin 3y ago
  In case of a complete redesign, yes you're right. With a generic API you will have about as much trouble writing the redesign as you did the original design. In practice, a complete redesign is an incredibly rare event, and a good problem to have. Usually design of individual pages and layouts undergoes gradual, incremental changes, driven by real needs. A redesign that requires front-end restructuring to such an extent that the data supplied is no longer sufficient should probably involve backend collaboration. Otherwise, you're tasking your backend team with building a generic site builder, which is much harder than a specific backend for your frontend.
simonbarker87 3y ago
So basically an old style MVC framework style app like RoR, CodeIgniter etc but made with a JSON response the describes the page rather like Shopify's Store Theme 2.0? Sounds great. Now since you have access to the server code (unlike Shopify's system described above) just return HTML and use your JS flavour of choice in the places where you need heavy JS ... like we used to make web apps. The happiest web devs I speak to are those working in the older MVC style, mainly RoR people. While everyone in AWS/Serverless/JS land is burning out and miserable with the house of cards stack we are balancing React on top of.
zianKazi 3y ago
I am not a big fan of building extremely fine grained general purpose APIs for frontends. A lot of the burden of juggling with data and presenting it is shifted to the UI layer. I am also not a big fan of building a composite API for "pages". Now you have a tightly coupled frontend/backend communication. What happens when we split the page or have to change the format of the data? Also the response of such an API can be quite big and nested. Ends up being a dump of data with poor documentation. I would try to find a middle path. Building composite APIs which have a concise purpose for the frontend but are modular enough to be reused across different areas of the application. Also, I feel that most of the problems discussed in the post are amplified when we have different groups working on the frontend and backend. If a single team is responsible for both the backend and frontend, API decisions are localized and can be changed as the development progresses.
theptip 3y ago
Counterpoint: I did this and it worked out great. We ended up wanting to support direct API access from our customers and we were already in a good place to do so. YMMV. An observation I have is that really flexible APIs make it easy for the frontend to make small changes, instead of needing to coordinate with backend changes. On the other hand this can lead to maintenance problems down the road as it’s hard to control exactly what data/state your system can transition to. I view this as one of the best pieces of tech debt to take on in an early-stage startup, basically keeping your options open and minimizing short-term friction, at the expense of taking on a bit more cleanup after you find PMF and start to make your backend more robust for the usecase you are scaling.
- Attummm 3y ago
  Engineering is about tradeoffs, everything has its place. But it would it be good standard? How would we look at that api when: New frontend is needed? Other team/company would like to acces the data? When we other clients for example anderiod/ios app? Wouldn't a generic api, with some custom api calls. Ensure that we are more consistent, and require less maintenance in the future. That way we could focus on adding business value, or solving those hard problems, we normally don't have time for.
codingclaws 3y ago
I read some of the HTMX essays [0] last week due to it being accepted into the GitHub accelerator program. So, a fully RESTful API must output HTML. I'm not going to go into why but that's what Roy Fielding (the creator of REST), Martin Fowler and the HTMX guy(s) say. The HTMX guy(s) also say that of course this fully RESTful API that outputs HTML is sometimes bad because you don't want to have to parse data out of HTML. So, the optimal setup is to have both a fully RESTful HTML API and the more common JSON API. I also think that for the fully RESTful HTML API, it doesn't make sense for the outputted HTML to have any JS. So, the question I'm pondering at the moment is, is a fully RESTful HTML API simply a website that doesn't use any JS? If so, then the optimal setup is to have an SSR website that doesn't use JS and a JSON API. Of course, under the hood, the website and the JSON API use the same lower level API (eg. SQL queries and back-end functions). What's crazy for me personally is that this is exactly how I have set up Comment Castles. There's the no JS website [1] and the JSON API [2]. But I didn't build a no JS website because of REST, I just prefer websites that have no front-end JS. [0] https://htmx.org/essays https://htmx.org/essays [1] https://www.commentcastles.org https://www.commentcastles.org [2] https://www.commentcastles.org/api https://www.commentcastles.org/api
- adamckay 3y ago
  > is a fully RESTful HTML API simply a website that doesn't use any JS? No, I don't think so. Sprinkling a little client side JS can certainly be beneficial and doesn't invalidate it being a RESTful HTML API. _hyperscript [1] was created by the htmx author for that purpose. There's a book [2] that you may not have seen written by some core htmx dev's that goes into this [3] in more detail. [1] - https://hyperscript.org/ https://hyperscript.org/ [2] - https://hypermedia.systems/ https://hypermedia.systems/ [3] - https://hypermedia.systems/client-side-scripting/ https://hypermedia.systems/client-side-scripting/
kgeist 3y ago
General purpose API helped us when we had several features to implement in a row on a tight schedule: when I was busy implementing feature N+1, our frontend dev was still polishing feature N (with changing UI/UX requirements), and she didn't need me (the backender) because the backend just exposed the model as is without knowing what the frontend looked like. She could make lots of changes to the UI without needing any changes to the backend. Yeah it translated to more API calls but it allowed us to decouple our work completely and deliver more features in parallel.
- wruza 3y ago
  Reading this may give the impression that non-api-decoupled things like e.g. ls or curl or inkscape cannot be developed in parallel. Also, think of it this way: if there was no general purpose API, it could be (figuratively) May instead of August when you did N and N+1. I mean, there’s no guarantee that it would, but this potential benefit is relative to a different baseline than in TFA.
rhuru 3y ago
I have building most of my modern sites with NextJS and could not have been happier as it gives me the flexibility of directly quering DB, I dont have to worry about the general purpose API. But if I wanted to I can just easily build it too.
yawnxyz 3y ago
doesn't this require you to nail down design requirements? If the designer gives a few different comps to try out, or wants to A/B test something, they'd have to align those changes with both backend AND frontend people to make sure they both know what the changes are. If the data dictates exactly what the frontend should look like / placement of content, shouldn't the backend just send styling / html / app code directly, and just skip frontend completely?
hankchinaski 3y ago
this is fine but makes the backend extremely slow and hard to maintain. it's effectively just glorified RPC - restful api low logic on backend and lots of logic on front-end - grpc style as the author suggests, more logic on backend no logic on front-end - keep api restful on backend, add backend for front-end(graphql/rpc), front-end remains simple, most logic is in the middle. this also allows you to isolate presentation spaghetti code in one place, keeps your backend api clean for use for other clients worked in all 3 probably having a custom presentation layer works best but require more setup upfront. another advantage of having a restful api and have the front-end do more work is being able to incrementally load data into the UI therefore reducing TTI/TTFB
gls2ro 3y ago
It might be that for the author specific case (which we dont know in details) maybe this is a good approach. In general I disagree with almost any idea that tries to fix some problems by adding coupling between components. I think the main thesis is where there is a false assumption: > Your frontend can finally focus on presentation and UI. Your backend can finally focus on implementing exactly what’s needed. If you make backend create one API call for each page, you will NOT get FE focusing on UI and backend on something else. You will get FE focusing on UI and BE focusing on UI and something else. I do get the feeling of rebellion against common wisdom that is FE and BE as separate components. When I was junior programmer (not suggesting here that author is junior, just talking about my experience) I often found myself rebelling against common programming wisdom. Now, after more than a decade of seeing so many changes and request I think those old programmers that gave us that old wisdom about separating concerns and losse coupling are right. I could boil down probably all old advice into the following 3 small pieces: 1. Write code. Not too much. Mostly simple, short and loosely coupled. 2. Go against common wisdom when you understand the tradeoffs. When not sure, pick the common wisdom. 3. You can maybe fix technical issues with more people, but can never fix organizational issues with code.
- hakunin 3y ago
  Author here. Here's an additional piece of wisdom that I picked up in over 20 years of doing this. Reliable communication allows for simpler, less coupled interfaces. Unreliable communication demands increasingly elaborate and coupled interfaces. I wrote a short note explaining this principle. https://notes.max.engineer/reliable-communication-allows-for-simpler-interfaces https://notes.max.engineer/reliable-communication-allows-for... > If you make backend create one API call for each page, you will NOT get FE focusing on UI and backend on something else. You will get FE focusing on UI and BE focusing on UI and something else. You need your backend to focus on something, otherwise it would be focused on nothing/everything. You need to tie yourself down to use cases, otherwise every use case is your use case.
  - gls2ro 3y ago
    First let me again say sorry if my reply made you think I assume you are young. That was not my intention. Second > You need your backend to focus on something, otherwise it would be focused on nothing/everything. You need to tie yourself down to use cases, otherwise every use case is your use case. What I was saying is that your backend will now do what was doing before AND also will need to handle UI elements and relation between them. So you have duplicate data architecture/knowledge in UI: - once in FE as it needs to know the data architecture - again in BE where it also needs to know what data structure UI needs. Thus it couples FE and BE together around key names, JSON structure and moreso on information architecture from UI concepts. I am not saying you should not do that. Maybe it works for you. I would not do it. For me it will be like saying in a traditional simple HTML app that I should name tables and columns in DB as they are in the UI. Maybe some might coincide but that will not be an architectural constraint when thinking the DB.
RamblingCTO 3y ago
If you do proper hexagonal architecture it wouldn't actually matter and you can replace REST with graphql or feature-based endpoints. Before you rip that out, start with that. (YMMV and don't apply every piece of advice to every problem, that's the biggest problem)
eureka-belief 3y ago
What’s missing from this article is a discussion about 1) team composition and software development lifecycle, and 2) which libraries are available to help easily provide and consume the obvious RESTful operations. 1. Team composition If you have a separate backend developer from front-end developer (or worse, on a completely separate dev team), then making a separate endpoint for each page means you have to manage the feature dependency. As in, you need to make a ticket for the backend work first and the front end work can only start once the backend work is finished. And you need to do this for every single new page you make. On the other hand, if the backend team in advance just makes a generic rest API that provides all access to all of the obvious resources, it gives the front-end team the freedom to develop on their own timeline. If all your teams devs are full-stack, this is simplified a bit but still you have the problem of deciding to intelligently architect your data access layer to not have massive amounts of duplicated code. 2. Supporting libraries Often it’s possible on the backend to just easily make restful crud for every single resource that makes sense. A single backend dev can do this in one go without much conversation besides the initial one about access policies. Similarly, the are often front-end libraries that simplify not only writing getters for these endpoints but also for things like handling local caching, business object creation from JSON, and refetch logic. Obviously the libraries that are available and how well they integrate with your workflow will depend on your front-end framework. React, for example, has libraries that directly connect your rest access into specific components. — A caveat: obviously the pro-con landscape changes if you have serverside operations that either: A) feature some interaction that is sensitive to race conditions, B) requires a guarantee that one data type not get written unless the a previous write succeeded, or C) would require special queries for performance reasons In these cases, I’d say make a custom endpoint to encapsulate the correct/safe behavior. But these one-offs can be created only when they are needed. No premature optimization/complexity is required.
- hakunin 3y ago
  > you need to make a ticket for the backend work first and the front end work can only start once the backend work is finished In practice, a lot of this can be done in parallel. Once the design is established, front-end can start working on the layout/UX without knowing the exact shape of supplied data. The back-end can start proposing the shape of data and populating writing SQL/other queries to populate it. The final sync will only require small adjustments. > it gives the front-end team the freedom to develop on their own timeline At the cost of backend having to support every single use case under the sun. > still you have the problem of deciding to intelligently architect your data access layer to not have massive amounts of duplicated code. All this involves, is passing the pages the data they need for their construction. The alternative is building and supporting a generic data access for infinite use cases that is hard to optimize. I see this suggested often, and I don't think it's true that the way to reduce duplication is by providing all data forever. You can provide specific entities wrapped in pages, and reuse the backend logic used for their construction. > it’s possible on the backend to just easily make restful crud for every single resource This shifts all the complexity of combining data onto the front-end, and the front-end is a lot more distant from the data storage. See the note where I describe that "Reliable communication allows for simpler interfaces"[1]. [1]: https://notes.max.engineer/reliable-communication-allows-for-simpler-interfaces https://notes.max.engineer/reliable-communication-allows-for...
MattyRad 3y ago
The `Accept` HTTP header is there to prove your frontend is using "dumb" data. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Ac... I think the author is generally correct that all data should be provided in a single request. And to take it a step further, you should be able to change your accept header to JSON to serve an API. API/dumb-frontend aren't mutually exclusive. In fact, this is what both GitLab and GitHub do. Try it out! `curl -L https://github.com/simonw/shot-scraper https://github.com/simonw/shot-scraper` (defaults to text/html) `curl --header "Accept: application/json" -L https://github.com/simonw/shot-scraper https://github.com/simonw/shot-scraper`

266 points by aabbcc1241 3y ago

aabbcc1241 3y ago

previous discussion: https://news.ycombinator.com/item?id=28511570 https://news.ycombinator.com/item?id=28511570

preommr 3y ago
Quite surprised at how much people agreed with this post. I personally don't think this is such a good idea - it's not that hard to write a general purpose API that covers 80% of obvious use cases, and have app-specific grouping of endpoints for the remainder. And I include things like rollilng multiple requests into one as part of the obvious use case. Because it's usually very easy to implement - either copy/paste, or 5 lines for more complex solutions like an orm, graphql, etc. It seems like a terrible idea to couple the backend with the front-end so strongly.
- padjo 3y ago
  Seems like an over correction. Yes trying to make your front end API be your public API is probably a bad idea, but making your front end API completely coupled to the structure of your app structure is probably a bad idea too. Best approach is probably in the middle somewhere.
- isaacremuant 3y ago
  It probably comes from misusing the API route and feeling they want to keep it simple, but instead heavily coupling things. A motivated BE that wants to be productive avoiding unnecessary work would probably hate being "support for frontend" whenever they decide to change something and a motivated frontend would probably want to turn themselves to full stack powers to not depend on someone else. It feels really bad as a pattern unless that webapp is your only concern and is constantly moving or something, or if there's no actually FE or BE but just few people maintaining stuff as a side concern (backend for frontend works there).

tanepiper 3y ago

I read this and got to the end, and laughed because that's what we're working on (more a composition tool from different data sources using a headless CMS and a Knowledge Graph) - so in the end we do need a general purpose API since that's part of the USP.

d_watt 3y ago

I'd caveat that it can be a good idea to make it a general api if you're building a B2B app. These days, purchasing checklists often include "do you have a public api?" Saying no to that can be a sign of product immaturity that is worth trying to get ahead of, and having to maintain internal and external apis is a lot of overhead. It also can be good as you scale to have an API interface you can have a services team learn how to work with to enable "special" flows for customers.

runeks 3y ago
There are clearly reasons for a backend to expose an API; no one is disputing that. The article simply points out that it should not be the default.

Y-bar 3y ago

> Imagine if you could just send it the whole “page” worth of JSON. Make an endpoint for /page/a and render the whole JSON for /page/a there. Do this for every page. Don’t force your front-end developers to send a bunch of individual requests to render a complex page. Stop annoying them with contrived limitations. Align yourselves. Why not just send HTML as a single response at this stage? Sometimes it feels like we are doing web development more complex than it needs to be.

tylercrompton 3y ago
Something about injecting HTML from an API, even from a controlled server, feels wrong.
- prmph 3y ago
  But that’s the point: its not an API, it’s just an app web server. There’s nothing wrong with a web server serving HTML; that’s their whole purpose.
hakunin 3y ago
The first 3 sentences of the article are implying that the whole thing is written with "if you must" attitude. That said, a lot of companies have front-end teams, and those front-end teams commit years to building React components. So even if you render server-side, this is how you'd put data into those components. I'm a backend dev. I don't like some of what frontend is doing, but you gotta work with people, right?
shaftoe444 3y ago
> Imagine if you could just send it the whole “page” Could have stopped here
- aatd86 3y ago
  Maybe for separation of concerns between data and UI? The browser is not going to be the only client?

sergioisidoro 3y ago

My personal take: Don't put it all in the same box. Just have a api/v1/rest/ base path for modular well defined resources that can be used across your application, without breaking changes, or even shared to third parties. Then have a api/v1/features for coupled endpoints that return non standard objects, and with a more flexible lifecycle. Chances are you're better off having both instead of trying to cram all the use cases into one approach

antonhag 3y ago
Most times you won't need a well-formed (REST) API to start with, and many times you will never need it. Most small projects should start without one and create one when need arises.
- sergioisidoro 3y ago
  Sure, you won't "need" them. But it's very likely that you'll have resources that behave very well for rest like operations - user comes to mind, for things like user profile page, user settings, user detail, etc A rest endpoint is likely handy to use across your application without having to replicate the same serialization over and over again. Also many frameworks remove all the boilerplate for those operations (eg. Django class views), so creating one is a really good starting point.
indymike 3y ago
I can't agree more: having a separate API for functionality and a separate one for non-standard (i.e. UI) is a great way to do this.
nicodjimenez 3y ago
That’s a nice approach.

lll-o-lll 3y ago

This is good general advice. Sometimes you are building a general library or API, and sometimes there just happens to be a network partition between two components that make one module. My general rule of thumb is that it takes 3-5 times as long to build a library/generic API over regular code. It also requires a very different mindset. Should you do that sometimes? Absolutely! In big companies you probably have entire teams doing this for API’s that are entirely internal. Just recognise that it is a huge cost and only do it when the benefits are likely to outweigh the costs. That’s not to say that you should never think about generalisation when writing normal code, (one of my pet peeves is interfaces taking a single “something” when a collection could just as easily have been used), there are many ways code can evolve and some up-front thought can save a good deal of refactoring later. Just don’t write a generic library when domain specific code will do.

unconed 3y ago

This is entirely backwards. Don't build a general purpose API so your front end can have half the the logic outside. Just build it the way you would as if it were local, and then hook it up to a back end with a git-like synchronization mechanism. You know, the same way you do all your own development. If you don't know how to do that, maybe you could learn that instead of arguing over REST vs GraphQL, or whether your line format should care what boxes are on the page. All I know is, it's fantastic to be able to add features to a front end, persisted in a back end, without having to write code on both sides. "What about validation??" Guess what, if users each have their own workspace, isolated and data driven, you don't need to worry about one bad request ruining your whole service.

urbandw311er 3y ago
Could you point me to a sample project/source that does this “git-like synchronisation” you describe? At the moment this proposal feels vague.

rblatz 3y ago

This pattern has a name Backend For Frontend (BFF) https://aws.amazon.com/blogs/mobile/backends-for-frontends-pattern/#:~:text=According%20to%20Sam%20Newman%2C%20the,one%20general%2Dpurpose%20API%20backend https://aws.amazon.com/blogs/mobile/backends-for-frontends-p....

jaza 3y ago

Gotta say I disagree with this idea. RESTful APIs have become the norm for a reason, and in my opinion it's dangerous and damaging to suggest abandoning them for a whole swathe of back-ends. Re: YAGNI. I worked on a product where, about 10 years ago, they decided to build proper APIs (for the first time), mainly to power their own front-end, but they also told the paying clients that they were welcome to use them. Didn't take some clients long to start using and loving those APIs. (Granted, they were enterprise clients, with their own significant dev resources, that's not the case for all software.) Re: front-end doesn't really have freedom. Yes it does! If one day page A needs a list of X, and there's already a nice generic API endpoint to list X, then only a front-end code change is needed.

weego 3y ago
No, what we have is everything trending towards a monoculture where every solution is a minor iteration or flavour change to systems that solved problems for companies at scale such as meta/google. We regularly see developers talking about optimising js loops, dom updates etc in products that are little more than blogs with a few interactive elements. Most of us work on things that will get ripped out and redone within a couple of years, yet dev culture tells us we should be focusing on developing for every possible future with reuse and scale in mind, and that's unfortunately false.
- szundi 3y ago
  At my company most code we did in 20 years ago is there and working. Providing b2b service for 5k customers. Thanks god it’s Java. Our competitors are lost trying to tinker with the old PHP or redoing everything with cutting edge node/js dead end techs.
  - whynotminot 3y ago
    You think node js is dead end tech?
    - halfcat 3y ago

Spiwux 3y ago

Don't think I could disagree any harder. Requirements change. Business needs change. You want your FE to be able to evolve independently without having to wait for the BE to implement the precise logic you need.

another-dave 3y ago
As well as your FE team forced to wait on BE changes for new features, you've also created busywork for the BE team anytime there's a FE refactor/restructure to boot.
- bramblerose 3y ago
  An organisation that separates frontend and backend teams will naturally converge to an architecture where the collaboration between frontend and backend is minimized (i.e., a generic REST API) -- Conway's law. This doesn't mean that an actual cross-functional team that contains both frontend and backend developers can't use the proposed single-purpose API to run circles around the split team.
  - another-dave 3y ago
    Regardless of cross-functional teams, I think having to make changes at an API level at the backend for UI changes will negate a lot of the benefit. For clarity, I don't think you need to treat your own front-end like an external consumer & think it's fine to have bespoke endpoints as needed. What I'm skeptical of is the author's suggestion that a backend endpoint brings back presentational info like "leftBoxTitle".

TekMol 3y ago

The way HN is build is the best way to build web applications. There is a concept of a "page". A page is loaded as an html page. This thread for example is a page. And then there is JavaScript which does all kinds of dynamic stuff on a page. Like when you update this commment or when you collapse/expand comments.

osrec 3y ago
I disagree. I understand a lot of people enjoy the simplicity of HN, but when you find a developer who really knows how to take advantage of the additional "bells and whistles" that come with modern browsers, without making them annoying, the experience can be significantly better than HN.
- laurels-marts 3y ago
  By "bells and whistles" you mean the Web APIs? https://developer.mozilla.org/en-US/docs/Web/API https://developer.mozilla.org/en-US/docs/Web/API
- TekMol 3y ago
  Sites like Reddit, Twitter, Facebook would be 100x better if they would be built the HN way. If these companies don't have the developers to show that "Bloated React Single Page General API sites" make a better UI then who has?
- throwsubsun 3y ago
  Can you elaborate on this? I'm wondering what those bells and whistles are, or how the experience could be "significantly better."
  - robertoandred 3y ago
    Like not having to go through two page loads just to add a comment.
MrPatan 3y ago
I don't think this applies to bigger apps like an excel or Photoshop. I don't think HN is very "appy", to be honest.

gabrielmrts 3y ago

is the first time that i see that idea, sounds crazy and out of the box imagine a real project using it..

slrainka 3y ago
Not so crazy. I can name at least 2 of the top 5 North American airlines running this in production for Mobile and Web supporting hundreds of millions of user requests, daily.

projektfu 3y ago

As a user, this approach often leaves things in a difficult state. The front end works, but automating data access is a hard slog through inconsistent API that's not fully revealed. Reporting is dependent on what is planned for the front end, not what is needed by the user. Because the front and back are tightly coupled, everything is fragile unless it's baked in. I think many business apps should focus on having an API be a first-class citizen, and there needs to be better standardization in how they're organized and used. GraphQL could be part of that solution. It's also nice when the API lets you do the things you can do from the front end as well as more data-oriented tasks. For example, in Zoho Books, reporting isn't automated by the API, you'd have to roll your own or script the UI. The API should cover both reading and writing data items like transactions and downloading a PDF or Excel report using the reporting interface.

caleblloyd 3y ago

The past 2 years have seen some advancements in full-stack frameworks with streaming components. Such as NextJS with React Server Components or .NET Blazor. Those would be my preference for an app that doesn't need a public API. If each partial layout and page component does it's own data loading server side, you can skip GraphQL or making a Full Page API and partial update APIs. Part of the problem with this is organizational though. For the past decade teams have largely been split between frontend/backend who typically work in 2 completely different languages/frameworks. The steaming components approach is a switch back to full stack and a single language/framework. Also every language doesn't have one of these streaming component frameworks, so it's not really possible to do an app completely in Go for example.

smarkov 3y ago

Based on my personal experience general purpose APIs lead to backends for frontends, which is a colossal footgun unless orchestrated by a mastermind. Even if you did have that kind of mastermind, if that person decides to leave you're left with a very expensive puzzle for everyone else to solve. Here's my take: don't build general purpose APIs, build general purpose domain actions. A user making an account is a domain action. Sending a notification to the user is a domain action. Make doing each separately easy. Make doing both at the same time easy. Do this with all of your domain actions and now you have an easy way of building exactly the kind of endpoints all your front ends need.

tootie 3y ago
Curious about this take, since I've done it successfully many times. Orchestrating a backend for frontend is usually a pretty trivial task. It's usually just mapping one json to another.
- smarkov 3y ago
  Here's just some of the things that become non-trivial in such a scenario: * Validation - you either replicate the same rules on the generalized API/BFF/FE and risk inconsistencies or figure out a way to extract that validation to a shared package * Making API calls inside the BFF - usually generalized APIs have a lot of features in individual endpoints, some of which aren't always documented so you'd need a really good documentation that gets constantly updated or some kind of a connector package that makes it easy to discover those features * Testing - most of your endpoints are just calls to another API, all you can do is mock those calls in your tests based on what you expect the API to return but this gets hairy if you're just copy pasting the same 200 line JSON body across multiple tests so you'll need some helpers/a package for that too. It also requires that you always have a person on your team who's very familiar with how the API you're making calls to works so you don't make false assumptions You either need someone who's aware of every little detail that goes on in both the generalized API and the BFF or you need all of these constant abstractions to be able to scale properly in any shape or form without introducing bugs with every new feature.

jgilias 3y ago

The problem is not building a general purpose API, the problem is over thinking it. There’s a sweet spot where you build the API just slightly more flexible than your FE needs at the time, yet without wasting time on making it fit ‘all possible future use cases’. Then you can later on whip up a mobile app if needed, let the clients use it, whatever. And evolve it accordingly.

runeks 3y ago
I disagree. Trying to generalize a single implementation into an interface is always problematic. And that's exactly what inserting an API between the FE and BE is if your only UI is a web page. If, later on, you want a mobile app, then an API can easily be deduced from your current, working UI, instead of trying to imagine what the UIs of your web FE and mobile app have in common.

indymike 3y ago

This article is good advice to get a working prototype or a decent solo-dev MVP website that will be re-written later. I've spent the last two months cleaning up the work of a developer who thought this way. Here's what you run into: * Front end and back end become conflated. Often times code that should run on the backend runs on the front end and vice versa. Often shortcuts like saving ui state using the same api endpoint you use to save data become complex, and slow down improvements to both the ui and the public API. Likewise data manipulation and in some cases even validation gets done on the wrong end, too. * You confuse UI code with application functionality. Code used to save the order of columns, stuff you entered the last time a dialog was displayed, and the convenient data structures (i.e. easy, not best) for the front end leak into the back end. So you end up spending a lot of time building back end functionality that has literally no utility to anyone else other than the front end developer. * You lose focus on your other users: developers who integrate and build on your product. Eventually, those developers become in-house people who are doing integrations and implementations for customers, and a messy API will really slow them down. Better advice: separate the concerns and have a separate API UI backend. Build exactly what you need to deal with UI state, and component specific UI settings. Your front end team is a perfect proxy for a third party developer for your public API. Take advantage of that. If they can't use it, or they get paper cuts, then paying customers will have problems, too. You'll also be able to on-board developers faster (easier training) and have better documentation.

zelphirkalt 3y ago
Do I understand your advice/recommendation correctly?: frontend <--> UI API, only has what frontend needs <--> backend service with its own API In that case: What design do you suggest for the backend's actual API?
ozim 3y ago
For me back end and front end being conflated is a good thing. Because most of the time it ends up one application anyway. If you need generic API you can make separate application that will be API. So if you make domain driven design your front end is separate concern. API for third parties is separate concern. Still in database you might have stuff not used by external parties but then you don’t expose it in your external API. But you can make models on your front end API. Making “generic” solution might seem like saving time and money but as times go by and it turns out you have to fight with generic API to get things done because one is stuck in his ways - it definitely pays off.

anymouse123456 3y ago

I've built both models (and more too) over decades. There are lots of questions one might ask before deciding between these two (or other) approaches: * Where does this team want the boundary between client and server in terms of user experience, server load and substantial UX interaction latency? * Can this frontend team independently plumb a changes all the way through the database? * Are backend teams always available for (or interested in) every minor update? * Are both halves of the application deployed on the same schedule? These are a handful of the first questions I'd ask before trying to push either solution at this point in time, the answers would likely lead to more questions.

deleted 3y ago

[deleted]

davidy123 3y ago

When are we going to get away from all the YAGNI engineering that has sold short the promise of the Web for more rented silos, and get to something more like HATEOAS?

FrustratedMonky 3y ago

Think this is describing other frameworks that do this. Isn't this how ELM works? Or ELM architecture in general? This is kind of how some 'functional' web api's work.

kakapo88 3y ago

I'm finishing a project now, using exactly this design pattern. Sort of backed into it, motivated pretty much by the issues outlined. In addition, I wanted something that is clear and sustainable far into the future. The simplicity of this approach supports that. Quite liberating. Complexity is the enemy. True, one might reasonably propose a number of what-if objections. But for many (most?) projects), imo this is a good approach.

xg15 3y ago

And the htmx-ification trend continues. I guess this is sort of the counter movement now to all the API-First/Data-First/SPA-First evangelism of the last few decades. I don't quite understand why though. Or rather, why those posts always seem to have an intense emotional dislike to anything data-driven. What exactly is wrong with just using React? Or, heaven forbid, plain Javascript, which browsers have spent the last decades optimizing it make working with REST APIs and JSON as easy as possible.

neon_electro 3y ago
Can you expand on "just" using React? Are you saying "what is wrong with using React (the framework including React the library and all of its friends)?", or are you saying "what is wrong with just using React (the library) only?"
- xg15 3y ago
  I was more thinking of React, the framework, with as little addons as possible. React's data model always seemed the most reasonable to me, if you have to use a framework or library at all.

over190bpm 3y ago

My experience working on "medium sized" B2B apps in 10-20 people teams is that the cleanest way to design API-s is to have a regular REST API for the basic CRUD operations on the entities, and then have more specialized endpoints for specific pages, tables, dashboards etc. on top of that. Having specialized endpoints is kind of required, as the data often needs to be searchable, sortable, filterable and paginated, which would be pretty wasteful, or sometimes even impossible to do on the frontend. I find these kind of REST API-s pretty straight forward to design and implement for the most part, and as others pointed out, onboarding is extremely easy if the new developer on the team has worked in software dev at least for a year or two. This also makes it pretty easy and quick to transition to a fully public API if needed (although I have rarely seen this as a requirement). Edit: fixed a typo.

tootie 3y ago

This is BFF pattern described many years ago. Here's the standard way I build this kind of thing: Develop your APIs per functional or really per system. One for content, one for identity, one for ads or whatever. Then you build an orchestration tier that does pretty much what OP says. It stitches the various services together and returns a page model. The ideal is to reduce round-trips and cognitive load on FE dev. If you need the top 3 pieces of relevant content, plus an ad plus the user's first name, return all of that and nothing else. The last piece is just SSR. Use next or nuxt or whatever and you can output HTML on the server or stich it on the client with one codebase seamlessly. https://samnewman.io/patterns/architectural/bff/ https://samnewman.io/patterns/architectural/bff/

hakunin 3y ago

One extra thing I have confirmed after writing this article: it's usually a bad idea to reuse back-end data in multiple places on the front-end. If you think about functions or object constructors in general, it kind of makes sense. One of the most important architectural practices I've ever discovered is: don't pass in the parameters that you have, pass in the parameters that the function needs. So if you have companyName, and you are constructing a page that has a title, you shouldn't pass it companyName: companyName, you should pass it pageTitle: companyName (parameter name: parameter value). It's crucial that the parameter is named after what is needed, not what you have. This is kind of the essence of what the article is suggesting. If your front-end is reusing a lot of fields from back-end, it's likely that you're passing it what you have in your database/resources, and not what it needs — fields for constructing the UI, configuring the views. Moreover, I rarely see front-end going through the exercise of defining exactly what they need. The article encourages more of that. Once the front-end starts reusing the generic back-end fields everywhere, you lose track of your use-cases. Everything you ship from the back-end becomes potentially important for mysterious reasons that not even front-end can easily understand anymore. Front-end already built complex dependency trees based on these generic fields. If you actually untangle these trees, it might turn out that your front-end can only exist in 5 different "modes", but you can no longer see that in the tangled mess. You can no longer streamline and optimize these 5 configurations. So to summarize: pass what is needed, not what you have is a good general architectural rule that's also applicable between back-end and front-end.

appplication 3y ago
This is really timely for me, thank you. I’m ignorant when it comes to web dev best practices (data engineering background mostly) but find myself suddenly having created a couple small backends for some of our internal (non-customer facing) tooling, and running into basically all of your questions. One of these I struggled with right away was “how do I ensure the front end and back end have consistent data sourcing and validation”, e.g. for various forms, dropdown selectors, etc. I have found surprisingly few resources for general patterns here. So I did your anti-pattern of having an endpoint for each little thing (e.g. /api/v1/country-codes or something like that). This has felt… ok at best. One challenge, perhaps entirely imagined, is coherent namespacing. Part of the backend is also a public (internally) API. We auto-generate all our docs with swagger, and I was finding that the few meaningful endpoints I had were diluted by the mess of individual endpoints for the frontend. So I threw these under /api/v1/config. Feels like a junk drawer but at least it cleaned things up. I like the idea you’re proposing here for having one API per page, I think it makes a lot of sense. It also eliminates the “I guess I’ll add another endpoint” and changed it to “let me add another field”. Feels more granular and scoped. I will be giving this a try!

ricardo81 3y ago

It's also an ideal vector for scraping your site-specific data if it's an XHR request, typically an enumerating id or something a sitemap will spell out for more sparse IDs.

deleted 3y ago

[deleted]

solatic 3y ago

Hard agree with the YAGNI conclusion. There's a certain minimal amount of complexity needed to build your system, you need a good reason to build out another layer of indirection / abstraction here, something like needing web + mobile for the MVP or offering users direct access to the API as part of the MVP. Remember: if you think there's value in a REST-style layer, you don't need to build a separate API to support that. You can just write a getUsers() method on the server instead. What do you get when you throw out the API layer? Easier refactoring (no wondering who all the consumers of GET /users are). No implicit contract with users who have reverse-engineered your API (and complain if you break it). Easier security (who needs API tokens?). No organizational footgun that leads to a frontend/backend team split and then requires a group manager to get new features shipped, because of course your API doesn't actually expose feature-specific data for features that Product hasn't dreamed up yet. Sure, if you're FAANG scale, it'll be easier to work with APIs. You'll have some Staff+ Engineer architecting everything and coordinating between Product and a half-dozen teams that'll take months if not years to ship, and everyone will be happy with that. Strong fences make good neighbors, and contracts keep teams independent of each other. But in the early days? Feel free to ship things that don't scale.

kstrauser 3y ago
I beat the drum of complexity vs complication. Solutions to a given problem have an inherent complexity. That’s the minimum amount of sophistication and cleverness required to implement them. And then there’s complications (think of the watchmaker’s term) which extra features that don’t directly solve the problem but, well, they look kinda neat! Strive to find that inherent, irreducible complexity and solve at that level. Resist the urge to say “for just 20% more effort…” I want to get a YAGNI tattoo.
- garethrowlands 3y ago
  No, you won’t need that.
rewmie 3y ago
I think your comment doesn't really make sense. Unless you're somehow assuming that the one and only way to do web apps is to go the dynamic html path, I don't see how your suggestion can even apply. Once you have a frontend that needs to retrieve data from a backend, how do you even address that? Your comment reads as if someone complains people don't actually need cars because they can just show up somewhere.

hot_gril 3y ago

The complaint about general-purpose APIs is valid. I've seen teammates, including senior ones, jump onto the bandwagon of some monstrous "data service" at work where the entire API to the frontend was an ORM. Absolute trainwreck. This article advocates for another extreme that I've also seen at work. They invented some UI language in JSON and built some default JS components to render it. It works for very simple cases, but it's too rigid for anything slightly advanced, so the end result is some very clunky UIs and unhappy devs. You said your experience was bored frontend devs; that's usually a sign that their value is going to waste, and probably not because the backend devs can do their job better. There's a good middle ground that's pretty common. You don't define the API to end all APIs, you don't pretend FE and BE are totally decoupled, you just have the frontend team ask for what they need and let the backend team serve it. Maybe you get an endpoint per page this way, but it's just sending back the data to populate the page rather than the actual layout. Maybe a few things are separate endpoints, but a reasonably low number.

hakunin 3y ago
> This article advocates for another extreme that I've also seen at work. They invented some UI language in JSON This is called server-driven UI, and not what the article is advocating. I try to make that distinction by calling the whole practice server-informed UI, rather than server-driven UI. The article is meant to be advocating almost exactly what you wrote in the 3rd paragraph.
- hot_gril 3y ago
  Your note says, "A Server Driven UI is when the API tells the client which components to display and with which content." Your description of server-informed UI is, "Render concrete boxes, sections, paragraphs, lists. Render the visual page structure." What's the difference? More concretely, in your example: { "section1": { "topBoxTitle": "Foo", "leftBoxTitle": "Bar", "linkToClose": "https://…" }, "section2": { … } } I was thinking these keys are special and a different page would also have "section1" with "topBoxTitle," but maybe I misunderstood. If these keys aren't special and the frontend dev is just humanly interpreting this differently on each page then defining the actual layout to display it, that's probably fine. But I wouldn't describe this as rendering the visual page structure in the backend.

ChrisMarshallNY 3y ago

I've done this, and agree. But it was a good exercise for me, and I ended up using the general-purpose backend for a specific frontend. It's just that it's way overkill. Much more complex than it needs to be. But it works really, really well, and is secure as hell.

ChicagoDave 3y ago

I’m going to get zapped once again, but if you’re building complex applications, custom read models automatically updated based on events from operational writes is the optimal way to connect a front-end to APIs. Each read model should have a distinct business purpose. Generic models will only confuse maintainers and they’re very likely going to see such code as brittle and tech debt.

qudat 3y ago

Very similar sentiment to this article https://bower.sh/dogma-of-restful-api https://bower.sh/dogma-of-restful-api

ahuth 3y ago

Nice article This is one thing I’m loving about Remix. It’s how the framework works, so there’s no need to enforce this pattern or educate folks about it. AND Remix’s nested routing still has you break down the data loading into a couple different parts (1 per route segment). So it feels like you’re not loading everything in one place, and may be less likely to make people feel like they’re doing something “different” or “wrong”. It’s

agentultra 3y ago

Better, just don’t write an HTTP API for your front end. Use a server side framework. Done. It’s a pain coming into a project that was built the way the article recommends. Endpoints become RPC calls instead of resources serving entities. The hodgepodge of “API” calls becomes difficult to scale horizontally due to implied (and never documented) data dependencies. So you miss out on transport layer caching, entity caching, and a host of other goodies. All because your front end “framework” wants to live in a separate codebase from the server.

deleted 3y ago
[deleted]
hakunin 3y ago
The "page" becomes the resource for reads. You get great scalability, because you can use your database in the most efficient way possible to provide entire data for the page. You can maintain 100% back-end clarity on what each page depends on (instead of letting front-end request anything they want in any part of the view). I personally prefer server-side frameworks. I also understand when a front-end team prefers to work a certain way, has a lot of knowledge and tooling built-up around a certain way, and is worth accommodating for that reason.

backendanon 3y ago

> This is similar but not quite the same as Server Driven UI1. Perhaps we could call it Server Informed UI. I think we should just move back to server side rendering, JSP and even ASP were great, plain HTML, some JavaScript where desired since we can't seem to get away from JS.

cpursley 3y ago
The sweet-spot seems to be LiveView (Phoenix) and it's derivatives.
9dev 3y ago
Oh, right. Tell me how well that would work for Figma, Photoshop, or VS Code?

kburman 3y ago

Have to disagree with this. API is not always for FE. It can be consumed by other service, or a script. Making generic API helps reuse of common API across web pages. For example notification, user profile.

ricardobayes 3y ago

This only works for small projects. Data-heavy frontend usually sends multiple requests to allow async rendering of different parts of a page with different loading times. This also doesn't work if your design changes a lot. AKA a design change will now mean a backend re-design too.

rrradical 3y ago
Your first point is addressed in the article. For the second point, my experience is that the backend probably needs some work in that case anyway. If the backend is well designed, then only the presentation API endpoint needs to change. So a BE engineer is doing the tweaking instead of a FE engineer; not a big deal. Less of a big deal if there's no line drawn between FE and BE engineers.

jensneuse 3y ago

An alternative to what's suggested in this post, how about going API first or even API only? Don't build a frontend at all, just build an API. E.g. you can deploy things on fly.io just by using their GraphQL and REST APIs. The advantage of focussing on the apis is that other developers can much easier integrate with your service and embed it into their own applications.

ggregoire 3y ago

I don't know when we started splitting people into "frontend" and "backend" teams (FAANG influence?) and for what reasons, but 10 years ago we hired people with a general background, smart and curious enough to learn SQL, HTML/CSS/JS and whatever language you want to use on the server, able to build some end-to-end CRUD features on their own. And that's like what 99% of the companies need. My point being: when you have the same person building the UI and the API, you avoid a lot of the problems described in this article.

hakunin 3y ago
In a way, this article tries to bring the alignment between the frontend and backend closer to the full stack days, while also acknowledging the increased complexity of the front-end, that may be deserving of its own team focused on great UX.
Phreaker00 3y ago
For someone who has been in this field for 20 years I remember starting out doing every aspect of webdevelopment myself. Then the bigger and more complex the projects became, the less that made sense. The requirements of the distinct fields have gotten to a point where it's very impractical and hardly worthwhile for one person to master them all. You can't expect someone to translate a design into modern components that support all types of devices, resolutions and contexts to also come up with optimized SQL queries and knowing how to choose between Docker and Ansible for their distributed serverless functions that handle a variety of external endpoints. Ofcourse not every project that uses this approach needs this approach, and there's still a market for the Do-It-All developer. My point is that the webdevelopment world of today isn't that of a decade ago.
- ggregoire 3y ago
  > You can't expect someone to translate a design into modern components that support all types of devices, resolutions and contexts to also come up with optimized SQL queries and knowing how to choose between Docker and Ansible for their distributed serverless functions that handle a variety of external endpoints. Why not? Obviously you can't expect this from someone fresh out of college/bootcamp, but that's what I do on a daily basis and I'd expect this from anyone with 10+ years of exp.

moomoo11 3y ago

I write all my api code using a business logic “service” layer and a data ops “repository” layer. The endpoints are by feature not entity, although most of the time they overlap. This lets me move fast while keeping it simple to build the UI whether it’s mobile or web.

willsmith72 3y ago

I totally agree with the general idea, and this is generally how I build backends these days. If there are multiple clients, it's usually even ok for each to have a backend which talks to the "master" backend. But I will never get on board with sending things like content or UI related stuff from the backend. I've seen too many times, engineers think "all our forms are the same, let's just have the backend send a list of input field names and types like 'address'". It's almost always a bad abstraction, causes pain on the frontend, and restricts the creativity of engineers and designers.

robcohen 3y ago
Do you have the same opinion of headless CMS systems like Strapi?
hakunin 3y ago
The article doesn't disagree with your second paragraph. It encourages to treat each "page" as a unique snowflake, including forms in it. I should probably do a better job to discourage consistency and standardization of these json structures. This is addressed in another one of my comments if you search for "my bad".

Toine 3y ago

The "Controller" part of an API conceptually belongs to the "presentation" layer.

algem 3y ago

It sounds almost like the author is advocating for a really dumbed down BFF layer (backend for frontend) but what they are missing is behind the bff layer you would have general purpose apis. And if I interpret the article to the extreme what is the difference between this pattern and how html is currently served. Salesforce works sort of similar to what is described and it’s a nightmare, these restrictive ideals in my experience just aren’t good in practice because they tend to slow down development.

SlickStef11 3y ago
For those who are new to BFF's or backend for frontends, This website offers a detailed explanation on the common pattern. https://bff-patterns.com/ https://bff-patterns.com/

jokethrowaway 3y ago

Great article. We used to call this common sense. It's insane we drifted so far from being sensible. I genuinely think 90% of frontend developers should not exist: the web would be better and companies would have better products. I include myself in the frontenders, I do terrible apps which should just be simple static pages with a sprinkle of js - in my defense, I need the money and nobody listens to me, they say this is modern frontend development. For my tech not-savy clients I can deliver awesome and simple solutions without using the dreaded R framework.

tuckerconnelly 3y ago

I see the point of the article, and even on a principled level I might agree with it (e.g., don't overly abstract). A few problems I see though: 1. If you make backend-only abstractions to keep your code even a little bit DRY, you'll have a duplication of code (both the abstraction and the BFF endpoint). I think it'd be better to go all in on the abstraction up-front. 2. This will impair the conceptual integrity of the system and arguably lead to development slowdowns in the long run. To quote Mythical Man Month: “I will contend that conceptual integrity is the most important consideration in system design. It is better to have a system omit certain anomalous features and improvements, but to reflect one set of design ideas, than to have one that contains many good but independent and uncoordinated ideas.” 3. Your ad-hoc logic will likely be duplicated anyways across pages. 4. A general purpose API makes security features like RBAC much simpler. I think a general-purpose API is a well established abstraction you can rely on for pretty much any project.

bofaGuy 3y ago

This is how you lock yourself into a single front end design. And when you get ready for the next generation of your app or site, now you’re locked in and changing that front end design becomes exponentially harder.

hakunin 3y ago
In case of a complete redesign, yes you're right. With a generic API you will have about as much trouble writing the redesign as you did the original design. In practice, a complete redesign is an incredibly rare event, and a good problem to have. Usually design of individual pages and layouts undergoes gradual, incremental changes, driven by real needs. A redesign that requires front-end restructuring to such an extent that the data supplied is no longer sufficient should probably involve backend collaboration. Otherwise, you're tasking your backend team with building a generic site builder, which is much harder than a specific backend for your frontend.

simonbarker87 3y ago

So basically an old style MVC framework style app like RoR, CodeIgniter etc but made with a JSON response the describes the page rather like Shopify's Store Theme 2.0? Sounds great. Now since you have access to the server code (unlike Shopify's system described above) just return HTML and use your JS flavour of choice in the places where you need heavy JS ... like we used to make web apps. The happiest web devs I speak to are those working in the older MVC style, mainly RoR people. While everyone in AWS/Serverless/JS land is burning out and miserable with the house of cards stack we are balancing React on top of.

zianKazi 3y ago

I am not a big fan of building extremely fine grained general purpose APIs for frontends. A lot of the burden of juggling with data and presenting it is shifted to the UI layer. I am also not a big fan of building a composite API for "pages". Now you have a tightly coupled frontend/backend communication. What happens when we split the page or have to change the format of the data? Also the response of such an API can be quite big and nested. Ends up being a dump of data with poor documentation. I would try to find a middle path. Building composite APIs which have a concise purpose for the frontend but are modular enough to be reused across different areas of the application. Also, I feel that most of the problems discussed in the post are amplified when we have different groups working on the frontend and backend. If a single team is responsible for both the backend and frontend, API decisions are localized and can be changed as the development progresses.

theptip 3y ago

Counterpoint: I did this and it worked out great. We ended up wanting to support direct API access from our customers and we were already in a good place to do so. YMMV. An observation I have is that really flexible APIs make it easy for the frontend to make small changes, instead of needing to coordinate with backend changes. On the other hand this can lead to maintenance problems down the road as it’s hard to control exactly what data/state your system can transition to. I view this as one of the best pieces of tech debt to take on in an early-stage startup, basically keeping your options open and minimizing short-term friction, at the expense of taking on a bit more cleanup after you find PMF and start to make your backend more robust for the usecase you are scaling.

Attummm 3y ago
Engineering is about tradeoffs, everything has its place. But it would it be good standard? How would we look at that api when: New frontend is needed? Other team/company would like to acces the data? When we other clients for example anderiod/ios app? Wouldn't a generic api, with some custom api calls. Ensure that we are more consistent, and require less maintenance in the future. That way we could focus on adding business value, or solving those hard problems, we normally don't have time for.

codingclaws 3y ago

I read some of the HTMX essays [0] last week due to it being accepted into the GitHub accelerator program. So, a fully RESTful API must output HTML. I'm not going to go into why but that's what Roy Fielding (the creator of REST), Martin Fowler and the HTMX guy(s) say. The HTMX guy(s) also say that of course this fully RESTful API that outputs HTML is sometimes bad because you don't want to have to parse data out of HTML. So, the optimal setup is to have both a fully RESTful HTML API and the more common JSON API. I also think that for the fully RESTful HTML API, it doesn't make sense for the outputted HTML to have any JS. So, the question I'm pondering at the moment is, is a fully RESTful HTML API simply a website that doesn't use any JS? If so, then the optimal setup is to have an SSR website that doesn't use JS and a JSON API. Of course, under the hood, the website and the JSON API use the same lower level API (eg. SQL queries and back-end functions). What's crazy for me personally is that this is exactly how I have set up Comment Castles. There's the no JS website [1] and the JSON API [2]. But I didn't build a no JS website because of REST, I just prefer websites that have no front-end JS. [0] https://htmx.org/essays https://htmx.org/essays [1] https://www.commentcastles.org https://www.commentcastles.org [2] https://www.commentcastles.org/api https://www.commentcastles.org/api

adamckay 3y ago
> is a fully RESTful HTML API simply a website that doesn't use any JS? No, I don't think so. Sprinkling a little client side JS can certainly be beneficial and doesn't invalidate it being a RESTful HTML API. _hyperscript [1] was created by the htmx author for that purpose. There's a book [2] that you may not have seen written by some core htmx dev's that goes into this [3] in more detail. [1] - https://hyperscript.org/ https://hyperscript.org/ [2] - https://hypermedia.systems/ https://hypermedia.systems/ [3] - https://hypermedia.systems/client-side-scripting/ https://hypermedia.systems/client-side-scripting/

kgeist 3y ago

General purpose API helped us when we had several features to implement in a row on a tight schedule: when I was busy implementing feature N+1, our frontend dev was still polishing feature N (with changing UI/UX requirements), and she didn't need me (the backender) because the backend just exposed the model as is without knowing what the frontend looked like. She could make lots of changes to the UI without needing any changes to the backend. Yeah it translated to more API calls but it allowed us to decouple our work completely and deliver more features in parallel.

wruza 3y ago
Reading this may give the impression that non-api-decoupled things like e.g. ls or curl or inkscape cannot be developed in parallel. Also, think of it this way: if there was no general purpose API, it could be (figuratively) May instead of August when you did N and N+1. I mean, there’s no guarantee that it would, but this potential benefit is relative to a different baseline than in TFA.

rhuru 3y ago

I have building most of my modern sites with NextJS and could not have been happier as it gives me the flexibility of directly quering DB, I dont have to worry about the general purpose API. But if I wanted to I can just easily build it too.

yawnxyz 3y ago

doesn't this require you to nail down design requirements? If the designer gives a few different comps to try out, or wants to A/B test something, they'd have to align those changes with both backend AND frontend people to make sure they both know what the changes are. If the data dictates exactly what the frontend should look like / placement of content, shouldn't the backend just send styling / html / app code directly, and just skip frontend completely?

hankchinaski 3y ago

this is fine but makes the backend extremely slow and hard to maintain. it's effectively just glorified RPC - restful api low logic on backend and lots of logic on front-end - grpc style as the author suggests, more logic on backend no logic on front-end - keep api restful on backend, add backend for front-end(graphql/rpc), front-end remains simple, most logic is in the middle. this also allows you to isolate presentation spaghetti code in one place, keeps your backend api clean for use for other clients worked in all 3 probably having a custom presentation layer works best but require more setup upfront. another advantage of having a restful api and have the front-end do more work is being able to incrementally load data into the UI therefore reducing TTI/TTFB

gls2ro 3y ago

It might be that for the author specific case (which we dont know in details) maybe this is a good approach. In general I disagree with almost any idea that tries to fix some problems by adding coupling between components. I think the main thesis is where there is a false assumption: > Your frontend can finally focus on presentation and UI. Your backend can finally focus on implementing exactly what’s needed. If you make backend create one API call for each page, you will NOT get FE focusing on UI and backend on something else. You will get FE focusing on UI and BE focusing on UI and something else. I do get the feeling of rebellion against common wisdom that is FE and BE as separate components. When I was junior programmer (not suggesting here that author is junior, just talking about my experience) I often found myself rebelling against common programming wisdom. Now, after more than a decade of seeing so many changes and request I think those old programmers that gave us that old wisdom about separating concerns and losse coupling are right. I could boil down probably all old advice into the following 3 small pieces: 1. Write code. Not too much. Mostly simple, short and loosely coupled. 2. Go against common wisdom when you understand the tradeoffs. When not sure, pick the common wisdom. 3. You can maybe fix technical issues with more people, but can never fix organizational issues with code.

hakunin 3y ago
Author here. Here's an additional piece of wisdom that I picked up in over 20 years of doing this. Reliable communication allows for simpler, less coupled interfaces. Unreliable communication demands increasingly elaborate and coupled interfaces. I wrote a short note explaining this principle. https://notes.max.engineer/reliable-communication-allows-for-simpler-interfaces https://notes.max.engineer/reliable-communication-allows-for... > If you make backend create one API call for each page, you will NOT get FE focusing on UI and backend on something else. You will get FE focusing on UI and BE focusing on UI and something else. You need your backend to focus on something, otherwise it would be focused on nothing/everything. You need to tie yourself down to use cases, otherwise every use case is your use case.
- gls2ro 3y ago
  First let me again say sorry if my reply made you think I assume you are young. That was not my intention. Second > You need your backend to focus on something, otherwise it would be focused on nothing/everything. You need to tie yourself down to use cases, otherwise every use case is your use case. What I was saying is that your backend will now do what was doing before AND also will need to handle UI elements and relation between them. So you have duplicate data architecture/knowledge in UI: - once in FE as it needs to know the data architecture - again in BE where it also needs to know what data structure UI needs. Thus it couples FE and BE together around key names, JSON structure and moreso on information architecture from UI concepts. I am not saying you should not do that. Maybe it works for you. I would not do it. For me it will be like saying in a traditional simple HTML app that I should name tables and columns in DB as they are in the UI. Maybe some might coincide but that will not be an architectural constraint when thinking the DB.

RamblingCTO 3y ago

If you do proper hexagonal architecture it wouldn't actually matter and you can replace REST with graphql or feature-based endpoints. Before you rip that out, start with that. (YMMV and don't apply every piece of advice to every problem, that's the biggest problem)

eureka-belief 3y ago

What’s missing from this article is a discussion about 1) team composition and software development lifecycle, and 2) which libraries are available to help easily provide and consume the obvious RESTful operations. 1. Team composition If you have a separate backend developer from front-end developer (or worse, on a completely separate dev team), then making a separate endpoint for each page means you have to manage the feature dependency. As in, you need to make a ticket for the backend work first and the front end work can only start once the backend work is finished. And you need to do this for every single new page you make. On the other hand, if the backend team in advance just makes a generic rest API that provides all access to all of the obvious resources, it gives the front-end team the freedom to develop on their own timeline. If all your teams devs are full-stack, this is simplified a bit but still you have the problem of deciding to intelligently architect your data access layer to not have massive amounts of duplicated code. 2. Supporting libraries Often it’s possible on the backend to just easily make restful crud for every single resource that makes sense. A single backend dev can do this in one go without much conversation besides the initial one about access policies. Similarly, the are often front-end libraries that simplify not only writing getters for these endpoints but also for things like handling local caching, business object creation from JSON, and refetch logic. Obviously the libraries that are available and how well they integrate with your workflow will depend on your front-end framework. React, for example, has libraries that directly connect your rest access into specific components. — A caveat: obviously the pro-con landscape changes if you have serverside operations that either: A) feature some interaction that is sensitive to race conditions, B) requires a guarantee that one data type not get written unless the a previous write succeeded, or C) would require special queries for performance reasons In these cases, I’d say make a custom endpoint to encapsulate the correct/safe behavior. But these one-offs can be created only when they are needed. No premature optimization/complexity is required.

hakunin 3y ago
> you need to make a ticket for the backend work first and the front end work can only start once the backend work is finished In practice, a lot of this can be done in parallel. Once the design is established, front-end can start working on the layout/UX without knowing the exact shape of supplied data. The back-end can start proposing the shape of data and populating writing SQL/other queries to populate it. The final sync will only require small adjustments. > it gives the front-end team the freedom to develop on their own timeline At the cost of backend having to support every single use case under the sun. > still you have the problem of deciding to intelligently architect your data access layer to not have massive amounts of duplicated code. All this involves, is passing the pages the data they need for their construction. The alternative is building and supporting a generic data access for infinite use cases that is hard to optimize. I see this suggested often, and I don't think it's true that the way to reduce duplication is by providing all data forever. You can provide specific entities wrapped in pages, and reuse the backend logic used for their construction. > it’s possible on the backend to just easily make restful crud for every single resource This shifts all the complexity of combining data onto the front-end, and the front-end is a lot more distant from the data storage. See the note where I describe that "Reliable communication allows for simpler interfaces"[1]. [1]: https://notes.max.engineer/reliable-communication-allows-for-simpler-interfaces https://notes.max.engineer/reliable-communication-allows-for...

MattyRad 3y ago

The `Accept` HTTP header is there to prove your frontend is using "dumb" data. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Accept https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Ac... I think the author is generally correct that all data should be provided in a single request. And to take it a step further, you should be able to change your accept header to JSON to serve an API. API/dumb-frontend aren't mutually exclusive. In fact, this is what both GitLab and GitHub do. Try it out! `curl -L https://github.com/simonw/shot-scraper https://github.com/simonw/shot-scraper` (defaults to text/html) `curl --header "Accept: application/json" -L https://github.com/simonw/shot-scraper https://github.com/simonw/shot-scraper`