6 ms·

The ‘Backends for Frontends’ Pattern at SoundCloud

130 points by sgeb 11y ago

andrewingram 11y ago
This pattern is essentially what technologies like GraphQL or Falcor make super easy. My GraphQL server talks to a bunch of microservices and presents a simple clean API for my web application, largely freeing the front-end from years worth of bloated APIs.
- zappo2938 11y ago
  This was my first thought also. They invented a name for the pattern but it is just a facade with the development responsibilities delegated to the front end team. Nothing ground breaking. This is probably more of a communication solution between front end and back end developers. The concept is very similar to GraphQL and Falcor. There isn't a benefit BFF gets with multiple end points over a single end point interface. I was trying to explain what Falcor was to a friend a couple weeks ago. It's a facade pattern and a correct use case for javascript and its asynchronous nature that has the added benefit of also being the language Facebook and Netflix frontend developers are using when they use React. Outside of the interface on the backend they are not using javascript but rather Python, Java, and in the case of Facebook, PHP, yuk. The point here is that even where the frontend interfaces with the backend it is still in the frontend's domain. The word, phrase, that came to mind, although I rarely use it, was clearing house. This pattern of managing transactions already exists in the business world -- the software used in clearing houses might already reflect this, that would be curious. It makes sense and there are similarities including error checking between a financial clearing house and these APIs. A clearing API might be an appropriate name.
- herval 11y ago
  These sorts of overarching architectures always leave me scratching my head too. IMHO a combination of a graph layer (eg the two you mentioned) and an scs architecture (scs-architecture.org) allow teams to remain largely independent and productive. That might sound superficially similar to "BFFs" as in "each team owns the client + api", but shifts the responsibility for larger units of work and away from individual microservices (which becomes a huge pain when you start to have dozens of BFFs interfacing with hundreds of microservices maintained by independent teams).
- bsaul 11y ago
  I don't think having a single entry point aggregating micro-services to provide a unified API for all your front-ends is what this pattern is about. Actually pretty much the opposite, but maybe i'm missing something...
  - kapad 11y ago
    It isn't really a unified API for all front ends, but a separate API for each front end. And sometimes that really makes sense.
lmm 11y ago
This is true at every interface though. Better to have the same people working on the backend for the backend for the frontend. Better to have the same people work on the database as the thing that queries it. Ultimately you should separate teams by the functionality they're supporting, not by which part of the stack they're notionally in. One interesting approach is to pair-program at each step - designer and frontendey dev do the page, frontendy dev and backendy dev do the front half of the backend, backendy dev and DBAey dev do the data access layer.
leonardinius 11y ago
I see SoundCloud brand appearing more and more frequently in my blog/news feeds. Are they in the "search for the developers" /quick grow phase?
- dominotw 11y ago
  doesn't look like it https://soundcloud.com/jobs/product_design_engineering https://soundcloud.com/jobs/product_design_engineering
bshimmin 11y ago
I love the diagrams on this blog post. Does anyone know if they're hand-drawn or if there's a tool that produces them?
- saj1th 11y ago
  The ipad app paper could be used to create similar looking diagrams
- lplotni 11y ago
  Thanks. The diagrams are hand-drawn and scanned :)
cdnsteve 11y ago
I actually found the article from Sam Newman went into greater depth: http://samnewman.io/patterns/architectural/bff/ http://samnewman.io/patterns/architectural/bff/ Interesting to see this creates new problems, such as blocking calls to dependant services downstream, and what services you declare as critical or not. Also duplicate code is unavoidable. So if you're attempting this pattern, keep those things in mind since you will have limited technology choices. Sound Cloud is using Scala here. https://developers.soundcloud.com/blog/building-products-at-soundcloud-part-3-microservices-in-scala-and-finagle https://developers.soundcloud.com/blog/building-products-at-... Edit: Found their mention of technology
- zappo2938 11y ago
  Isn't this the same problem Facebook is solving with GraphGL and Netflix with Falcor? Doesn't it also make sense to use node.js and express where the front end meets the back end for two reasons; one, javascript is asynchronous in nature, and, two, javascript is the language used by frontend developers?
  - herval 11y ago
    By "frontend", you might have ios or android apps, for instance (the case in this study).
    - deleted 11y ago
      [deleted]
      - Raphmedia 11y ago
        I'll speak as a frontend dev and say that it's not about the languages but about the final objective. I'll use "backend languages" as needed to get the job done.
    - zappo2938 11y ago
      Touche! I'm just so used to this discussion managing microservices and APIs coming out of the Facebook and Netflix camps I had to stop and think, oh, not everybody is using React wrapped in a native app. Woosh, right over my head. It is possible this approach by SoundCloud is more of a solution to developer teams not communicating. I reread OP's article and the question I'm asking now is why wouldn't they want a consistent API across all platforms? I think that it is BS that Andriod and iOS apps for the same product require a different backend API. That makes no sense! This seems like symptom of the Andriod group doing whatever they want and the iOS group doing whatever they want.
deleted 11y ago
[deleted]
dominotw 11y ago
why is this on thoughtworks website when sound cloud has their own dev blog (https://developers.soundcloud.com/blog/ https://developers.soundcloud.com/blog/)
kapad 11y ago
I'd read a very similar article from back in 2013 that spoke about the same concept without really calling it "BFF". The article in question actually talks about just using nodejs, but what we implemented from that at my company was really what "BFF" is, and I really think that every microservice architecture needs the backed for front end layer. Here's the link: https://www.nczonline.net/blog/2013/10/07/node-js-and-the-new-web-front-end/ https://www.nczonline.net/blog/2013/10/07/node-js-and-the-ne...
PaulHoule 11y ago
Since communications overhead is so large in internet applications, particularly on mobile devices, there really has to be "one mind" in charge of both the front and back end of an app. That "one mind" can be a consensus in a bunch of people's minds -- in fact it really should be. My experience is that most programmers don't have a complete set of skills and attitudes to optimize distributed systems. From the systems programmer who writes mad C++ but can't understand why David Fielding hasn't won a Fields Medal to the application programmer who has never heard of O(N^2) and doesn't care, or the machine learning genius who is oblivious to locks and race conditions in development and seems to fix them by patching the production system directly multiple times over the course of a nail biting week for customers, management and staff. (Somehow planning to take 2 or 3 days of one person's time to understand the problem is a "waste of time") Anyway, the key to this is working as a team that is closer to the sum of the parts rather than the difference.
hackercomplex 11y ago
I don't get it. Why is this so complicated when we have nginx and consul-template ?
drudru11 11y ago
I like the solution. It is a classic solution too. A famous aphorism of David Wheeler goes: "All problems in computer science can be solved by another level of indirection";[1] this is often deliberately mis-quoted with "abstraction layer" substituted for "level of indirection". Kevlin Henney's corollary to this is, "...except for the problem of too many layers of indirection." https://en.m.wikipedia.org/wiki/Indirection https://en.m.wikipedia.org/wiki/Indirection
corsibu 11y ago
I am really curious how is this different to a hexagonal architecture

130 points by sgeb 11y ago

This pattern is essentially what technologies like GraphQL or Falcor make super easy. My GraphQL server talks to a bunch of microservices and presents a simple clean API for my web application, largely freeing the front-end from years worth of bloated APIs.

zappo2938 11y ago
This was my first thought also. They invented a name for the pattern but it is just a facade with the development responsibilities delegated to the front end team. Nothing ground breaking. This is probably more of a communication solution between front end and back end developers. The concept is very similar to GraphQL and Falcor. There isn't a benefit BFF gets with multiple end points over a single end point interface. I was trying to explain what Falcor was to a friend a couple weeks ago. It's a facade pattern and a correct use case for javascript and its asynchronous nature that has the added benefit of also being the language Facebook and Netflix frontend developers are using when they use React. Outside of the interface on the backend they are not using javascript but rather Python, Java, and in the case of Facebook, PHP, yuk. The point here is that even where the frontend interfaces with the backend it is still in the frontend's domain. The word, phrase, that came to mind, although I rarely use it, was clearing house. This pattern of managing transactions already exists in the business world -- the software used in clearing houses might already reflect this, that would be curious. It makes sense and there are similarities including error checking between a financial clearing house and these APIs. A clearing API might be an appropriate name.
herval 11y ago
These sorts of overarching architectures always leave me scratching my head too. IMHO a combination of a graph layer (eg the two you mentioned) and an scs architecture (scs-architecture.org) allow teams to remain largely independent and productive. That might sound superficially similar to "BFFs" as in "each team owns the client + api", but shifts the responsibility for larger units of work and away from individual microservices (which becomes a huge pain when you start to have dozens of BFFs interfacing with hundreds of microservices maintained by independent teams).
bsaul 11y ago
I don't think having a single entry point aggregating micro-services to provide a unified API for all your front-ends is what this pattern is about. Actually pretty much the opposite, but maybe i'm missing something...
- kapad 11y ago
  It isn't really a unified API for all front ends, but a separate API for each front end. And sometimes that really makes sense.

lmm 11y ago

This is true at every interface though. Better to have the same people working on the backend for the backend for the frontend. Better to have the same people work on the database as the thing that queries it. Ultimately you should separate teams by the functionality they're supporting, not by which part of the stack they're notionally in. One interesting approach is to pair-program at each step - designer and frontendey dev do the page, frontendy dev and backendy dev do the front half of the backend, backendy dev and DBAey dev do the data access layer.

leonardinius 11y ago

I see SoundCloud brand appearing more and more frequently in my blog/news feeds. Are they in the "search for the developers" /quick grow phase?

dominotw 11y ago
doesn't look like it https://soundcloud.com/jobs/product_design_engineering https://soundcloud.com/jobs/product_design_engineering

bshimmin 11y ago

I love the diagrams on this blog post. Does anyone know if they're hand-drawn or if there's a tool that produces them?

saj1th 11y ago
The ipad app paper could be used to create similar looking diagrams
lplotni 11y ago
Thanks. The diagrams are hand-drawn and scanned :)

cdnsteve 11y ago

I actually found the article from Sam Newman went into greater depth: http://samnewman.io/patterns/architectural/bff/ http://samnewman.io/patterns/architectural/bff/ Interesting to see this creates new problems, such as blocking calls to dependant services downstream, and what services you declare as critical or not. Also duplicate code is unavoidable. So if you're attempting this pattern, keep those things in mind since you will have limited technology choices. Sound Cloud is using Scala here. https://developers.soundcloud.com/blog/building-products-at-soundcloud-part-3-microservices-in-scala-and-finagle https://developers.soundcloud.com/blog/building-products-at-... Edit: Found their mention of technology

zappo2938 11y ago
Isn't this the same problem Facebook is solving with GraphGL and Netflix with Falcor? Doesn't it also make sense to use node.js and express where the front end meets the back end for two reasons; one, javascript is asynchronous in nature, and, two, javascript is the language used by frontend developers?
- herval 11y ago
  By "frontend", you might have ios or android apps, for instance (the case in this study).
  - deleted 11y ago
    [deleted]
    - Raphmedia 11y ago
      I'll speak as a frontend dev and say that it's not about the languages but about the final objective. I'll use "backend languages" as needed to get the job done.
  - zappo2938 11y ago
    Touche! I'm just so used to this discussion managing microservices and APIs coming out of the Facebook and Netflix camps I had to stop and think, oh, not everybody is using React wrapped in a native app. Woosh, right over my head. It is possible this approach by SoundCloud is more of a solution to developer teams not communicating. I reread OP's article and the question I'm asking now is why wouldn't they want a consistent API across all platforms? I think that it is BS that Andriod and iOS apps for the same product require a different backend API. That makes no sense! This seems like symptom of the Andriod group doing whatever they want and the iOS group doing whatever they want.

deleted 11y ago

[deleted]

dominotw 11y ago

why is this on thoughtworks website when sound cloud has their own dev blog (https://developers.soundcloud.com/blog/ https://developers.soundcloud.com/blog/)

kapad 11y ago

I'd read a very similar article from back in 2013 that spoke about the same concept without really calling it "BFF". The article in question actually talks about just using nodejs, but what we implemented from that at my company was really what "BFF" is, and I really think that every microservice architecture needs the backed for front end layer. Here's the link: https://www.nczonline.net/blog/2013/10/07/node-js-and-the-new-web-front-end/ https://www.nczonline.net/blog/2013/10/07/node-js-and-the-ne...

PaulHoule 11y ago

Since communications overhead is so large in internet applications, particularly on mobile devices, there really has to be "one mind" in charge of both the front and back end of an app. That "one mind" can be a consensus in a bunch of people's minds -- in fact it really should be. My experience is that most programmers don't have a complete set of skills and attitudes to optimize distributed systems. From the systems programmer who writes mad C++ but can't understand why David Fielding hasn't won a Fields Medal to the application programmer who has never heard of O(N^2) and doesn't care, or the machine learning genius who is oblivious to locks and race conditions in development and seems to fix them by patching the production system directly multiple times over the course of a nail biting week for customers, management and staff. (Somehow planning to take 2 or 3 days of one person's time to understand the problem is a "waste of time") Anyway, the key to this is working as a team that is closer to the sum of the parts rather than the difference.

hackercomplex 11y ago

I don't get it. Why is this so complicated when we have nginx and consul-template ?

drudru11 11y ago

I like the solution. It is a classic solution too. A famous aphorism of David Wheeler goes: "All problems in computer science can be solved by another level of indirection";[1] this is often deliberately mis-quoted with "abstraction layer" substituted for "level of indirection". Kevlin Henney's corollary to this is, "...except for the problem of too many layers of indirection." https://en.m.wikipedia.org/wiki/Indirection https://en.m.wikipedia.org/wiki/Indirection

corsibu 11y ago

I am really curious how is this different to a hexagonal architecture