7 ms·

> Hence, over-fetching is not an option If you actually measure real latencies (e.g. collect metrics from your mobile apps), this is not true most of the time,

by yashap 4y ago

> Hence, over-fetching is not an option If you actually measure real latencies (e.g. collect metrics from your mobile apps), this is not true most of the time, at least in my experience. Most responses are fairly small, making them a bit smaller doesn’t have much impact on mobile app performance in the real world. I’d guess for most apps, there are maybe a handful of endpoints where the response size can be reduced a tonne for mobile clients, AND that makes a big performance difference in practice. But instead of hopping straight to BFFs, or other approaches to solving this like GraphQL, you can just add a “fields” query param to those few key endpoints. If provided by the caller, you return only the fields they ask for. No massive re-architecture to BFFs or GQL, just a small change to a couple endpoints that a single dev can implement in a day or two. Now, there are other benefits to BFFs than minimal, client specific payloads. One is that you can make breaking changes to underlying APIs, and not break always-so-slow-to-upgrade mobile clients, by preserving backwards compatibility in your mobile BFF layer. Another is coordinating multiple calls - turning many sequential round trips into a single round trip can be a big win. But if your main concern is payload size, BFFs are probably an over engineered solution.

eurasiantiger 4y ago
Overfetching is a massive problem when your entity has multiple N-ary references to other entities, and those entities have multiple N-ary references to other entities, and those entities have multiple N-ary references to other entities. Suddenly, your frontend or API exhibits high-degree polynomial time complexity.
- cjonas 4y ago
  Are those references not split out as separate api "resources" that can be queried independently?
  - eurasiantiger 4y ago
    That just moves the problem to the frontend. Often it can be worked around with HATEOAS at the architectural level or lazy loading, pagination, etc. in the frontend, but not always.
- davedx 4y ago
  Yeah this is where GraphQL (+ DataLoader or something like it) really shines. Still don’t see the need for architecting hydras like BFFs
- jcparkyn 4y ago
  But this is only a problem if one client _needs_ all that data at once, and another client doesn't. Otherwise you would just never return the referenced data (which isn't the type of overfetching the article is talking about).
- xwowsersx 4y ago
  Yeah so I'm at this point in our main service which is a rather large REST API. For certain endpoints — naturally lists of things — we run into polynomial time complexity, the N+1 queries problem. We query for a list of entities which, when serialized, queries for some child entity and on and on it goes. We've been able to stop some of the bleeding by refactoring queries to use `select_related` and `prefetch_related` (it's a Django app) so that we limit the number of queries, but this only gets us so far. I'm now at a point where I'm not sure what the next step should be. I believe caching may be the next logical step, but I've been hesitant due to concerns about the staleness problem, knowing when to bust the cache, etc. And then even if we do go the caching route, I'm unsure at what level we want to cache. Do we cache individual objects or do we cache the large, fully hydrated models (the ones that require many queries to assemble)? Then there's the problem of having to mix and match fetching objects from the cache in the context of other arbitrary DB queries that can be constructed. How do people generally go about introducing caching? Do you try to limit the scope of caching as much as possible (like to the lowest/smallest objects) and see whether that's sufficient to get the performance you want and then increase the caching surface area from there? Also, what approaches do people take from the technical side? We already use Redis for some things so I implemented a quick proof-of-concept that seemed to work in the small scale. I've also seen people use Elasticsearch to keep fully hydrated/rich models in there and then use ES as the read store. Are there merits to this approach? Crazy? Really keen to hear anyone's thoughts or guidance on this!
  - eurasiantiger 4y ago
    My advice would be to avoid attempting to solve the problem by introducing a cache layer or any auxiliary query cache, and instead remove the problem at storage level by leveraging a database solution that 1) stores relationships as links 2) allows queries to resolve relationships to an arbitrary level without specifying join conditions. These two things make sure your entity resolution can happen with direct lookups instead of index lookups (or pog forbid, table scans). If you absolutely cannot do that, cache at entity resolution level. Although, it really doesn’t solve the actual problem, just makes it more bearable by offloading primary DB workload to another DB, so why not run a read replica instead?
KronisLV 4y ago
> ...you can just add a “fields” query param to those few key endpoints. Dylan Beattie actually talked about resource expansion (with HAL) about 6 years ago: https://youtu.be/g8E1B7rTZBI?t=1654 https://youtu.be/g8E1B7rTZBI?t=1654 Essentially, it was a really nice and concise explanation of links to other data, problems with those and adding resource expansion to API endpoints. That said, it feels like HAL never really took off and the implementation of something like that is very framework specific. > ...or other approaches to solving this like GraphQL... GraphQL does seem to solve this, but it's like someone made an equivalent to SQL just for APIs, except that they forgot to include the bit that actually talks to your data store based on the queries and returns data, so you have to write that yourself (e.g. query resolvers). Unless you're using something like PostGraphile/Prisma/Hasura which exposes your database (provided you use one, which might be the situation in most cases) through GraphQL, but then you run into problems with intermediate data processing or additional access controls, that you might need to be API logic, before your data is returned. So, instead of the old approach: [client] <-RESTful request-> [your API] <-SQL query (with parameters)-> [SQL database] You might be tempted to go with something like this: [client] <-dynamic GraphQL query-> [GraphQL intermediate solution] <-SQL query (with parameters)-> [SQL database] But since that might prove to be untenable (unsafe), you might need something like: [client] <-dynamic GraphQL query-> [your API] <-dynamic GraphQL query-> [GraphQL intermediate solution] <-SQL query (with parameters)-> [SQL database] Which kind of feels worse from a complexity standpoint from where we were previously, especially if the queries are dynamic instead of largely static. Back to the topic, BFFs seem like a good idea, especially if they take over some of the gateway functionality as well. I've heard them be described as facade APIs in some cases.
- acoard 4y ago
  > Which kind of feels worse from a complexity standpoint from where we were previously, especially if the queries are dynamic instead of largely static. Agreed on increased complexity, but isn't the point largely the dynamic queries? You setup all this extra architecture, but in response you don't have to write as many "controllers" with custom API response. Just let the frontend request what it needs via GraphQL. So it's more upfront work, less down the line. Or is writing all the API level sanitizers and query resolvers more work?
hk1337 4y ago
From my understanding GraphQLs powerplay is in consolidating data sources rather than endpoints. That it all came through one endpoint was just a bonus.

by yashap 4y ago

eurasiantiger 4y ago

Overfetching is a massive problem when your entity has multiple N-ary references to other entities, and those entities have multiple N-ary references to other entities, and those entities have multiple N-ary references to other entities. Suddenly, your frontend or API exhibits high-degree polynomial time complexity.

cjonas 4y ago
Are those references not split out as separate api "resources" that can be queried independently?
- eurasiantiger 4y ago
  That just moves the problem to the frontend. Often it can be worked around with HATEOAS at the architectural level or lazy loading, pagination, etc. in the frontend, but not always.
davedx 4y ago
Yeah this is where GraphQL (+ DataLoader or something like it) really shines. Still don’t see the need for architecting hydras like BFFs
jcparkyn 4y ago
But this is only a problem if one client _needs_ all that data at once, and another client doesn't. Otherwise you would just never return the referenced data (which isn't the type of overfetching the article is talking about).
xwowsersx 4y ago
Yeah so I'm at this point in our main service which is a rather large REST API. For certain endpoints — naturally lists of things — we run into polynomial time complexity, the N+1 queries problem. We query for a list of entities which, when serialized, queries for some child entity and on and on it goes. We've been able to stop some of the bleeding by refactoring queries to use `select_related` and `prefetch_related` (it's a Django app) so that we limit the number of queries, but this only gets us so far. I'm now at a point where I'm not sure what the next step should be. I believe caching may be the next logical step, but I've been hesitant due to concerns about the staleness problem, knowing when to bust the cache, etc. And then even if we do go the caching route, I'm unsure at what level we want to cache. Do we cache individual objects or do we cache the large, fully hydrated models (the ones that require many queries to assemble)? Then there's the problem of having to mix and match fetching objects from the cache in the context of other arbitrary DB queries that can be constructed. How do people generally go about introducing caching? Do you try to limit the scope of caching as much as possible (like to the lowest/smallest objects) and see whether that's sufficient to get the performance you want and then increase the caching surface area from there? Also, what approaches do people take from the technical side? We already use Redis for some things so I implemented a quick proof-of-concept that seemed to work in the small scale. I've also seen people use Elasticsearch to keep fully hydrated/rich models in there and then use ES as the read store. Are there merits to this approach? Crazy? Really keen to hear anyone's thoughts or guidance on this!
- eurasiantiger 4y ago
  My advice would be to avoid attempting to solve the problem by introducing a cache layer or any auxiliary query cache, and instead remove the problem at storage level by leveraging a database solution that 1) stores relationships as links 2) allows queries to resolve relationships to an arbitrary level without specifying join conditions. These two things make sure your entity resolution can happen with direct lookups instead of index lookups (or pog forbid, table scans). If you absolutely cannot do that, cache at entity resolution level. Although, it really doesn’t solve the actual problem, just makes it more bearable by offloading primary DB workload to another DB, so why not run a read replica instead?

KronisLV 4y ago

> ...you can just add a “fields” query param to those few key endpoints. Dylan Beattie actually talked about resource expansion (with HAL) about 6 years ago: https://youtu.be/g8E1B7rTZBI?t=1654 https://youtu.be/g8E1B7rTZBI?t=1654 Essentially, it was a really nice and concise explanation of links to other data, problems with those and adding resource expansion to API endpoints. That said, it feels like HAL never really took off and the implementation of something like that is very framework specific. > ...or other approaches to solving this like GraphQL... GraphQL does seem to solve this, but it's like someone made an equivalent to SQL just for APIs, except that they forgot to include the bit that actually talks to your data store based on the queries and returns data, so you have to write that yourself (e.g. query resolvers). Unless you're using something like PostGraphile/Prisma/Hasura which exposes your database (provided you use one, which might be the situation in most cases) through GraphQL, but then you run into problems with intermediate data processing or additional access controls, that you might need to be API logic, before your data is returned. So, instead of the old approach: [client] <-RESTful request-> [your API] <-SQL query (with parameters)-> [SQL database] You might be tempted to go with something like this: [client] <-dynamic GraphQL query-> [GraphQL intermediate solution] <-SQL query (with parameters)-> [SQL database] But since that might prove to be untenable (unsafe), you might need something like: [client] <-dynamic GraphQL query-> [your API] <-dynamic GraphQL query-> [GraphQL intermediate solution] <-SQL query (with parameters)-> [SQL database] Which kind of feels worse from a complexity standpoint from where we were previously, especially if the queries are dynamic instead of largely static. Back to the topic, BFFs seem like a good idea, especially if they take over some of the gateway functionality as well. I've heard them be described as facade APIs in some cases.

acoard 4y ago
> Which kind of feels worse from a complexity standpoint from where we were previously, especially if the queries are dynamic instead of largely static. Agreed on increased complexity, but isn't the point largely the dynamic queries? You setup all this extra architecture, but in response you don't have to write as many "controllers" with custom API response. Just let the frontend request what it needs via GraphQL. So it's more upfront work, less down the line. Or is writing all the API level sanitizers and query resolvers more work?

hk1337 4y ago

From my understanding GraphQLs powerplay is in consolidating data sources rather than endpoints. That it all came through one endpoint was just a bonus.