4 ms·

We’ve been using data loader a lot and it’s working ok. We were very careful to dump all sql queries our app was making per request to ensure we were not making

by kevsim 6y ago

We’ve been using data loader a lot and it’s working ok. We were very careful to dump all sql queries our app was making per request to ensure we were not making more queries than necessary. One thing I will say as we do make a lot of “IN” queries as a result. You fetch a bunch of rows with a data loader and then you want to fetch a bunch of rows from some other table based on those first results so you end up with an IN query where you might have had crafted a JOIN if you were writing it by hand.

xyzzy_plugh 6y ago
But does the query plan perform a join? The text of your query matters less than the plan it generates.
- kevsim 6y ago
  No, it can't because it's actually making two separate queries. For example, let's say we're fetching classified ads with the users that posted them. First your GraphQL resolvers would fetch the ads (and you won't know that the users are being asked for). Then for each ad, you'd tell your data loader to fetch the user associated with that add. That will ultimately result in a single SQL query with an IN like `SELECT * FROM users WHERE id IN (?)` where you'd pass in all the user IDs you gathered from the results of the ads query. However, another commenter pointed out above that some implementations allow peeking into the GraphQL query. That way if you know you're going to need the user anyway, you can already perform the join proactively when you fetch the ads. In a few cases we've done this because we always want the joined results so we just do the join regardless of the query we get.
  - xyzzy_plugh 6y ago
    Erm, are you telling me there is no way to pipeline the ads query to be inline within the IN clause? Because that'd generate a join, generally. I still don't understand why this abstraction is useful, if not harmful.
    - kevsim 6y ago
      I think there is a way to do it, that was the peeking I referred to. But it's not how it works out of the box. I think the abstraction is somewhat useful if your GraphQL queries vary tremendously (one device asks for the user along with the ads, another device doesn't). Expand this out to a bunch of different data types and you'd find yourself writing a lot of variations of the queries. GraphQL with Data loader allows you to write these things exactly once albeit with potentially suboptimal queries. All of that being said, we're finding that our queries vary very little and are somewhat questioning the value of data loaders and of GraphQL altogether in our case.
snuxoll 6y ago
> One thing I will say as we do make a lot of “IN” queries as a result. You fetch a bunch of rows with a data loader and then you want to fetch a bunch of rows from some other table based on those first results so you end up with an IN query where you might have had crafted a JOIN if you were writing it by hand. I don't know if it's common across other GraphQL implementations, but the graphql-java library lets you peek [0] into the selections so you can opt to limit data you select, execute a join instead of using an IN query, etc. I've used this with good results when making a prototype graphql interface to our billing software, anything that had a 1:1 relationship with a parent record would get JOINed by the resolver, while 1:N or N:M relationships would execute separate queries. [0]:https://www.graphql-java.com/documentation/v14/fieldselection/ https://www.graphql-java.com/documentation/v14/fieldselectio...

by kevsim 6y ago

xyzzy_plugh 6y ago

But does the query plan perform a join? The text of your query matters less than the plan it generates.

kevsim 6y ago
No, it can't because it's actually making two separate queries. For example, let's say we're fetching classified ads with the users that posted them. First your GraphQL resolvers would fetch the ads (and you won't know that the users are being asked for). Then for each ad, you'd tell your data loader to fetch the user associated with that add. That will ultimately result in a single SQL query with an IN like `SELECT * FROM users WHERE id IN (?)` where you'd pass in all the user IDs you gathered from the results of the ads query. However, another commenter pointed out above that some implementations allow peeking into the GraphQL query. That way if you know you're going to need the user anyway, you can already perform the join proactively when you fetch the ads. In a few cases we've done this because we always want the joined results so we just do the join regardless of the query we get.
- xyzzy_plugh 6y ago
  Erm, are you telling me there is no way to pipeline the ads query to be inline within the IN clause? Because that'd generate a join, generally. I still don't understand why this abstraction is useful, if not harmful.
  - kevsim 6y ago
    I think there is a way to do it, that was the peeking I referred to. But it's not how it works out of the box. I think the abstraction is somewhat useful if your GraphQL queries vary tremendously (one device asks for the user along with the ads, another device doesn't). Expand this out to a bunch of different data types and you'd find yourself writing a lot of variations of the queries. GraphQL with Data loader allows you to write these things exactly once albeit with potentially suboptimal queries. All of that being said, we're finding that our queries vary very little and are somewhat questioning the value of data loaders and of GraphQL altogether in our case.

snuxoll 6y ago

> One thing I will say as we do make a lot of “IN” queries as a result. You fetch a bunch of rows with a data loader and then you want to fetch a bunch of rows from some other table based on those first results so you end up with an IN query where you might have had crafted a JOIN if you were writing it by hand. I don't know if it's common across other GraphQL implementations, but the graphql-java library lets you peek [0] into the selections so you can opt to limit data you select, execute a join instead of using an IN query, etc. I've used this with good results when making a prototype graphql interface to our billing software, anything that had a 1:1 relationship with a parent record would get JOINed by the resolver, while 1:N or N:M relationships would execute separate queries. [0]:https://www.graphql-java.com/documentation/v14/fieldselection/ https://www.graphql-java.com/documentation/v14/fieldselectio...