5 ms·

Our agentic builder has a single tool. It is called graphql. The agent writes a query and executes it. If the agent does not know how to do particular type of

by _pdp_ 11mo ago

Our agentic builder has a single tool. It is called graphql. The agent writes a query and executes it. If the agent does not know how to do particular type of query then it can use graphql introspection. The agent only receives the minimal amount of data as per the graphql query saving valuable tokens. It works better! Not only we don't need to load 50+ tools (our entire SDK) but it also solves the N+1 problem when using traditional REST APIs. Also, you don't need to fall back to write code especially for query and mutations. But if you need to do that, the SDK is always available following graphql typed schema - which helps agents write better code! While I was never a big fan of graphql before, considering the state of MCP, I strongly believe it is one of the best technologies for AI agents. I wrote more about this here if you are interested: https://chatbotkit.com/reflections/why-graphql-beats-mcp-for-agentic-ai https://chatbotkit.com/reflections/why-graphql-beats-mcp-for...

roflyear 11mo ago
I do think that using graphql will solve a lot of problems for people but it's super surprising how many people absolutely hate it.
- dcre 11mo ago
  GraphQL is just a typed schema (good) with a server capable of serving any subset of the entire schema at a time (pain in the ass).
  - jlouis 11mo ago
    No. It's a way to transmit a program from client to server. It then executes that program on the server side.
    - dcre 11mo ago
      That sounds even worse!
      - jlouis 11mo ago
        It's not. The fragments you can execute are limited if you do it right. A client isn't allowed to just execute anything it wants, because the valid operations are pre-determined. The client sends a reference which executes a specific pre-planned fragment of code. In development, you let clients roam free, so you have access to the API in a full manner. Deployments then lock-down the API. If you just let a client execute anything it wants in production, you get into performance-trouble very easily once a given client decides to be adventurous. GraphQL is an execution semantics. It's very close to a lambda calculus, but I don't think that was by design. I think that came about by accident. A client is really sending a small fragment of code to the server, which the server then executes. The closest thing you have is probably SQL queries: the client sends a query to the server, which the server then executes. It's fundamental to the idea of GraphQL as well. You want to put power into the hands of the client, because that's what allows a top-down approach to UX design. If you always have to manipulate the server-side whenever a client wants to change call structure, you've lost.
        DenisM 11mo ago
        No one exposes SQL to clients though. I think where Gql differs from sql is it’s at a higher level. SQL bleeds performance and data layout (e.g. normalizing, limits), GraphQL does not. It’s not clear if it’s high enough to abstract knowledge from storage. In the end it’s tension between enabling client to wander around productively vs being a bull in a china shop.
  - wrs 11mo ago
    It doesn’t actually require that second part. Every time I’ve used it in a production system, we had an approved list of query shapes that were accepted. If the client wanted to use a new kind of query, it was performance tested and sometimes needed to be optimized before approval for use. If you open it up for any possible query, then give that to uncontrolled clients, it’s a recipe for disaster.
    - kaoD 11mo ago
      Oh, we have that too! But we call it HTTP endpoints.
      - johnfn 11mo ago
        Really? Hmm... where in the HTTP spec does it allow for returning an arbitrary subset of any specific request, rather than the whole thing? And where does it ensure all the results are keyed by id so that you can actually build and update a sensible cache around all of it rather than the mess that totally free-form HTTP responses lead to? Oh weird HTTP doesn't have any of that stuff? Maybe we should make a new spec, something which does allow for these patterns and behaviors? And it might be confusing if we use the exact same name as HTTP, since the usage patterns are different and it enables new abilities. If only we could think of such a name...
        eli 11mo ago
        An HTTP Range request asks the server to send parts of a resource back to a client. Range requests are useful for various clients, including media players that support random access, data tools that require only part of a large file, and download managers that let users pause and resume a download. https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Ran...
        867-5309 11mo ago
        also handy for bypassing bandwidth restrictions: capped at 100kbps? launch 1000 workers to grab chunks then assemble the survivors
- koakuma-chan 11mo ago
  I wish people at least stopped using JavaScript and stopped writing requests to back-end by hand.
deleted 11mo ago
[deleted]
notpachet 11mo ago
Reading this was such an immediate "aha" for me. Of course we should be using GraphQL for this. Damn. Where was this comment three months ago!
bnchrch 11mo ago
1000% 2 years ago I gave a talk on Vector DB's and LLM use. https://www.youtube.com/watch?v=U_g06VqdKUc https://www.youtube.com/watch?v=U_g06VqdKUc TLDR but it shows how you could teach an LLM your GraphQL query language to let it selectively load context into what were very small context windows at the time. After that the MCP specification came out. Which from my vantage point is a poor and half implemented version of what GraphQL already is.
refibrillator 11mo ago
> It works better! > I strongly believe it is one of the best technologies for AI agents Do you have any quantitative evidence to support this? Sincere question. I feel it would add some much needed credibility in a space where many folks are abusing the hype wave and low key shilling their products with vibes instead of rigor.
- steveklabnik 11mo ago
  I have thought about this for all of thirty seconds, but it wouldn't shock me if this was the case. The intuition here is about types, and the ability to introspect them. Agents really love automated guardrails. It makes sense to me that this would work better than RESTish stuff, even with OpenAPI.
  - ibash 11mo ago
    Better than rest is a low bar though. Ultimately agents should rarely be calling raw rest and graphql apis, which are meant for programmatic use. Agents should be calling one level of abstraction higher. Eg calling a function to “find me relevant events in this city according to this users preferences” instead of “list all events in this city”.
  - MrDarcy 11mo ago
    Same in terms of time spent. The hypothesis graphql is superior passes the basic sniff test. Assuming graphql does what it says on the tin, which my understanding is it does based on my work with Ent, then the claim it’s better for tool and api use by agents follows from common sense.
  - s900mhz 11mo ago
    This is a task I think is suited for a sub agent that is small in size. It can can take the context beating to query for relevant tools and return only what is necessary to the main agent thread.
- brulard 11mo ago
  If you knew GraphQL, you may immediately see it - you ask for specific nested structure of the data, which can span many joins across different related collections. This is not the case with common REST API or CLI for example. And introspection is another good reason.
ramnivasl 11mo ago
That is also the approach we took with Exograph (https://exograph.dev https://exograph.dev). Here is our reasoning (https://exograph.dev/blog/exograph-now-supports-mcp#comparing-with-alternatives https://exograph.dev/blog/exograph-now-supports-mcp#comparin...). We found that LLMs do a very good job of crafting GraphQL queries for the given schema. While they do make mistakes, returning good descriptive error messages make is easy for them fix queries.
adverbly 11mo ago
This is actually a really good use of graphql! IMO the biggest pain points of graphql are authorization/rate limiting, caching, and mutations... But for selective context loading none of those matter actually. Pretty cool!
esafak 11mo ago
Can anyone recommend an open source GraphQL-based MCP/tool gateway?
bravura 11mo ago
Isn't the challenge that introspecting graphql will lead to either a) a very long set of definitions consuming many tokens or b) many calls to drill into the introspection?
- peacebeard 11mo ago
  In my experience, this was the limitation we ran into with this approach. If you have a large API this will blow up your context. I have had the best luck with hand-crafted tools that pre-digest your API so you don't have to waste tokens or deal with context rot bugs.
- _pdp_ 11mo ago
  Well either that or stuff the tool usage examples into the prompt for every single request. If you have only 2-3 tools GraphQL is certainly not necessary - but it wont blow up the context either. If you have 50+ tools, I don't see any other way to be honest, unless you create your own tool discovery solution - which is what GraphQL does really well with the caveat that whatever you decide to do is certainly not natural to these LLMs. Keep in mind that all LLMs are trained on many GraphQL examples because the technology has been in existence since 2015. While anything custom might just work it is certainly not part of the model training set unless you fine-tune. So yes, if I need to decide on formats I will go for GraphQL, SQL and Markdown.
AIorNot 11mo ago
your use-case is NOT Everyones use-case..(working in depth across one codebase or api but instead sampling dozens of abilities across the web or with other systems) thats the thing how is that going to work with my use case, do a web search, do a local api call, do a graphql search, do an integration with slack, do a message etc..
- alfonsodev 11mo ago
  Does it matter ? if it's well defined, each of those would be a node in the graph, or can you elaborate ? Dozens seems not that much, for a graph where a higher level node would be slack, and the agent only loads further if it needs anything related with slack. Or I'm not understanding.
geoffhill 11mo ago
One of my agents is kinda like this too. The only operation is SPARQL query, and the only accessible state is the graph database. Since most of the ontologies I'm using are public, I just have to namedrop them in prompt; no schemas and little structure introspection needed. At worst, it can just walk and dump triples to figure out structure; it's all RDF triples and URIs. One nice property: using structured outputs, you can constrain outputs of certain queries to only generate valid RDF to avoid syntax errors. Probably can do similar stuff with GraphQL.
fy20 11mo ago
Whoa there, you don't need to be so sadistic to your team. It's not GraphQL, but having a document describing how your API works, including types, that is important. I expect you could achieve the same with a comprehensive OpenAPI specification. If you want something a bit stricter I guess SOAP would work too, LLMs love XML after all.
- _pdp_ 11mo ago
  We have well described OpenAPI and GraphQL specifications already. :) Being AI-first means we are naturally aligned with that kind of structured documentation. It helps both humans and robots.
deleted 11mo ago
[deleted]
sibeliuss 11mo ago
GraphQL FTW

by _pdp_ 11mo ago

roflyear 11mo ago

I do think that using graphql will solve a lot of problems for people but it's super surprising how many people absolutely hate it.

dcre 11mo ago
GraphQL is just a typed schema (good) with a server capable of serving any subset of the entire schema at a time (pain in the ass).
- jlouis 11mo ago
  No. It's a way to transmit a program from client to server. It then executes that program on the server side.
  - dcre 11mo ago
    That sounds even worse!
    - jlouis 11mo ago
      It's not. The fragments you can execute are limited if you do it right. A client isn't allowed to just execute anything it wants, because the valid operations are pre-determined. The client sends a reference which executes a specific pre-planned fragment of code. In development, you let clients roam free, so you have access to the API in a full manner. Deployments then lock-down the API. If you just let a client execute anything it wants in production, you get into performance-trouble very easily once a given client decides to be adventurous. GraphQL is an execution semantics. It's very close to a lambda calculus, but I don't think that was by design. I think that came about by accident. A client is really sending a small fragment of code to the server, which the server then executes. The closest thing you have is probably SQL queries: the client sends a query to the server, which the server then executes. It's fundamental to the idea of GraphQL as well. You want to put power into the hands of the client, because that's what allows a top-down approach to UX design. If you always have to manipulate the server-side whenever a client wants to change call structure, you've lost.
      - DenisM 11mo ago
        No one exposes SQL to clients though. I think where Gql differs from sql is it’s at a higher level. SQL bleeds performance and data layout (e.g. normalizing, limits), GraphQL does not. It’s not clear if it’s high enough to abstract knowledge from storage. In the end it’s tension between enabling client to wander around productively vs being a bull in a china shop.
- wrs 11mo ago
  It doesn’t actually require that second part. Every time I’ve used it in a production system, we had an approved list of query shapes that were accepted. If the client wanted to use a new kind of query, it was performance tested and sometimes needed to be optimized before approval for use. If you open it up for any possible query, then give that to uncontrolled clients, it’s a recipe for disaster.
  - kaoD 11mo ago
    Oh, we have that too! But we call it HTTP endpoints.
    - johnfn 11mo ago
      Really? Hmm... where in the HTTP spec does it allow for returning an arbitrary subset of any specific request, rather than the whole thing? And where does it ensure all the results are keyed by id so that you can actually build and update a sensible cache around all of it rather than the mess that totally free-form HTTP responses lead to? Oh weird HTTP doesn't have any of that stuff? Maybe we should make a new spec, something which does allow for these patterns and behaviors? And it might be confusing if we use the exact same name as HTTP, since the usage patterns are different and it enables new abilities. If only we could think of such a name...
      - eli 11mo ago
        An HTTP Range request asks the server to send parts of a resource back to a client. Range requests are useful for various clients, including media players that support random access, data tools that require only part of a large file, and download managers that let users pause and resume a download. https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Range_requests https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Ran...
        867-5309 11mo ago
        also handy for bypassing bandwidth restrictions: capped at 100kbps? launch 1000 workers to grab chunks then assemble the survivors
koakuma-chan 11mo ago
I wish people at least stopped using JavaScript and stopped writing requests to back-end by hand.

deleted 11mo ago

[deleted]

notpachet 11mo ago

Reading this was such an immediate "aha" for me. Of course we should be using GraphQL for this. Damn. Where was this comment three months ago!

bnchrch 11mo ago

1000% 2 years ago I gave a talk on Vector DB's and LLM use. https://www.youtube.com/watch?v=U_g06VqdKUc https://www.youtube.com/watch?v=U_g06VqdKUc TLDR but it shows how you could teach an LLM your GraphQL query language to let it selectively load context into what were very small context windows at the time. After that the MCP specification came out. Which from my vantage point is a poor and half implemented version of what GraphQL already is.

refibrillator 11mo ago

> It works better! > I strongly believe it is one of the best technologies for AI agents Do you have any quantitative evidence to support this? Sincere question. I feel it would add some much needed credibility in a space where many folks are abusing the hype wave and low key shilling their products with vibes instead of rigor.

steveklabnik 11mo ago
I have thought about this for all of thirty seconds, but it wouldn't shock me if this was the case. The intuition here is about types, and the ability to introspect them. Agents really love automated guardrails. It makes sense to me that this would work better than RESTish stuff, even with OpenAPI.
- ibash 11mo ago
  Better than rest is a low bar though. Ultimately agents should rarely be calling raw rest and graphql apis, which are meant for programmatic use. Agents should be calling one level of abstraction higher. Eg calling a function to “find me relevant events in this city according to this users preferences” instead of “list all events in this city”.
- MrDarcy 11mo ago
  Same in terms of time spent. The hypothesis graphql is superior passes the basic sniff test. Assuming graphql does what it says on the tin, which my understanding is it does based on my work with Ent, then the claim it’s better for tool and api use by agents follows from common sense.
- s900mhz 11mo ago
  This is a task I think is suited for a sub agent that is small in size. It can can take the context beating to query for relevant tools and return only what is necessary to the main agent thread.
brulard 11mo ago
If you knew GraphQL, you may immediately see it - you ask for specific nested structure of the data, which can span many joins across different related collections. This is not the case with common REST API or CLI for example. And introspection is another good reason.

ramnivasl 11mo ago

That is also the approach we took with Exograph (https://exograph.dev https://exograph.dev). Here is our reasoning (https://exograph.dev/blog/exograph-now-supports-mcp#comparing-with-alternatives https://exograph.dev/blog/exograph-now-supports-mcp#comparin...). We found that LLMs do a very good job of crafting GraphQL queries for the given schema. While they do make mistakes, returning good descriptive error messages make is easy for them fix queries.

adverbly 11mo ago

This is actually a really good use of graphql! IMO the biggest pain points of graphql are authorization/rate limiting, caching, and mutations... But for selective context loading none of those matter actually. Pretty cool!

esafak 11mo ago

Can anyone recommend an open source GraphQL-based MCP/tool gateway?

bravura 11mo ago

Isn't the challenge that introspecting graphql will lead to either a) a very long set of definitions consuming many tokens or b) many calls to drill into the introspection?

peacebeard 11mo ago
In my experience, this was the limitation we ran into with this approach. If you have a large API this will blow up your context. I have had the best luck with hand-crafted tools that pre-digest your API so you don't have to waste tokens or deal with context rot bugs.
_pdp_ 11mo ago
Well either that or stuff the tool usage examples into the prompt for every single request. If you have only 2-3 tools GraphQL is certainly not necessary - but it wont blow up the context either. If you have 50+ tools, I don't see any other way to be honest, unless you create your own tool discovery solution - which is what GraphQL does really well with the caveat that whatever you decide to do is certainly not natural to these LLMs. Keep in mind that all LLMs are trained on many GraphQL examples because the technology has been in existence since 2015. While anything custom might just work it is certainly not part of the model training set unless you fine-tune. So yes, if I need to decide on formats I will go for GraphQL, SQL and Markdown.

AIorNot 11mo ago

your use-case is NOT Everyones use-case..(working in depth across one codebase or api but instead sampling dozens of abilities across the web or with other systems) thats the thing how is that going to work with my use case, do a web search, do a local api call, do a graphql search, do an integration with slack, do a message etc..

alfonsodev 11mo ago
Does it matter ? if it's well defined, each of those would be a node in the graph, or can you elaborate ? Dozens seems not that much, for a graph where a higher level node would be slack, and the agent only loads further if it needs anything related with slack. Or I'm not understanding.

geoffhill 11mo ago

One of my agents is kinda like this too. The only operation is SPARQL query, and the only accessible state is the graph database. Since most of the ontologies I'm using are public, I just have to namedrop them in prompt; no schemas and little structure introspection needed. At worst, it can just walk and dump triples to figure out structure; it's all RDF triples and URIs. One nice property: using structured outputs, you can constrain outputs of certain queries to only generate valid RDF to avoid syntax errors. Probably can do similar stuff with GraphQL.

fy20 11mo ago

Whoa there, you don't need to be so sadistic to your team. It's not GraphQL, but having a document describing how your API works, including types, that is important. I expect you could achieve the same with a comprehensive OpenAPI specification. If you want something a bit stricter I guess SOAP would work too, LLMs love XML after all.

_pdp_ 11mo ago
We have well described OpenAPI and GraphQL specifications already. :) Being AI-first means we are naturally aligned with that kind of structured documentation. It helps both humans and robots.

deleted 11mo ago

[deleted]

sibeliuss 11mo ago

GraphQL FTW