4 ms·

Show HN: Use LLMs to Interact with APIs

35 points by jsajal1993 2y ago

Recently featured in a LangChain blog https://blog.langchain.dev/empowering-development-with-flowtestai/ https://blog.langchain.dev/empowering-development-with-flowt... , use LLMs to construct an API first runnable workflow with an IDE experience.

xpe 2y ago
Who has analyzed the failure modes of this tool? In particular, what happens when the LLM makes mistakes? Does this lead to malformed or unpredictable API calls? What are the risks?
- deleted 2y ago
  [deleted]
- joshuanapoli 2y ago
  Seems like the AI feature is optional, and it defines a workflow. So the user has a chance to review and debug the workflow before using it, similar to if they had created it manually.
- jsajal1993 2y ago
  Hi, this is the author of the tool here. What joshuanapoli said is absolutely true, "user has a chance to review and debug the workflow before using it" but majorly we have put guardrails that filters out the part of the output that LLM hallucinated or is incorrect
spxneo 2y ago
cool but i dont think anybody on the enterprise is going to use this right now LLM-to-automation-api seems really great on paper but the statistics are simply against any sort of wide adoption at least that is the general consensus speaking to clients
- jsajal1993 2y ago
  Hi spxneo, this is the author of the tool here. Thank you for voicing your opinion. We actually talked to a lot of individuals and we found that particularly people who are part of frontend team or part of a QA/QE or CRE team who are not the primary developers of the backend/api were most interested in the LLM to API workflow feature, partly because going through the OpenAPI spec and understanding each endpoint is a lot of work. What our tool does it that it not only identifies the right endpoints and chain them but also pre-fills the request parameters or request body which gives them a chance to quickly experiment with backend. Of course you always have the option of drag and drop. But we hope that one day using LLMs will get commoditized with models being quite good and then that day a natural language translation will seem obvious Aside from that one of the major problems that we found that not a lot of folks want to expose their api authentication credentials or in-development/private apis to tools like postman or zappier. So giving an IDE like experience that works as a local desktop app helps alleviate a lot of privacy issues. Graphical representation of end to end tests was also a big ask

35 points by jsajal1993 2y ago

xpe 2y ago

Who has analyzed the failure modes of this tool? In particular, what happens when the LLM makes mistakes? Does this lead to malformed or unpredictable API calls? What are the risks?

deleted 2y ago
[deleted]
joshuanapoli 2y ago
Seems like the AI feature is optional, and it defines a workflow. So the user has a chance to review and debug the workflow before using it, similar to if they had created it manually.
jsajal1993 2y ago
Hi, this is the author of the tool here. What joshuanapoli said is absolutely true, "user has a chance to review and debug the workflow before using it" but majorly we have put guardrails that filters out the part of the output that LLM hallucinated or is incorrect

spxneo 2y ago

cool but i dont think anybody on the enterprise is going to use this right now LLM-to-automation-api seems really great on paper but the statistics are simply against any sort of wide adoption at least that is the general consensus speaking to clients

jsajal1993 2y ago
Hi spxneo, this is the author of the tool here. Thank you for voicing your opinion. We actually talked to a lot of individuals and we found that particularly people who are part of frontend team or part of a QA/QE or CRE team who are not the primary developers of the backend/api were most interested in the LLM to API workflow feature, partly because going through the OpenAPI spec and understanding each endpoint is a lot of work. What our tool does it that it not only identifies the right endpoints and chain them but also pre-fills the request parameters or request body which gives them a chance to quickly experiment with backend. Of course you always have the option of drag and drop. But we hope that one day using LLMs will get commoditized with models being quite good and then that day a natural language translation will seem obvious Aside from that one of the major problems that we found that not a lot of folks want to expose their api authentication credentials or in-development/private apis to tools like postman or zappier. So giving an IDE like experience that works as a local desktop app helps alleviate a lot of privacy issues. Graphical representation of end to end tests was also a big ask