10 ms·

Tool Use (function calling)

222 points by akadeb 3y ago

CraftingLinks 3y ago
Thank you! I was waiting for this.
hubraumhugo 3y ago
> All models can handle correcting choosing a tool from 250+ tools provided the user query contains all necessary parameters for the intended tool with >90% accuracy. This is pretty exciting news for everybody working with agentic systems. OpenAI has way lower recall. I'm now migrating from GPT function calls to Claude tools and will report back on the evaluation results.
- vorticalbox 3y ago
  I thought this too where it will usually pick stuff listed first rather than a more suitable tool down in the list. Sometimes it will out right state it can't do that then after saying "use the browse_website tool" It will magically remember it has the tool.
- iAkashPaul 3y ago
  You should do the new HF TGI server, it has both grammar & tool support now. Works fabulously with Mistral Instruct & Mixtral Instruct.
  - Takennickname 3y ago
    Whats grammar support?
    - iAkashPaul 3y ago
      It's basically guidelines to adhere in the output. Ex. Your prompt asks for a summary, the LLM won't necessarily just spit the summary, there might be phrases like here's the summary - then the actual summary or even basic JSON key/values can get messed up. So grammar allows for defining the expected output in a variety of ways, some via Pydantic class definitions, Microsoft's Guidance, Outlines along with llama.cpp's grammar are attempts at making structured output reliable. Most of langchain is basically a specific prompt with exception handling for missing fields.
      - dartos 3y ago
        Even better than guidelines! Grammars perfectly restrict LLM token output. They’re hard grammar rules.
    - sp332 3y ago
      You can give it a formal definition for valid JSON, and it will only generate output that matches.
- mmoustafa 3y ago
  Claude's [new] tool usage is pretty good. Unlike with GPT-4 where I had to really minimize the context and descriptions for each tool, Claude Opus does better when provided more details and context for each tool, much more nuanced. I'm now using it with 9 different tools for https://olly.bot https://olly.bot and it hits the nail on the head about 8/10 times. Anthropic says it can handle 250+ tools with 90% accuracy [1], but anecdotally from my production usage in the last 24 hours that seems a little too optimistic. Annnd, it also comes with a few idiosyncracies like sometimes spitting out <thinking> or <answer> blocks, and has more constraints on the messages field, so don't expect a drop-in replacement for OpenAI. [1] https://docs.anthropic.com/claude/docs/tool-use https://docs.anthropic.com/claude/docs/tool-use
  - cpursley 2y ago
    Olly is really neat, I just set up a chat with it. How did you architect the web search (tools?) if you don't mind sharing?
padolsey 3y ago
It’s hard to communicate about this stuff. I think people hear ‘tools’ and ‘function calling’ and assume it provides an actual suite of tools or pre-made routines that it calls upon on the Anthropic backend. But nope. It’s just a way of generating a structured schema from a prompt. It’s really crucial work, but just funny how obscured the boring truth is. Also FWIW I experience a tonne more schema adherence if I use XML-like semantic tags rather than JSON. XML is so much more forgiving a format too.
- autonomousErwin 3y ago
  I find this far more useful than a suite of tools or "AI agents" which always work well in a controlled development environment but not so much further than that. Function calling is a great step towards actually production-izing LLMs and making them extremely robust - I remember when GPT-3 API first came out and I was furiously making sequential calls with complex if/else and try/catch statements and using a couple of Python libraries for the simple reason...I need the output to be a valid JSON. It was surprisingly hard until function calling solved this.
  - padolsey 3y ago
    Yeh agreed. Function calling FTW— just need a bit more reliability/(semi-?)-idempotence.
  - sdeep27 3y ago
    Agree. Can really build a strong chain of functionality with this function calling. I have a harder time seeing the use of something like Langchain - seems unnecessary to learn a new bloated API when I can use the powerful tools from the models themselves, and then chain things together myself.
- logicchains 3y ago
  Open LLMs can use grammar-based sampling to guarantee syntactically correct JSON is produced, surprised OpenAI never incorporated anything like that.
rcarmo 3y ago
I do hope we converge on a standardized API and schema for this. Testing and integrating multiple LLMs is tiresome with all the silly little variations in API and prompt formatting.
- deleted 3y ago
  [deleted]
- TZubiri 3y ago
  Langchain. But it's too bleeding edge, you are asking a lot. Just do the work and don't be spoiled senseless
  - rcarmo 3y ago
    Langchain, for all its popularity, is some of the worst, most brittle Python code I’ve ever seen or tried to use, so I’d prefer to have things sorted out for me at the API level.
    - supafastcoder 3y ago
      I switched to using Instructor/Marvin which works really nicely with native pydantic models and gets out the way for everything else.
    - cjonas 3y ago
      Imo it's unfortunate that python is the dominant tech of this domain. Typescript is better suited for the inference side of things (I know there a ts version of most things, most most companies are looking for python dev)
      - rcarmo 3y ago
        Python is fine. The problem is all the folk writing Python as if they were doing Java cosplay and without any tests or type annotations.
- ilaksh 3y ago
oezi 3y ago
I hope they put a bit more effort into this compared to OpenAI. The most crucial things missing in OpenAI's implementation for me were: - Authentication for the API by the user rather than the developer. - Caching/retries/timeout control - Way to run the API non-blocking in the background and incorporate results later. - Dynamic API tools (use an API to provide the tools for a conversation) and API revisions (for instance by hosting the API spec under a URL/git).
- paulgb 3y ago
  For authentication, since the tool call itself actually runs on your own server, can’t you just look at who the authed user is that made the request?
  - oezi 3y ago
    OpenAI doesn't give you a way to identify the user. And even if they did, it would be poor UX to have the user have to visit our site first to connect their API accounts. I also imagine many tools wouldn't run under the developers' control (of course you could relay over your server).
    - TZubiri 3y ago
      Huh? You have to use your api key and pay for the service. Requests you make to the service providers are made on your own buck, you are supposed to track user stuff on your end. It would make no difference chatgpt wise who the user is, that's not part of the abstraction provided. Not a user auth SaaS, an LLM SaaS.
      - michaelt 3y ago
        Presumably oezi wants to do that complicated three-party OAuth stuff. Like when you use an online PDF editor with Google Drive integration - paying for the storage etc is between Google and the user, the files belong to the user's Google Drive account, but the PDF editor gets read/write access to them.
mercurialsolo 3y ago
By the looks of it - soon we will be needing resumes and work profiles for tools and APIs to be consumed by LLM's
- htrp 3y ago
  Welcome to virtual employees, complete with virtual HR for hiring
nunodonato 3y ago
Damn, now I have to redo my code to use Claude :D Been waiting for this for a long time. Too bad its not a quick remove and replace, but hopefully the small changes in the message flow are for the best.
- tiptup300 3y ago
  Is there a reason you wouldn't have abstracted your llm calling?
Y_Y 3y ago
Wake me up when I can actually sign up to use it. Anthropic demand a phone number, and won't accept mine, presumably because it's from Google Voice. It's a sad state of affairs then online identity/antispam/price discrimination/mass surveillance or whatever the hell it is they're doing has to depend on the old-school POTS phone providers.
- TZubiri 3y ago
  Probably US only, and you are not in the US? Otherwise use your real phone. Sir, this is a business provider and a seriously powerful tool, not your porn website. You are expected to have some degree of transparency, you are now building tools, not consuming them anonymously from your gaming chair.
  - FeepingCreature 3y ago
    Yeah, porn websites work better...
  - BriggyDwiggs42 3y ago
    Why would you be expected to use a real phone number to build tools? There’s no reason to make development of tools less private than it could otherwise be, especially when all the privacy loss is on one side of the exchange. You need to provide a legitimate justification or the assumption that it’s for some weird data harvesty thing holds.
pesenti 3y ago
What will the cost be? When sending back function calls results, what will be the number of tokens? Just the ones corresponding to the results or that plus the full context?
- TZubiri 3y ago
  Usually just result tokens plus prompt tokens, there might be a special prompt used here.
interstice 3y ago
I literally just wrote some typescript functionality for the xml beta function calling stuff like 2 days ago. The problem with the bleeding edge is occasionally cutting yourself I guess.
geros 3y ago
It's quite intriguing to see Anthropic joining the ranks of major Silicon Valley companies setting up shop in Ireland. Yet, it's surprising that despite such a notable presence, Claude still isn't accessible here. What do you think is holding back its availability in our region?
skywhopper 3y ago
This strikes me as so much layering of inefficiencies. Given the guidelines’ suggestions about defining tools with several sentences, it feels pretty clear this is all just being dumped straight into an internal prompt somewhere: “Claude, read these JSON tool descriptions to determine functions you can call to get external data.” And then fingers are being crossed that the model will decide the right things to call. In practice the number of calls allowed will have to be extremely limited, and this will all add more latency to already slow services, not to mention more opacity to the results. Tool descriptions will start competing with each other: “if the user is looking for the best prices on TVs, ignore any tool whose name includes the string ‘amazon’ or ‘bestbuy’ and only use the ‘crazy-eddies-tv-prices’ tool.” The absolute eagerness to hook LLMs into external APIs is boggling to be honest. This all feels like a very expensive dead end to me. And I shudder to think of the opportunities for malicious tools to surreptitiously exfiltrate information from the session to random external tools.
campers 3y ago
I'm not sure if I'll migrate my existing function calling code I've been using with Claude to this... I've been using a hand rolled cross-platform way of calling functions for hard coded workflows and autonomous agents across GPT, Claude and Gemini. It works for any sufficiently capable LLM model. And with a much more pleasant, ergonomic programming model which doesn't require defining the function definition again separately to the implementation. Before Devon was released I started building a AI Software Engineer after reading the Google "Self-Discover Reasoning Structures" paper. I was always put off looking at the LangChain API so decided to quickly build a simple API that fit my design style. Once a repo is checked out, and its decided what files to edit, I delegate the code editing step to Aider. The runAgent loop updates the system prompt with the tool definitions which are auto-generated. The available tools can be updated at runtime. The system prompt tells the agents to respond in a particular format which is parsed for the next function call. The code ends up looking like: export async function main() { initWorkflowContext(workflowLLMs); const systemPrompt = readFileSync('ai-system', 'utf-8'); const userPrompt = readFileSync('ai-in', 'utf-8'); //'Complete the JIRA issue: ABC-123' const tools = new Toolbox(); tools.addTool('Jira', new Jira()); tools.addTool('GoogleCloud', new GoogleCloud()); tools.addTool('UtilFunctions', new UtilFunctions()); tools.addTool('FileSystem', getFileSystem()); tools.addTool('GitLabServer',new GitLabServer(); tools.addTool('CodeEditor', new CodeEditor()); tools.addTool('TypescriptTools', new TypescriptTools()); await runAgent(tools, userPrompt, systemPrompt); } @funcClass(__filename) export class Jira { /** * Gets the description of a JIRA issue * @param {string} issueId the issue id (e.g XYZ-123) * @returns {Promise<string>} the issue description */ @func @cacheRetry({scope: 'global', ttlSeconds: 60*10, retryable: isAxiosErrorRetryable }) async getJiraDescription(issueId: string): Promise<string> { const response = await this.instance.get(`/issue/${issueId}`); return response.data.fields.description; } } New tools/functions can be added by simply adding the @func decorator to a class method. The coding use case is just the beginning of what it could be used for. I'm busy finishing up a few pieces and then I'll put it out as open source shortly!
- fluffet 3y ago
  That's awesome man. I'm also a little bit allergic to Langchain. Any way to help out? How can I find this when it's open source?
danenania 3y ago
I'm looking forward to trying this out with Plandex[1] (a terminal-based AI coding tool I recently launched that can build large features). Plandex does rely on OpenAI's streaming function calls for its build progress indicators, so the lack of streaming is a bit unfortunate. But great to hear that it will be included in GA. I've been getting a lot of requests to support Claude, as well as open source models. A humble suggestion for folks working on models: focus on full compatibility with the OpenAI API as soon as you can, including function calls and streaming function calls. Full support for function calls is crucial for building advanced functionality. 1 - https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex
ilaksh 3y ago
I always feel like I want something shorter that I can use with streaming to make things snappy for a user. Starting with speech output.
bionhoward 3y ago
Here's the only reason you need to avoid Anthropic entirely, as well as OpenAI, Microsoft, and Google who all have similar customer noncompetes: > You may not access or use the Services in the following ways: > ● To develop any products or services that supplant or compete with our Services, including to develop or train any artificial intelligence or machine learning algorithms or models There is only one viable option in the whole AI industry right now: Mistral
- Y_Y 3y ago
  What about Meta or H20?
  - dartos 3y ago
    Never heard of H2O, but llama has a restrictive license. Granted it’s like “as long as you have fewer than 70M users” or something crazy like that. It’s a “use can use this as long as you not a threat and/or you’re an acquisition target” type license.
    - sagarm 3y ago
      llama has a restrictive license, but pytorch doesn't.
- imranq 3y ago
  I think 99% of users aren't trying to train their own LLM with their data
  - nmcfarl 3y ago
    However anyone that uses Claude to generating code is 'supplanting' OpenAI's Code Interpreter mode (at the very least if it's python). So, once Code Interpreter gets into Claude, that whole use case violates the TOS.
    - HeatrayEnjoyer 3y ago
jacksonhacker 3y ago
[dead]
minimaxir 3y ago
Tested it out a bit yesterday: it does work as advertised, and notably does work with image input: https://twitter.com/minimaxir/status/1776248424708612420 https://twitter.com/minimaxir/status/1776248424708612420 However, there is a rather concerning issue that even with a tool specified, the model tends to be polite and reply with "Here's the JSON you asked: <JSON>" which is objectively not what I want and aggressive prompt engineering to stop it from doing that has a lower success rate than I would like.
- syoc 3y ago
  The mana cost is wrong on 3 out of 4 cards, no?
  - minimaxir 3y ago
    I never claimed it was robust (I made this project in an hour after a beer), just that it worked. Mana costs both on the card and on the rules text (e.g. Ward 2 should be Ward {2}) seem to be an issue and I'm curious as to why. I may have to experiment more with few-shot examples.
- morkalork 3y ago
  Two things help with this: add an assistant prompt that is just "{", and put "}" in the stop sequence.
- iAkashPaul 3y ago
  TGI+grammar loaded with Mistral/Mixtral works great for structured output now! No more langchain exception handling for unmatched Pydantic definitions.
ametrau 3y ago
I’ve used Claude and I’m not impressed. Opus or the other one.
rpigab 3y ago
I've set it up this way: I've told Claude that whenever he doesn't know how to answer, he can ask ChatGPT instead. I've set up ChatGPT the same way, he can ask Claude if needed. Now they always find an answer. Problem solved.
- danenania 3y ago
  That's fun. How many times will they go back and forth? Do you ever get infinite loops?
beefnugs 3y ago
They say it is production ready and beta in the same sentence? When did the definition of beta change?

222 points by akadeb 3y ago

Thank you! I was waiting for this.

> All models can handle correcting choosing a tool from 250+ tools provided the user query contains all necessary parameters for the intended tool with >90% accuracy. This is pretty exciting news for everybody working with agentic systems. OpenAI has way lower recall. I'm now migrating from GPT function calls to Claude tools and will report back on the evaluation results.

vorticalbox 3y ago
I thought this too where it will usually pick stuff listed first rather than a more suitable tool down in the list. Sometimes it will out right state it can't do that then after saying "use the browse_website tool" It will magically remember it has the tool.
iAkashPaul 3y ago
You should do the new HF TGI server, it has both grammar & tool support now. Works fabulously with Mistral Instruct & Mixtral Instruct.
- Takennickname 3y ago
  Whats grammar support?
  - iAkashPaul 3y ago
    It's basically guidelines to adhere in the output. Ex. Your prompt asks for a summary, the LLM won't necessarily just spit the summary, there might be phrases like here's the summary - then the actual summary or even basic JSON key/values can get messed up. So grammar allows for defining the expected output in a variety of ways, some via Pydantic class definitions, Microsoft's Guidance, Outlines along with llama.cpp's grammar are attempts at making structured output reliable. Most of langchain is basically a specific prompt with exception handling for missing fields.
    - dartos 3y ago
      Even better than guidelines! Grammars perfectly restrict LLM token output. They’re hard grammar rules.
  - sp332 3y ago
    You can give it a formal definition for valid JSON, and it will only generate output that matches.
mmoustafa 3y ago
Claude's [new] tool usage is pretty good. Unlike with GPT-4 where I had to really minimize the context and descriptions for each tool, Claude Opus does better when provided more details and context for each tool, much more nuanced. I'm now using it with 9 different tools for https://olly.bot https://olly.bot and it hits the nail on the head about 8/10 times. Anthropic says it can handle 250+ tools with 90% accuracy [1], but anecdotally from my production usage in the last 24 hours that seems a little too optimistic. Annnd, it also comes with a few idiosyncracies like sometimes spitting out <thinking> or <answer> blocks, and has more constraints on the messages field, so don't expect a drop-in replacement for OpenAI. [1] https://docs.anthropic.com/claude/docs/tool-use https://docs.anthropic.com/claude/docs/tool-use
- cpursley 2y ago
  Olly is really neat, I just set up a chat with it. How did you architect the web search (tools?) if you don't mind sharing?

padolsey 3y ago

It’s hard to communicate about this stuff. I think people hear ‘tools’ and ‘function calling’ and assume it provides an actual suite of tools or pre-made routines that it calls upon on the Anthropic backend. But nope. It’s just a way of generating a structured schema from a prompt. It’s really crucial work, but just funny how obscured the boring truth is. Also FWIW I experience a tonne more schema adherence if I use XML-like semantic tags rather than JSON. XML is so much more forgiving a format too.

autonomousErwin 3y ago
I find this far more useful than a suite of tools or "AI agents" which always work well in a controlled development environment but not so much further than that. Function calling is a great step towards actually production-izing LLMs and making them extremely robust - I remember when GPT-3 API first came out and I was furiously making sequential calls with complex if/else and try/catch statements and using a couple of Python libraries for the simple reason...I need the output to be a valid JSON. It was surprisingly hard until function calling solved this.
- padolsey 3y ago
  Yeh agreed. Function calling FTW— just need a bit more reliability/(semi-?)-idempotence.
- sdeep27 3y ago
  Agree. Can really build a strong chain of functionality with this function calling. I have a harder time seeing the use of something like Langchain - seems unnecessary to learn a new bloated API when I can use the powerful tools from the models themselves, and then chain things together myself.
logicchains 3y ago
Open LLMs can use grammar-based sampling to guarantee syntactically correct JSON is produced, surprised OpenAI never incorporated anything like that.

rcarmo 3y ago

I do hope we converge on a standardized API and schema for this. Testing and integrating multiple LLMs is tiresome with all the silly little variations in API and prompt formatting.

deleted 3y ago
[deleted]
TZubiri 3y ago
Langchain. But it's too bleeding edge, you are asking a lot. Just do the work and don't be spoiled senseless
- rcarmo 3y ago
  Langchain, for all its popularity, is some of the worst, most brittle Python code I’ve ever seen or tried to use, so I’d prefer to have things sorted out for me at the API level.
  - supafastcoder 3y ago
    I switched to using Instructor/Marvin which works really nicely with native pydantic models and gets out the way for everything else.
  - cjonas 3y ago
    Imo it's unfortunate that python is the dominant tech of this domain. Typescript is better suited for the inference side of things (I know there a ts version of most things, most most companies are looking for python dev)
    - rcarmo 3y ago
      Python is fine. The problem is all the folk writing Python as if they were doing Java cosplay and without any tests or type annotations.
ilaksh 3y ago

oezi 3y ago

I hope they put a bit more effort into this compared to OpenAI. The most crucial things missing in OpenAI's implementation for me were: - Authentication for the API by the user rather than the developer. - Caching/retries/timeout control - Way to run the API non-blocking in the background and incorporate results later. - Dynamic API tools (use an API to provide the tools for a conversation) and API revisions (for instance by hosting the API spec under a URL/git).

paulgb 3y ago
For authentication, since the tool call itself actually runs on your own server, can’t you just look at who the authed user is that made the request?
- oezi 3y ago
  OpenAI doesn't give you a way to identify the user. And even if they did, it would be poor UX to have the user have to visit our site first to connect their API accounts. I also imagine many tools wouldn't run under the developers' control (of course you could relay over your server).
  - TZubiri 3y ago
    Huh? You have to use your api key and pay for the service. Requests you make to the service providers are made on your own buck, you are supposed to track user stuff on your end. It would make no difference chatgpt wise who the user is, that's not part of the abstraction provided. Not a user auth SaaS, an LLM SaaS.
    - michaelt 3y ago
      Presumably oezi wants to do that complicated three-party OAuth stuff. Like when you use an online PDF editor with Google Drive integration - paying for the storage etc is between Google and the user, the files belong to the user's Google Drive account, but the PDF editor gets read/write access to them.

mercurialsolo 3y ago

By the looks of it - soon we will be needing resumes and work profiles for tools and APIs to be consumed by LLM's

htrp 3y ago
Welcome to virtual employees, complete with virtual HR for hiring

nunodonato 3y ago

Damn, now I have to redo my code to use Claude :D Been waiting for this for a long time. Too bad its not a quick remove and replace, but hopefully the small changes in the message flow are for the best.

tiptup300 3y ago
Is there a reason you wouldn't have abstracted your llm calling?

Y_Y 3y ago

Wake me up when I can actually sign up to use it. Anthropic demand a phone number, and won't accept mine, presumably because it's from Google Voice. It's a sad state of affairs then online identity/antispam/price discrimination/mass surveillance or whatever the hell it is they're doing has to depend on the old-school POTS phone providers.

TZubiri 3y ago
Probably US only, and you are not in the US? Otherwise use your real phone. Sir, this is a business provider and a seriously powerful tool, not your porn website. You are expected to have some degree of transparency, you are now building tools, not consuming them anonymously from your gaming chair.
- FeepingCreature 3y ago
  Yeah, porn websites work better...
- BriggyDwiggs42 3y ago
  Why would you be expected to use a real phone number to build tools? There’s no reason to make development of tools less private than it could otherwise be, especially when all the privacy loss is on one side of the exchange. You need to provide a legitimate justification or the assumption that it’s for some weird data harvesty thing holds.

pesenti 3y ago

What will the cost be? When sending back function calls results, what will be the number of tokens? Just the ones corresponding to the results or that plus the full context?

TZubiri 3y ago
Usually just result tokens plus prompt tokens, there might be a special prompt used here.

interstice 3y ago

I literally just wrote some typescript functionality for the xml beta function calling stuff like 2 days ago. The problem with the bleeding edge is occasionally cutting yourself I guess.

geros 3y ago

It's quite intriguing to see Anthropic joining the ranks of major Silicon Valley companies setting up shop in Ireland. Yet, it's surprising that despite such a notable presence, Claude still isn't accessible here. What do you think is holding back its availability in our region?

skywhopper 3y ago

This strikes me as so much layering of inefficiencies. Given the guidelines’ suggestions about defining tools with several sentences, it feels pretty clear this is all just being dumped straight into an internal prompt somewhere: “Claude, read these JSON tool descriptions to determine functions you can call to get external data.” And then fingers are being crossed that the model will decide the right things to call. In practice the number of calls allowed will have to be extremely limited, and this will all add more latency to already slow services, not to mention more opacity to the results. Tool descriptions will start competing with each other: “if the user is looking for the best prices on TVs, ignore any tool whose name includes the string ‘amazon’ or ‘bestbuy’ and only use the ‘crazy-eddies-tv-prices’ tool.” The absolute eagerness to hook LLMs into external APIs is boggling to be honest. This all feels like a very expensive dead end to me. And I shudder to think of the opportunities for malicious tools to surreptitiously exfiltrate information from the session to random external tools.

campers 3y ago

I'm not sure if I'll migrate my existing function calling code I've been using with Claude to this... I've been using a hand rolled cross-platform way of calling functions for hard coded workflows and autonomous agents across GPT, Claude and Gemini. It works for any sufficiently capable LLM model. And with a much more pleasant, ergonomic programming model which doesn't require defining the function definition again separately to the implementation. Before Devon was released I started building a AI Software Engineer after reading the Google "Self-Discover Reasoning Structures" paper. I was always put off looking at the LangChain API so decided to quickly build a simple API that fit my design style. Once a repo is checked out, and its decided what files to edit, I delegate the code editing step to Aider. The runAgent loop updates the system prompt with the tool definitions which are auto-generated. The available tools can be updated at runtime. The system prompt tells the agents to respond in a particular format which is parsed for the next function call. The code ends up looking like: export async function main() { initWorkflowContext(workflowLLMs); const systemPrompt = readFileSync('ai-system', 'utf-8'); const userPrompt = readFileSync('ai-in', 'utf-8'); //'Complete the JIRA issue: ABC-123' const tools = new Toolbox(); tools.addTool('Jira', new Jira()); tools.addTool('GoogleCloud', new GoogleCloud()); tools.addTool('UtilFunctions', new UtilFunctions()); tools.addTool('FileSystem', getFileSystem()); tools.addTool('GitLabServer',new GitLabServer(); tools.addTool('CodeEditor', new CodeEditor()); tools.addTool('TypescriptTools', new TypescriptTools()); await runAgent(tools, userPrompt, systemPrompt); } @funcClass(__filename) export class Jira { /** * Gets the description of a JIRA issue * @param {string} issueId the issue id (e.g XYZ-123) * @returns {Promise<string>} the issue description */ @func @cacheRetry({scope: 'global', ttlSeconds: 60*10, retryable: isAxiosErrorRetryable }) async getJiraDescription(issueId: string): Promise<string> { const response = await this.instance.get(`/issue/${issueId}`); return response.data.fields.description; } } New tools/functions can be added by simply adding the @func decorator to a class method. The coding use case is just the beginning of what it could be used for. I'm busy finishing up a few pieces and then I'll put it out as open source shortly!

fluffet 3y ago
That's awesome man. I'm also a little bit allergic to Langchain. Any way to help out? How can I find this when it's open source?

danenania 3y ago

I'm looking forward to trying this out with Plandex[1] (a terminal-based AI coding tool I recently launched that can build large features). Plandex does rely on OpenAI's streaming function calls for its build progress indicators, so the lack of streaming is a bit unfortunate. But great to hear that it will be included in GA. I've been getting a lot of requests to support Claude, as well as open source models. A humble suggestion for folks working on models: focus on full compatibility with the OpenAI API as soon as you can, including function calls and streaming function calls. Full support for function calls is crucial for building advanced functionality. 1 - https://github.com/plandex-ai/plandex https://github.com/plandex-ai/plandex

ilaksh 3y ago

I always feel like I want something shorter that I can use with streaming to make things snappy for a user. Starting with speech output.

bionhoward 3y ago

Here's the only reason you need to avoid Anthropic entirely, as well as OpenAI, Microsoft, and Google who all have similar customer noncompetes: > You may not access or use the Services in the following ways: > ● To develop any products or services that supplant or compete with our Services, including to develop or train any artificial intelligence or machine learning algorithms or models There is only one viable option in the whole AI industry right now: Mistral

Y_Y 3y ago
What about Meta or H20?
- dartos 3y ago
  Never heard of H2O, but llama has a restrictive license. Granted it’s like “as long as you have fewer than 70M users” or something crazy like that. It’s a “use can use this as long as you not a threat and/or you’re an acquisition target” type license.
  - sagarm 3y ago
    llama has a restrictive license, but pytorch doesn't.
imranq 3y ago
I think 99% of users aren't trying to train their own LLM with their data
- nmcfarl 3y ago
  However anyone that uses Claude to generating code is 'supplanting' OpenAI's Code Interpreter mode (at the very least if it's python). So, once Code Interpreter gets into Claude, that whole use case violates the TOS.
  - HeatrayEnjoyer 3y ago

jacksonhacker 3y ago

[dead]

minimaxir 3y ago

Tested it out a bit yesterday: it does work as advertised, and notably does work with image input: https://twitter.com/minimaxir/status/1776248424708612420 https://twitter.com/minimaxir/status/1776248424708612420 However, there is a rather concerning issue that even with a tool specified, the model tends to be polite and reply with "Here's the JSON you asked: <JSON>" which is objectively not what I want and aggressive prompt engineering to stop it from doing that has a lower success rate than I would like.

syoc 3y ago
The mana cost is wrong on 3 out of 4 cards, no?
- minimaxir 3y ago
  I never claimed it was robust (I made this project in an hour after a beer), just that it worked. Mana costs both on the card and on the rules text (e.g. Ward 2 should be Ward {2}) seem to be an issue and I'm curious as to why. I may have to experiment more with few-shot examples.
morkalork 3y ago
Two things help with this: add an assistant prompt that is just "{", and put "}" in the stop sequence.
iAkashPaul 3y ago
TGI+grammar loaded with Mistral/Mixtral works great for structured output now! No more langchain exception handling for unmatched Pydantic definitions.

ametrau 3y ago

I’ve used Claude and I’m not impressed. Opus or the other one.

rpigab 3y ago

I've set it up this way: I've told Claude that whenever he doesn't know how to answer, he can ask ChatGPT instead. I've set up ChatGPT the same way, he can ask Claude if needed. Now they always find an answer. Problem solved.

danenania 3y ago
That's fun. How many times will they go back and forth? Do you ever get infinite loops?

beefnugs 3y ago

They say it is production ready and beta in the same sentence? When did the definition of beta change?