13 ms·

Function calling and other API updates

• melovy 3y ago
  openai function call + langchain for SQL query with vector search: https://docs.myscale.com/en/sample-applications/openai-function-call/ https://docs.myscale.com/en/sample-applications/openai-funct... https://github.com/myscale/examples/blob/main/openai_function_call.ipynb https://github.com/myscale/examples/blob/main/openai_functio...
• eminence32 3y ago
  The changes/improvements to `gpt-3.5-turbo` are very welcome, considering how hard it is to get access to gpt-4
  • ignoramous 3y ago
    The blog post indicates that GPT-4 (0613) will be generally available soon. gpt-4-32k-0613 includes an extended context length for better comprehension of larger texts. With these updates, we'll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model.
• wilg 3y ago
  LLMs orchestrating, pulling data from, and coordinating between existing systems in this manner seems very powerful. I feel like we haven't even really seen many of the possibilities there.
  • voiper1 3y ago
    ... or the hacking. They mention exploring the security implications.
    • wilg 3y ago
      I'm not living in fear of that.
      • galleywest200 3y ago
        Not fear, sounds like fun.
• demarq 3y ago
  > 20 pages of text in a single request. Yessssssss!
• aarkay 3y ago
  Function calling is a great feature. I've been using LLMs for function calls for the past few months and gpt-4 has worked great for this out of the box. Awesome to see both the models specifically trained for this.
• minimaxir 3y ago
  The big feature here is the function calls, as this is effectively a replacement for the "Tools" feature of Agents popularized by LangChain, except in theory much more efficient since it may not require an extra call to the API. In the case of LangChain which selects Tools and their functional outputs through JSON Markdown shennanigans (which often fails and causes ParsingErrors), this variant of ChatGPT appears to be finetuned for it so perhaps it'll be more reliable. While developing a more-simple LangChain alternative (https://github.com/minimaxir/simpleaichat https://github.com/minimaxir/simpleaichat) I discovered a neat trick for allowing ChatGPT to select tools from a list reliably: put the list of tools into a numbered list, and force the model to return only a single number by using the logit_bias parameter: https://github.com/minimaxir/simpleaichat/blob/main/PROMPTS.md#tools https://github.com/minimaxir/simpleaichat/blob/main/PROMPTS.... The slight price drop for ChatGPT inputs is of course welcome, since inputs are the bulk of the costs for longer conversations. A 4x context window at 2x the price is a good value too. The notes for the updated ChatGPT also say "more reliable steerability via the system message" which will also be huge if it works as advertised.
  • misterdata 3y ago
    As they are accepting a JSON schema for the function calls, it is likely they are using token biasing based on the schema (using some kind of state machine that follows along with the tokens and only allows the next token to be a valid one given the grammar/schema). I have successfully implemented this for JSON Schema (limited subset) on llama.cpp. See also e.g. this implementation: https://github.com/1rgs/jsonformer https://github.com/1rgs/jsonformer
    • newhouseb 3y ago
      As someone also building constrained decoders against JSON [1], I was hopeful to see the same but I note the following from their documentation: The model can choose to call a function; if so, the content will be a stringified JSON object adhering to your custom schema (note: the model may generate invalid JSON or hallucinate parameters). So sadly, it is just fine tuning. There's no hard biasing applied :(. You were so close, but so far OpenAI! [1] https://github.com/newhouseb/clownfish https://github.com/newhouseb/clownfish [2] https://platform.openai.com/docs/guides/gpt/function-calling https://platform.openai.com/docs/guides/gpt/function-calling
• Escapado 3y ago
  This is great. I wonder if the price decrease comes from the competition (on the side of anthropic and from local LLMs). If so, I guess we will have to wait for a general GPT-4 competitor to come along before we see price decreases there aswell. Right now it's quiet expensive. We are incorporating it in a new product in the education space and we have to be fairly conservative in rate limiting things so that the cost won't go out of hand. I also wonder how much of an impact the new Nvidia HGX systems will have on the medium term infra cost on running these services and whether we will see some benefits from that.
  • ilaksh 3y ago
    Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4. I guess it just uses a lot of GPU and RAM.
    • kgwgk 3y ago
      Why would they reduce the price when they have a waiting list?
    • dragonwriter 3y ago
      > Right this is a very exciting release but disappointing that there was no price reduction at all or rate limit increase for gpt-4 They are planning on reducing the pricing from $infinite to $current-listed (or, viewed another way, to increase the quota from 0 to current-listed) by clearing the waiting list. This, obviously, doesn’t benefit (may even, competitively, hurt) those who already have GPT-4 access, but for everyone else, its a win.
• deleted 3y ago
  [deleted]
• bioemerl 3y ago
  I hate seeing these guys succeed because everyone of their successes is a new day that AI becomes less accessible to the average person and more locked behind their APIs.
  • sashank_1509 3y ago
    I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65B model at a very slow pace. Let us say we do have the model weights to GPT-4 and GPT3.5, me as the average consumer I don't know how this helps me in any way. I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference, and even then it will be a slow, unpolished experience. On the other hand OpenAI’s API makes things blazingly fast and dirt cheap to the average consumer. It honestly does feel like they have enabled the power of AI to be accessible to anyone with a laptop. If that requires fending off competition from Behemoths like Google, Meta by not releasing model weights then so be it. This critique would be more apt to Nvidia who are artificially increasing datacenter GPU prices thus pricing out the average consumer. OpenAI is doing the opposite.
    • hackernewds 3y ago
      An "average" person is not someone who knows how to call an API. Perhaps only on HN
      • IMTDb 3y ago
        Then an "average" person is certainly not someone who is able to download and run an LLM on their device.
      • rolisz 3y ago
        If they don't know how to call an API, they won't know how to run local models (at the moment it's quite a pain to set-up all the dependencies)
        • 3y ago
• com2kid 3y ago
  GPT3.5 has been undergoing constant improvements, this price decrease (and context length increase) is great news! The main problem I see with people using GPT3.5 is they try and ask it to "write a short story about aliens" and then they get back a crap boring response that sounds like it was written by an AI that was asleep at the wheel. Good creative prompts are long and detailed, and to get the best results you really need to be able to tune temperature / top_p. Even small changes to a 3 paragraph prompt can result in a dramatic changes in the output, and unless people are willing to play around with prompting, they won't get good results. None of the prompt guides I've seen really cover pushing GPT3.5 to its limit, I've published one of my more complicated prompts[1] but getting GPT3.5 to output good responses in just this limited sense has taken a lot of work. As for the longer context, output length is different than following instructions, especially for a lot of use cases, pushing more input tokens is of as much interest as having more output tokens. From what I have explored, even at 4k context length, with a detailed prompt earlier instructions in the prompt are "forgotten" (or maybe just ignored). The blog post calls out better understanding of input text, but again, I hope that isn't orthogonal to following instructions! Finally in regards to function outputs, I wonder if it is a second layer they are running on top of the initial model output. I have always had a challenge getting the model to output parsable responses, there is a definite trade off between written creativity and well formatted responses, and to some extent having a creative AI extend out the format I specify has been really nice because it has allowed me to add features I did not think of myself! [1] https://github.com/devlinb/arcadia/blob/main/backend/src/routes/promptstreamingws.ts#L133 https://github.com/devlinb/arcadia/blob/main/backend/src/rou...
  • RugnirViking 3y ago
    I just tell it how to write a good story before asking for a story (show, don't tell. Don't list descriptions each time a new thing appears, instead let it become apparent. Withhold information from the reader to build tension, hint and further lore and be creative with your world building) etc, maybe I'll come back and publish some of my prompts in full but I'm getting great results
  • hackernewds 3y ago
    Would you mind sharing what are then good prompt structures? Seems you have a grasp
• civilitty 3y ago
  Finally! I’ve been getting the shakes waiting for next OpenAI release. 16k context with 3.5-turbo is huge. It’ll make all those dime a dozen document driven assistants a lot more useful. I’m curious to see if people will figure out ways to hack functions to get more reliable structured JSON data out out of GPT without tons of examples, giving lots more context room to play with
  • SkyPuncher 3y ago
    This is awesome. We were finding a lot of frustrating with 4k context being far too short to properly chunk documents. In a worst case scenario, you have to assume that output is going to be the same length as input. That means useful context is actually half of the total context. Add in a bit of fixed size for chunking/overlap (maybe ~500 tokens), suddenly you're looking at only 1k to 1.5k being reliably available for input. 16k context bumps that number up to 7.5k available for input. That's massive.
  • Dowwie 3y ago
    Can you provide some examples of what document driven assistants you're referring to?
    • N3cr0ph4g1st 3y ago
      I'm assuming something like humata.ai
• jumploops 3y ago
  This seems like a direct result of Plugins not hitting PMF -- rather than give API developers access to Plugins, give them the underpinnings to build Plugin-like experiences. Love it!
  • deleted 3y ago
    [deleted]
  • andybak 3y ago
    I've forgotten what pmf means from the last time someone used it and someone else explained it. (Can we please all lighten up on the acronyms a touch?)
    • lukaslezevicius 3y ago
      Product market fit.
      • andybak 3y ago
        I can see why my brain refuses to hold on to that one.
        • replwoacause 3y ago
          My brain remembered it from the other thread, but it hated seeing it again just the same
• iamflimflam1 3y ago
  Being able to give the LLM an API surface to call against is really powerful.
• personjerry 3y ago
  The 16k context window for gpt3.5 is exciting but unfortunately I think many of us were hoping for a gpt4 price drop!
  • JieJie 3y ago
    It's an effective price drop for a smarter GPT-4 model, though, isn't it? A smarter and more steerable model for the same price?
• sashank_1509 3y ago
  I can't wait for vision model access and the massive new opportunities that presents!
• armatav 3y ago
  Hm, still don’t see the 32k context in the Playground
  • naiv 3y ago
    I think they never put 8k into the playground? Edit: I can call the api now with gpt-4-32k-0613
    • ilaksh 3y ago
      So they opened 32k up to everyone?
      • heliophobicdude 3y ago
        Not yet. I just have the GPT-4 which is 8K limit
        • sagarpatil 3y ago
          [dead]
    • sagarpatil 3y ago
      [dead]
  • capybara_2020 3y ago
    Check under Mode > Complete. I see it listed there but not under Mode > Chat. Also try curl -X GET https://api.openai.com/v1/models https://api.openai.com/v1/models \ -H "Authorization: Bearer your-api-key" It is listed there for me.
    • EvgeniyZh 3y ago
      I've seen it on playground yesterday night (European time) and it disappeared by today
• braindead_in 3y ago
  > With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4! What about the rate limits? The docs say that it's 200 RPM and "We are unable to accommodate requests for rate limit increases due to capacity constraints."
  • janalsncm 3y ago
    I’d assume part of that $10B is going towards getting on the short list to buy GPUs, but there’s only so many to go around.
• sergiotapia 3y ago
  What are you thoughts here regarding functions: I have some data I can pass in CSV format to the context and ask a question against that data. "Who are my best customers?" and pass in a CSV of the top 100 customers. vs I create a function that returns my best customers and call a ChatGPT function. When would I use one or the other? The function call seems like it would be more accurate with better guardrails, but it does require me to know what questions my users will make beforehand. Maybe that's the point, use functions when you know what kind of questions your users will make.
  • ilaksh 3y ago
    Unless it's a small CSV then I would put it in sqlite or something and tell GPT-4 to write a query given the user's question. I have done that before and worked pretty well. Even worked fairly ok with gpt-3.5. You have to give it context like schema etc. I was even able to get it to output custom-coded embedded Chart.js charts if requested by the user.
    • kgumba 3y ago
      We're[0] building a tool that helps you do what you described. We mainly advertise our ability to do this over your data warehouse, like Snowflake, but we also use DuckDB to help people query CSVs. 3.5 has worked pretty well for us in most cases. It's also a good amount faster. GPT-4 seems to really stand out for our complex joins. - 0 = https://www.definite.app https://www.definite.app
• mesmertech 3y ago
  Seems like I was right to wait for price drop on GPT3.5 a few months ago, was hoping for mostly a drastic drop for GPT4 but I guess 25%(for input tokens, effectively 12.5% on average) on 3.5 works as well
  • Fiahil 3y ago
    Despite sharing a common prefix, GPT3.5 and GPT4 are completely different models. So, if you were hoping for a price drop on GPT4, the 3.5 drop might not be of any use to you
    • mesmertech 3y ago
      I know, I meant it'd have been better if GPT4 got a slash but for my usecase 3.5 is still cost-efficient, basically would've been happy with either of them getting a price cut
• MuffinFlavored 3y ago
  I don't understand it. You tell the gpt-4 session a function exists and supply the signature, but I don't see any URLs being defined on where the APIs live? > 1. Call the model with functions and the user’s input > 2. Use the model response to call your API I don't get this part. Where do you tell it where the API lives?
  • kgwgk 3y ago
    You don't. You take the model response and you call your API.
    • MuffinFlavored 3y ago
      Ah, I missed this part. ChatGPT response after you tell it a function call exists, it can decide whether or not to call it. { "id": "chatcmpl-123", ... "choices": [{ "index": 0, "message": { "role": "assistant", "content": null, "function_call": { "name": "get_current_weather", "arguments": "{ \"location\": \"Boston, MA\"}" } }, "finish_reason": "function_call" }] } Gotcha. This makes it so that instead of English, gpt-4 can basically spit out/decide when to make function calls now, got it. Thanks. I wonder how scary this will get if people mistakenly trust it with anything more than a GET. Imagine not being able to trust the reliability of some of the parameters of POST / PUT / DELETE on some reports/internal databases, etc.
      • naiv 3y ago
        Every api should have its own validation so I don't even see this as a problem. What is returned from openai should be treated like any other user input.
        • MuffinFlavored 3y ago
          > Every api should have its own validation so I don't even see this as a problem. No. I'm saying, little by little people will rely on OpenAI hypothetically for more and more. How long until they are calling POST /credit/customer/bank/account and it just randomly goofs the ID/numbers? A "human" may or may not have made that mistake, where an LLM will never be a 100% perfect trustable entity by design (aka, hallucinations). Now you're just giving it a way to hallucinate into a JSON request body.
• egonschiele 3y ago
  OpenAI continues to impress. Function calls will make working with JSON much easier, a current pain point. Dropping the price of embeddings and increasing context length means searching through your own content should become faster and more accurate. > $0.0015 per 1K input tokens and $0.002 per 1K output tokens, which equates to roughly 700 pages per dollar. This is such an incredible steal, especially when you consider that no open source option comes close to GPT3.5.
  • freediver 3y ago
    > This is such an incredible steal, especially when you consider that no open source option comes close to GPT3.5. Orca comes close or is better. Good explainer here: https://www.youtube.com/watch?v=Dt_UNg7Mchg https://www.youtube.com/watch?v=Dt_UNg7Mchg
    • ilaksh 3y ago
      But isn't Orca non-commercial? Or.. do people just ignore that part.
    • egonschiele 3y ago
      Is it accessible anywhere?
• jablongo 3y ago
  This seems great, but what does this mean for fine-tuning? Will we be able to fine-tune models prompted with function calls? Should we fine-tune models prompted with function calls? Depending on the complexity of the function-call/text-query pairs I think we may still want to...
  • egonschiele 3y ago
    I'm not sure about function calls but the lower price for embeddings and longer context lengths should help with fine tuning?
  • MallocVoidstar 3y ago
    They don't support fine-tuning GPT-3.5 or GPT-4. https://platform.openai.com/docs/guides/fine-tuning/what-models-can-be-fine-tuned https://platform.openai.com/docs/guides/fine-tuning/what-mod...
• egonschiele 3y ago
  Anyone know how they pick who to invite off the waitlist for GPT 4? I've been on there for a while. My project is open source and I wonder if that is getting me deprioritized.
  • civilitty 3y ago
    I sacrificed an albino goat with red eyes at midnight while chanting “sam-a sam-a sam-a” in the ancient R'lyehian language.
  • paulgb 3y ago
    You can get GPT 4 access by submitting an eval if gets merged (https://github.com/openai/evals https://github.com/openai/evals). Here's the one that got me access[1] Although from the blog post it looks like they're planning to open up to everyone soon, so that may happen before you get through the evals backlog. 1: https://github.com/openai/evals/pull/778 https://github.com/openai/evals/pull/778
  • gwd 3y ago
    I didn't say I had a project at all; I think I just said I wanted to learn about it. I got access w/in 3 weeks. Maybe I was lucky and it was random? Or maybe they figured I wouldn't add that much load?
  • sagarpatil 3y ago
    [dead]
• hubraumhugo 3y ago
  Nice! Real life use case of these updates for my autonomous web scraping product: - Bigger context window means less slicing and less calls for generating the web scrapers on the fly - The functions will help to reliably build our data transformation steps (e.g. mapping different sources into the same structure) - Way better unit economics
  • wallawe 3y ago
    Can this handle multiple input URLs? For example, I have 100 local business home pages, and I want you to get the email and phone number if they exists for each. Here are the 100 URLs.. I would be a happy paying customer if so.
    • savy91 3y ago
      Have you tried this? https://apify.com/vdrmota/contact-info-scraper https://apify.com/vdrmota/contact-info-scraper it can work with 50 input urls at a time.
      • wallawe 3y ago
        amazing, thank you
• weinzierl 3y ago
  16k context sounds exciting. The day I can throw a whole book at it and ask it arbitrary questions about it will be great. With 16k we are getting into full article realm and that is already incredibly useful. Is there any open model with a similar context length? [I'm not talking about the dubious for long context fine-tuned LLaMA variants, I mean the real thing.]
  • 037 3y ago
    Not open, but with Anthropic the limit is 100k now https://news.ycombinator.com/item?id=35904773 https://news.ycombinator.com/item?id=35904773
  • 101011 3y ago
    Try looking into vector databases to solve that problem. You can chunk up a book and embed those partitions into a vector database. Then you can take a query and fuzzy match the most relevant documents in your vector database, then feed it back to open AI to resolve an answer. It's brilliant. Postgres has an extension to support indexing the vectors, and there are some other open source and turnkey solutions in the market as well.
    • weinzierl 3y ago
      Brilliant, but not the same. I think both approaches have their place and are not mutually exclusive.
      • 101011 3y ago
        They are not the same, but your initial problem of throw a whole book at it and have OpenAI give you an answer is a demo that is solvable in 20 lines of langchain code when leveraging a vector DB. Demo here: https://www.youtube.com/watch?v=h0DHDp1FbmQ https://www.youtube.com/watch?v=h0DHDp1FbmQ (github code is linked within as well)
• zh217 3y ago
  Some observations: 1. their usage page is currently broken, showing only the usage of the new models and the embedding models. usage for the deprecated models are not included now. 2. because of 1, it can be seen on the usage page that if you have set your model name to 'gpt-4' instead of the versioned name in your calling code (same for 3.5), you have already been using the new models for the past two days!
• anotherpaulg 3y ago
  I've been spending a lot of time figuring out how to make these GPT models successful at *editing* existing code. This is much more difficult than having them write brand new code. So far, gpt-4 has been significantly better at editing code than gpt-3.5-turbo. This is for two reasons (which I previously discussed here [1]): 1. GPT-4's bigger context window lets it understand and edit larger codebases. The new 16k window for 3.5 might solve this problem. 2. GPT-4 is much better at following instructions about how to format code edits into a diff-like output format. Perhaps the improved instruction following will help 3.5 succeed here. Essentially, 3.5 isn't capable of outputting any sort of diff-based edit format. The only thing it can reliably do is send back *all* of the code (say an entire source file) with the changes included. On the other hand, GPT-4 is capable of reliably outputting (some) diff-like formats. So I'm very curious to see if the new 3.5 model solves both of these problems. I'll be running some benchmarks to find out! [1] https://github.com/paul-gauthier/aider#gpt-4-vs-gpt-35 https://github.com/paul-gauthier/aider#gpt-4-vs-gpt-35 EDIT: On point (2) above... early experiments with gpt-3.5-turbo-16k seem to indicate that it is NOT able to follow system prompt instructions to use a diff-like output format. Still need to try the new functions capability.
  • ilaksh 3y ago
    What I'm thinking is to give it a function called replaceAll that just replaces text, and another one called insertAfter. Maybe also replaceBetween.
    • anotherpaulg 3y ago
      Absolutely! I will certainly be experimenting with the new functions as you suggest. One thing which I haven't seen discussed elsewhere is the tension between the output format and the underlying task. When I ask GPT to use a simple, natural output format it does *better* at the actual code editing task. If I ask it to output using a more technical format like `diff -c` or heavily structured json formats... it "gets distracted" and does worse at the underlying coding request. It writes worse code if you ask it to output edits in a terse, machine-readable diff format. It writes better code if you let it show you the code edits in a simple way. For GPT3.5 this means I have to let it just type the whole source file back to me, with the edits included. GPT-4 is able to output a very simple diff-like format, but struggles with `diff -c`, etc. So it will be interesting to see how the new function capabilities affect this tradeoff. Perhaps it is now "fluent" in function-json and so won't be "distracted" by that output formatting.
• cbowal 3y ago
  Has anyone seen speed differences with the new gpt-3.5-turbo-0613 model? I've been testing for the past hour and I'm getting responses in about a quarter of the time.
  • saliagato 3y ago
    pretty much the same. slightly worse at following instructions
  • rattray 3y ago
    How's the quality?
• JoshMandel 3y ago
  Interesting to see this plugin-adjacent functionality landing in the chat API. It seems like there is no way to provide in context examples of calling functions, since they are now no longer just "assistant"-authored chat turns with text, but rather a distinct kind of output. This can make it hard to demonstrate how to use functions effectively. I haven't played with this feature yet; maybe the model will somehow be able to leverage in context chat turn examples and use those to inform its function call outputs?
  • naiv 3y ago
    https://platform.openai.com/docs/guides/gpt/function-calling https://platform.openai.com/docs/guides/gpt/function-calling they also have a notebook with more examples: https://github.com/openai/openai-cookbook/blob/main/examples/How_to_call_functions_with_chat_models.ipynb https://github.com/openai/openai-cookbook/blob/main/examples...
    • JoshMandel 3y ago
      I have read both of these but I did not notice any in-context examples, meaning prompts fed to the model showing it how to call a function and response to a user query, rather than docstrings telling it how to call a function.
• growthwtf 3y ago
  Just updated my little experiments to use the new models. Can confirm that this works far better. The JSON has been valid 100% of the time, but it still is a little fuzzy on string vs number in some cases for ids. Great update overall.
• alpark3 3y ago
  I wonder how much of these changes are pushed by the local LLM shift we've seen recently. I would've expected them to totally focus on GPT-4 updates, but it's nice that we're getting 3.5 improvements. It's pretty clear that there's a large demand for much cheaper, if weaker LLMs. I'll need to test the "more reliable steerability via the system message" feature, but GPT-3.5's largely monotonic tone and lack of response to the system message was one of its largest weaknesses imo. I'm all for ggml and LLaMa, but there's almost zero need for me to invest in hardware/expensive GPUs (or /hour options) if 3.5 is this cheap. Only downsides I can see are data privacy and OpenAI's "safety" restrictions. Function calls seem amazing, too. No need to use tokens commanding GPT about its ability to do function calls. I need to test it out though.
  • tedsanders 3y ago
    Describing functions to GPT still costs extra tokens, unfortunately.
    • TeMPOraL 3y ago
      Especially given they picked just about the most verbose way of doing it, second only to XML. While this is to be tested, given the examples they give, I somehow doubt that minified description with single-letter function names will perform as well as human-readable (verbose) schemas.
• benjamoon 3y ago
  Anyone know how long it will take Azure to get this latest model in its OpenAI service?
  • ldehaan 3y ago
    [dead]
  • chrisrickard 3y ago
    "Unfortunately, the new GPT-3.5-turbo-16k model is not yet available on Azure OpenAI. So, we can't share much information with you regarding this. We don't have any ETA at this moment. Once we have anything will share it with you." https://learn.microsoft.com/en-us/answers/questions/1305032/is-there-a-way-to-use-the-new-gpt-3-5-turbo-16k-(1 https://learn.microsoft.com/en-us/answers/questions/1305032/...
• orasis 3y ago
  This is for GPT 3.5 as well.
  • nlh 3y ago
    Good catch - updated title!
• ramraj07 3y ago
  > With this capability also comes potential risks. We strongly recommend building in user confirmation flows before taking actions that impact the world on behalf of users (sending an email, posting something online, making a purchase, etc). Yeah thanks for the heads up Sam.
  • dumpsterdiver 3y ago
    "When performing actions such as sending emails or posting comments on internet forums, the language learning model may mistakenly attempt to take control of powerful weaponry and engage with humans in physical warfare. For this reason we strongly suggest building in user confirmation prompts for such functionality."
    • dumpsterdiver 3y ago
      Bonus points if the LLM replies sinisterly with something like, "As a language learning model I am not legally liable for any property damage, loss of life, or sudden regime changes that may occur due to receiving user input. Are you sure that you would like to send this email?"
  • ChatGTP 3y ago
    [flagged]
    • EamonnMR 3y ago
      I believe it's called moving fast and breaking things.
      • CatWChainsaw 3y ago
        Emphasis on the second part?
    • cortesoft 3y ago
      This is a bit of an extreme take... have you never heard of a beta release?
• v3ss0n 3y ago
  I noticed amount of spam doubled as soon as i woke up - this explains me.
• valyagolev 3y ago
  It can also decently serve as a way to output structured data, which wasn't extremely hard before but had its failure modes. This is a much more typical and understandable use-case for many apps
• tmaly 3y ago
  > note: the model may generate invalid JSON or hallucinate parameters This seems like it could be a really big problem for this feature.
  • roughly 3y ago
    The Langchain package does something similar to this and has a feature to combat that kind of thing that basically feeds the invalid input back into the bot and says “you were asked for x but gave this, please correct it” - it’s shockingly effective.
  • Spivak 3y ago
    Which is surprising since jsonformer exists and the same approach works with the text-completion API just fine.
• tmaly 3y ago
  I was looking at the models. I noticed the function calling for 3.5 is only with the 4,096 token length. The larger 16k context length for 3.5 does not mention it despite having a similar naming of 0613 suffix. https://platform.openai.com/docs/models/gpt-3-5 https://platform.openai.com/docs/models/gpt-3-5 gpt-3.5-turbo-0613 Snapshot of gpt-3.5-turbo from June 13th 2023 with function calling data. Unlike gpt-3.5-turbo, this model will not receive updates, and will be deprecated 3 months after a new version is released. 4,096 tokens Up to Sep 2021 gpt-3.5-turbo-16k-0613 Snapshot of gpt-3.5-turbo-16k from June 13th 2023. Unlike gpt-3.5-turbo-16k, this model will not receive updates, and will be deprecated 3 months after a new version is released. 16,384 tokens Up to Sep 2021
• taf2 3y ago
  16k is tokens on 3.5 is amazing but they also complicated the pricing from a simple flat per 1k token to two separate fees for input and output … I think it’s better maybe but token based pricing is challenging to reason with and even more so to explain to customers
  • wordpad25 3y ago
    simplify the pricing and offer it as your own service = profit
  • visarga 3y ago
    They should work towards making repetitive prompts be free or much cheaper because they don't have to generate it token by token and can be cached.
    • amitport 3y ago
      As a customer of course I would want that, but they really shouldn't from their perspective. The customer is getting the value it wants and is willing to pay for this is where a lot of the profit will come from.
• avereveard 3y ago
  Too bad they haven't documented how to disperse functions in the system message ourself, that would have made langchain.js crazy powerful. (I.e create a function that answer the user question in JavaScript, you can call llm(prompt, context) to process the data into natural language and search(query) to find data for the user) - and then you let langchainjs execute the output as apart of its loop
  • avereveard 3y ago
    https://i.imgur.com/Cie0IJY.png https://i.imgur.com/Cie0IJY.png ahah it works! even on 'older' gpt! this is amazing. and crazy.
• vorticalbox 3y ago
  I was playing with this today made a simple add function that just takes two arguments and adds them and asked it "what is 10+5" and it called my function as you would expect. Then I made my function multiple the numbers. Weirdly gpt called my function twice then simply ignored the functions data and returned the correct answer. Pretty impressive.
  • gawi 3y ago
    Not sure I like the "I know better" attitude of the LLM in this case. What other function response the LLM is likely to discard?
    • vorticalbox 3y ago
      I updated a slack bot to support calling dalle but none of the images work because it striped the query parameters from the URLs. Very annoying
    • famouswaffles 3y ago
      This one isn't new at least. https://vgel.me/posts/tools-not-needed/ https://vgel.me/posts/tools-not-needed/