3 ms·

For authentication, since the tool call itself actually runs on your own server, can’t you just look at who the authed user is that made the request?

by paulgb 3y ago

oezi 3y ago
OpenAI doesn't give you a way to identify the user. And even if they did, it would be poor UX to have the user have to visit our site first to connect their API accounts. I also imagine many tools wouldn't run under the developers' control (of course you could relay over your server).
- TZubiri 3y ago
  Huh? You have to use your api key and pay for the service. Requests you make to the service providers are made on your own buck, you are supposed to track user stuff on your end. It would make no difference chatgpt wise who the user is, that's not part of the abstraction provided. Not a user auth SaaS, an LLM SaaS.
  - michaelt 3y ago
    Presumably oezi wants to do that complicated three-party OAuth stuff. Like when you use an online PDF editor with Google Drive integration - paying for the storage etc is between Google and the user, the files belong to the user's Google Drive account, but the PDF editor gets read/write access to them.
    - oezi 3y ago
      Yes, exactly. Many existing APIs are hard/impossible to connect to unless you are the user.
    - cjonas 3y ago
      I think the disconnect is that he's talking about building plugins/"gpts" inside of chat GPT while others are thinking about using the API to build something from scratch?
      - kristjansson 3y ago
        That's my read. And he's totally right! Plugins/GPTs aren't a good platform or product, partly for some of the technical reasons he mentioned, but really because they're basically a tech demo for the real product (the tool API).
  - oezi 3y ago
    Many interesting API usages must be bound to the user and must be payed based on usage so must be tied to the user. OpenAI doesn't provide ways to monetize GPTs so it is hard to justify spending on behalf of the user.
- simonw 3y ago
  I think you might be talking about GPTs with actions? This implementation of function calling works differently from those. OpenAI/Anthropic don't make any API calls themselves at all here. You call their chat APIs with a list of your own available functions, then they may reply to you saying "run function X yourself with these parameters, and once you've run that tell us what the result was". This is useful for more than just tool usage - it can help with structured data extraction too, where you don't execute functions at all: https://til.simonwillison.net/gpt3/openai-python-functions-data-extraction https://til.simonwillison.net/gpt3/openai-python-functions-d...
- randomdata 3y ago
  > I also imagine many tools wouldn't run under the developers' control How? There is no execution model. The LLM simply responds, in JSON format, with the name of a function and its corresponding arguments in alignment with the JSON Schema spec you provided beforehand. It is entirely on you to do something with that information. At the end of the day it is really not all that different to asking an LLM to respond with JSON in the prompt, but offers greater stability in the response as it will strictly adhere to the spec you defined and not sometimes go completely off the rails with unparseable gobbledygook as LLMs are known to do.

For authentication, since the tool call itself actually runs on your own server, can’t you just look at who the authed user is that made the request?

by paulgb 3y ago

For authentication, since the tool call itself actually runs on your own server, can’t you just look at who the authed user is that made the request?

oezi 3y ago

OpenAI doesn't give you a way to identify the user. And even if they did, it would be poor UX to have the user have to visit our site first to connect their API accounts. I also imagine many tools wouldn't run under the developers' control (of course you could relay over your server).

TZubiri 3y ago
Huh? You have to use your api key and pay for the service. Requests you make to the service providers are made on your own buck, you are supposed to track user stuff on your end. It would make no difference chatgpt wise who the user is, that's not part of the abstraction provided. Not a user auth SaaS, an LLM SaaS.
- michaelt 3y ago
  Presumably oezi wants to do that complicated three-party OAuth stuff. Like when you use an online PDF editor with Google Drive integration - paying for the storage etc is between Google and the user, the files belong to the user's Google Drive account, but the PDF editor gets read/write access to them.
  - oezi 3y ago
    Yes, exactly. Many existing APIs are hard/impossible to connect to unless you are the user.
  - cjonas 3y ago
    I think the disconnect is that he's talking about building plugins/"gpts" inside of chat GPT while others are thinking about using the API to build something from scratch?
    - kristjansson 3y ago
      That's my read. And he's totally right! Plugins/GPTs aren't a good platform or product, partly for some of the technical reasons he mentioned, but really because they're basically a tech demo for the real product (the tool API).
- oezi 3y ago
  Many interesting API usages must be bound to the user and must be payed based on usage so must be tied to the user. OpenAI doesn't provide ways to monetize GPTs so it is hard to justify spending on behalf of the user.
simonw 3y ago
I think you might be talking about GPTs with actions? This implementation of function calling works differently from those. OpenAI/Anthropic don't make any API calls themselves at all here. You call their chat APIs with a list of your own available functions, then they may reply to you saying "run function X yourself with these parameters, and once you've run that tell us what the result was". This is useful for more than just tool usage - it can help with structured data extraction too, where you don't execute functions at all: https://til.simonwillison.net/gpt3/openai-python-functions-data-extraction https://til.simonwillison.net/gpt3/openai-python-functions-d...
randomdata 3y ago
> I also imagine many tools wouldn't run under the developers' control How? There is no execution model. The LLM simply responds, in JSON format, with the name of a function and its corresponding arguments in alignment with the JSON Schema spec you provided beforehand. It is entirely on you to do something with that information. At the end of the day it is really not all that different to asking an LLM to respond with JSON in the prompt, but offers greater stability in the response as it will strictly adhere to the spec you defined and not sometimes go completely off the rails with unparseable gobbledygook as LLMs are known to do.