10 ms·

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking

by swyx 3y ago

i think people are underestimating the potential here for agents building - it is now a lot easier for GPT4 to call other models, or itself. while i was taking notes for our emergency pod yesterday (https://www.latent.space/p/function-agents https://www.latent.space/p/function-agents) we had this interesting debate with Simon Willison on just how many functions will be supplied to this API. Simon thinks it will be "deep" rather than "wide" - eg a few functions that do many things, rather than many functions that do few things. I think i agree. you can now trivially make GPT4 decide whether to call itself again, or to proceed to the next stage. it feels like the first XOR circuit from which we can compose a "transistor", from which we can compose a new kind of CPU.

minimaxir 3y ago
"Trivial" is misleading. From OpenAI's docs and demos, the full ReAct workflow is an order of magnitude more difficult than typical ChatGPT API usage with a new set of constaints (e.g. schema definitions) Even OpenAI's notebook demo has error handling workflows which was actually necessary since ChatGPT returned incorrect formatted output.
- cjonas 3y ago
  Maybe trivial isn't the right word, but it's still very straight-forward to get something basic, yet really powerful... ReAct Setup Prompt (goal + available actions) -> Agent "ReAction" -> Parse & Execute Action -> Send Action Response (success or error) -> Agent "ReAction" -> repeat As long as each action has proper validation and returns meaningful error messages, you don't need to even change the control flow. The agent will typically understand what went wrong, and attempt to correct it in the next "ReAction". I've been refactoring some agents to use "functions" and so far it seems to be a HUGE improvement in reliability vs the "Return JSON matching this format" approach. Most impactful is that fact that "3.5-turbo" will now reliability return JSON (before you'd be forced to use GPT-4 for an ReAct style agent of modest complexity). My agents also seem to be better at following other instructions now that the noise of the response format is gone (of course it's still there, but in a way it has been specifically trained on). This could also just be a result of the improvements to the system prompt though.
  - deleted 3y ago
    [deleted]
  - devbent 3y ago
    For 3.5, I found it easiest to specify a simple, but parsable, format for responses and then convert that to JSON myself. I'll have to see if the new JSON schema support is easier than what I already have in place.
majormajor 3y ago
GPT-4 was already a massive improvement on 3.5 in terms of replying consistently in a certain JSON structure - I often don't even need to give examples, just a sentence describing the format. It's great to see they're making it even better, but where I'm currently hitting the limit still in GPT-4 for "shelling out" is about it being truly "creative" or "introspective" about "do I need to ask for clarifications" or "can I find a truly novel away around this task" type of things vs "here's a possible but half-baked sequence I'm going to follow".
- fumar 3y ago
  It is “good enough”. Where I struggle is maintaining its memory through a longer request where multiple iterations fail or succeed and then all of a sudden its memory is exceeded and starts fresh. I wish I could store “learnings” that it could revisit.
  - ehsanu1 3y ago
    Sounds like you want something like tree of thoughts: https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2305.10601
    - jimmySixDOF 3y ago
      Interestingly the paper's repo starts off : Blah Blah "...is NOT the correct implementation to replicate paper results. In fact, people have reported that his code cannot properly run, and is probably automatically generated by ChatGPT, and kyegomez has done so for other popular ML methods, while intentionally refusing to link to official implementations for his own interests" Love a good GitHub Identity Theft Star farming ML story But this method could have potential for a chain of function
lbeurerkellner 3y ago
It's interesting to think about this form of computation (LLM + function call) in terms of circuitry. It is still unclear to me however, if the sequential form of reasoning imposed by a sequence of chat messages is the right model here. LLM decoding and also more high-level "reasoning algorithms" like tree of thought are not that linear. Ever since we started working on LMQL, the overarching vision all along was to get to a form of language model programming, where LLM calls are just the smallest primitive of the "text computer" you are running on. It will be interesting to see what kind of patterns emerge, now that the smallest primitive becomes more robust and reliable, at least in terms of the interface.
ftxbro 3y ago
> "you can now trivially make GPT4 decide whether to call itself again, or to proceed to the next stage." Does this mean the GPT-4 API is now publicly available, or is there still a waitlist? If there's a waitlist and you literally are not allowed to use it no matter how much you are willing to pay then it seems like it's hard to call that trivial.
- bayesianbot 3y ago
  "With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4!" https://openai.com/blog/function-calling-and-other-api-updates https://openai.com/blog/function-calling-and-other-api-updat...
- Tostino 3y ago
  Not GP, but it's still the latter...i've been (im)patiently waiting. From their blog post the other day: With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4!
  - londons_explore 3y ago
    If you put contact info in your HN profile - especially an email address that matches one you use to login to openai, someone will probably give you access... Anyone with access can share it with any other user via the 'invite to organisation' feature. Obviously that allows the invited person do requests billed to the inviter, but since most experiments are only a few cents that doesn't really matter much in practice.
    - Tostino 3y ago
      Good to know, but I've racked up a decent bill for just my GPT 3.5 use. I can get by with experiments using my ChatGPT Plus subscription, but I really need my own API access to start using it for anything serious.
jonplackett 3y ago
It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been using this to make a few ‘choose your own adventure’ type games and I can see there’s a TONNE of potential useful things.
- cwxm 3y ago
  even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.
  - ztratar 3y ago
    Nah, this was solved by most teams a while ago.
    - bel423 3y ago
      I feel like I’m taking crazy pills with the amount of people saying this is game changing. Did they not even try asking gpt to format the output as json?
      - worik 3y ago
        > I feel like I’m taking crazy pills....try asking gpt to format the output as json You are taking crazey pills. Stop gpt-? is unreliable! That is not a bug in it, it is the nature of the beast. It is not an expert at anything except natural language, and even then it is an idiot savant
  - jonplackett 3y ago
    Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}
ilaksh 3y ago
The thing is the relevant context often depends on what it's trying to do. You can give it a lot of context in 16k but if there are too many different types of things then I think it will be confused or at least have less capacity for the actual selected task. So what I am thinking is that some functions might just be like gateways into a second menu level. So instead of just edit_file with the filename and new source, maybe only select_files_for_edit is available at the top level. In that case I can ensure it doesn't try to overwrite an existing file without important stuff that was already in there, by providing the requested files existing contents along with the function allowing the file edit.
- naiv 3y ago
  I think big context only makes sense for document analysis. For programming you want to keep it slim. Just like you should keep your controllers and classes slim. Also people with 32k access report very very long response times of up to multiple minutes which is not feasible if you only want a smaller change or analysis.
- throwuwu 3y ago
  Not sure that’s true. I haven’t completely filled the context with examples but I do provide 8 or so exchanges between user and assistant along with a menu of available commands and it seems to be able to generalize from that very well. No hallucinations either. Good idea about sub menus though, I’ll have to use that.
moneywoes 3y ago
Wow your brand is huge. Crazy growth. i wonder how much these subtle mentions on forums help
- TeMPOraL 3y ago
  They're the only one commenter on HN I noticed keeps writing "smol" instead of "small", and is associated with projects with "smol" in their name. Surely I'm not the only one who missed it being a meme around 2015 or sth., and finds this word/use jarring - and therefore very attention-grabbing? Wonder how much that helps with marketing. This is meant with no negative intentions. It's just that 'swyx was, in my mind, "that HN-er that does AI and keeps saying 'smol'" for far longer than I was aware of latent.space articles/podcasts.
  - memefrog 3y ago
    Personally, I associate "smol" with "doggo" and "chonker" and other childish redditspeak.
  - swyx 3y ago
    and fun fact i used to work at Temporal too heheh.
- swyx 3y ago
  i mean hopefully its relevant content to the discussion, i hope enough pple know me here by now that i fully participate in The Discourse rather than just being here to cynically plug my stuff. i had a 1.5 hr convo with simon willison and other well known AI tinkerers on this exact thing, and so I shared it, making the most out of their time that they chose to share with me.
babyshake 3y ago
What would be an example where there needs to be an arbitrary level of recursive ability for GPT4 to call itself?
- swyx 3y ago
  writing code of higher complexity (we know from CICERO that longer time spent on inference is worth orders of magnitude more than the equivalent in training when it comes to improving end performance), or doing real world tasks with unknown fractal depth (aka yak shave)
freezed88 3y ago
100%, if the API itself can choose to call a function or an LLM, then it's way easier to build any agent loop without extensive prompt engineering + worrying about errors. Tweeted about it here as well: https://twitter.com/jerryjliu0/status/1668994580396621827?s=20 https://twitter.com/jerryjliu0/status/1668994580396621827?s=...
- bel423 3y ago
  You still have to worry about errors. You will probably have to add an error handler function that it can call out to. Otherwise the LLM will hallucinate a valid output regardless of the input. You want it to be able to throw an error and say I could produce the output given this format.
killingtime74 3y ago
Who is Simon Willison? Is he big in AI?
- swyx 3y ago
  formerly cocreator of Django, now Datasette, but pretty much the top writer/hacker on HN making AI topics accessible to engineers https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=true&query=https%3A%2F%2Fsimonwillison.net%2F&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...
  - killingtime74 3y ago
    Oh wow, nice! Big fan of his work
boringuser2 3y ago
Do you people always have to overhype this shit?
- throwuwu 3y ago
  What’s your problem? There’s nothing overhyped about that comment. People, including me, are building complex agents that can execute multi stage prompts and perform complex tasks. Comparing these first models to a basic unit of logic is more than fair given how much more capable they are. Do you just have an axe to grind?
  - boringuser2 3y ago
    [flagged]
    - pyinstallwoes 3y ago
      How is it inappropriate? How is it not building?
      - deleted 3y ago
        [deleted]
- delhanty 3y ago
  Do you have to be nasty? That's a person you're replying to with feelings, so why not default to being kind in comments as per HN guidelines? As it happens, swyx has built notable AI related things, for example smol-developer https://twitter.com/swyx/status/1657892220492738560 https://twitter.com/swyx/status/1657892220492738560 and it would be nice to be able to read his and other perspectives without having to read shallow, mean, dismissive replies such as yours.
  - boringuser2 3y ago
    [flagged]
jarulraj 3y ago
Interesting observation, @swyx. There seems to be a connection to transitive closure in SQL queries, where the output of the query is fed as the input to the query in the next iteration [1]. We are thinking about how to best support such recursive functions in EvaDB [2]. [1] http://dwhoman.com/blog/sql-transitive-closure.html http://dwhoman.com/blog/sql-transitive-closure.html [2] https://evadb.readthedocs.io/en/stable/source/tutorials/11-similarity-search-for-motif-mining.html#image-level-similarity-search-pipeline https://evadb.readthedocs.io/en/stable/source/tutorials/11-s...
quickthrower2 3y ago
The first transistors were slow, and it seems this "GPT3/4 calling itself" stuff is quite slow. GPT3/4 as a direct chat is about as slow as I can take. Once this gets sped up. I am sure it will, as you can scale out, scale up and build more efficient code and build more efficient architectures and "tool for the job" different parts of the process. The problem now (using auto gpt, for example) is accuracy is bad, so you need human feedback and intervention AND it is slow. Take away the slow, or the needing human intervention and this can be very powerful. I dream of the breakthrough "shitty old laptop is all you need" paper where they figure out how to do amazing stuff with a 1Gb of space on a spinny disk and 1Gb RAM and a CPU.
SimianLogic 3y ago
I agree with this. We’ve already gotten pretty good at json coercion, but this seems like it goes one step further by bundling decision making in to the model instead of junking up your prompt or requiring some kind of eval on a single json response. It should also be much easier to cache these functions. If you send the same set of functions on every API hit, OpenAI should be able to cache that more intelligently than if everything was one big text prompt.
jillesvangurp 3y ago
Exactly, we humans can use specialized models and traditional tool APIs and models and orchestrate the use of all these without understanding how these things work in detail. To do accounting, GPT 4 (or future models) doesn't have to know how to calculate. All it needs to know how to interface with tools like calculators, spreadsheets, etc. and parse their outputs. Every script, program, etc. becomes a thing that has such an API. A lot what we humans do to solve problems is breaking down big problems into problems where we know the solution already. Real life tool interfaces are messy and optimized for humans with their limited language and cognitive skills. Ironically, that means they are relatively easy to figure out for AI language models. Relative to human language the grammar of these tool "languages" is more regular and the syntax less ambiguous and complicated. Which is why gpt 3 and 4 are reasonably proficient with even some more obscure programming languages and in the use of various frameworks; including some very obscure ones. Given a lot of these tools with machine accessible APIs with some sort of description or documentation, figuring out how to call these things is relatively straightforward for a language model. The rest is just coming up with a high level plan and then executing it. Which amounts to generating some sort of script that does this. As soon as you have that, that in itself becomes a tool that may be used later. So, it can get better over time. Especially once it starts incorporating feedback about the quality of its results. It would be able to run mini experiments and run its own QA on its own output as well.

by swyx 3y ago

minimaxir 3y ago

"Trivial" is misleading. From OpenAI's docs and demos, the full ReAct workflow is an order of magnitude more difficult than typical ChatGPT API usage with a new set of constaints (e.g. schema definitions) Even OpenAI's notebook demo has error handling workflows which was actually necessary since ChatGPT returned incorrect formatted output.

cjonas 3y ago
Maybe trivial isn't the right word, but it's still very straight-forward to get something basic, yet really powerful... ReAct Setup Prompt (goal + available actions) -> Agent "ReAction" -> Parse & Execute Action -> Send Action Response (success or error) -> Agent "ReAction" -> repeat As long as each action has proper validation and returns meaningful error messages, you don't need to even change the control flow. The agent will typically understand what went wrong, and attempt to correct it in the next "ReAction". I've been refactoring some agents to use "functions" and so far it seems to be a HUGE improvement in reliability vs the "Return JSON matching this format" approach. Most impactful is that fact that "3.5-turbo" will now reliability return JSON (before you'd be forced to use GPT-4 for an ReAct style agent of modest complexity). My agents also seem to be better at following other instructions now that the noise of the response format is gone (of course it's still there, but in a way it has been specifically trained on). This could also just be a result of the improvements to the system prompt though.
- deleted 3y ago
  [deleted]
- devbent 3y ago
  For 3.5, I found it easiest to specify a simple, but parsable, format for responses and then convert that to JSON myself. I'll have to see if the new JSON schema support is easier than what I already have in place.

majormajor 3y ago

GPT-4 was already a massive improvement on 3.5 in terms of replying consistently in a certain JSON structure - I often don't even need to give examples, just a sentence describing the format. It's great to see they're making it even better, but where I'm currently hitting the limit still in GPT-4 for "shelling out" is about it being truly "creative" or "introspective" about "do I need to ask for clarifications" or "can I find a truly novel away around this task" type of things vs "here's a possible but half-baked sequence I'm going to follow".

fumar 3y ago
It is “good enough”. Where I struggle is maintaining its memory through a longer request where multiple iterations fail or succeed and then all of a sudden its memory is exceeded and starts fresh. I wish I could store “learnings” that it could revisit.
- ehsanu1 3y ago
  Sounds like you want something like tree of thoughts: https://arxiv.org/abs/2305.10601 https://arxiv.org/abs/2305.10601
  - jimmySixDOF 3y ago
    Interestingly the paper's repo starts off : Blah Blah "...is NOT the correct implementation to replicate paper results. In fact, people have reported that his code cannot properly run, and is probably automatically generated by ChatGPT, and kyegomez has done so for other popular ML methods, while intentionally refusing to link to official implementations for his own interests" Love a good GitHub Identity Theft Star farming ML story But this method could have potential for a chain of function

lbeurerkellner 3y ago

It's interesting to think about this form of computation (LLM + function call) in terms of circuitry. It is still unclear to me however, if the sequential form of reasoning imposed by a sequence of chat messages is the right model here. LLM decoding and also more high-level "reasoning algorithms" like tree of thought are not that linear. Ever since we started working on LMQL, the overarching vision all along was to get to a form of language model programming, where LLM calls are just the smallest primitive of the "text computer" you are running on. It will be interesting to see what kind of patterns emerge, now that the smallest primitive becomes more robust and reliable, at least in terms of the interface.

ftxbro 3y ago

> "you can now trivially make GPT4 decide whether to call itself again, or to proceed to the next stage." Does this mean the GPT-4 API is now publicly available, or is there still a waitlist? If there's a waitlist and you literally are not allowed to use it no matter how much you are willing to pay then it seems like it's hard to call that trivial.

bayesianbot 3y ago
"With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4!" https://openai.com/blog/function-calling-and-other-api-updates https://openai.com/blog/function-calling-and-other-api-updat...
Tostino 3y ago
Not GP, but it's still the latter...i've been (im)patiently waiting. From their blog post the other day: With these updates, we’ll be inviting many more people from the waitlist to try GPT-4 over the coming weeks, with the intent to remove the waitlist entirely with this model. Thank you to everyone who has been patiently waiting, we are excited to see what you build with GPT-4!
- londons_explore 3y ago
  If you put contact info in your HN profile - especially an email address that matches one you use to login to openai, someone will probably give you access... Anyone with access can share it with any other user via the 'invite to organisation' feature. Obviously that allows the invited person do requests billed to the inviter, but since most experiments are only a few cents that doesn't really matter much in practice.
  - Tostino 3y ago
    Good to know, but I've racked up a decent bill for just my GPT 3.5 use. I can get by with experiments using my ChatGPT Plus subscription, but I really need my own API access to start using it for anything serious.

jonplackett 3y ago

It was already quite easy to get GPT-4 to output json. You just append ‘reply in json with this format’ and it does a really good job. GPT-3.5 was very haphazard though and needs extensive babysitting and reminding, so if this makes gpt3 better then it’s useful - it does have an annoying disclaimer though that ‘it may not reply with valid json’ so we’ll still have to do some sense checks into he output. I have been using this to make a few ‘choose your own adventure’ type games and I can see there’s a TONNE of potential useful things.

cwxm 3y ago
even with gpt 4, it hallucinates enough that it’s not reliable, forgetting to open/close brackets and quotes. This sounds like it’d be a big improvement.
- ztratar 3y ago
  Nah, this was solved by most teams a while ago.
  - bel423 3y ago
    I feel like I’m taking crazy pills with the amount of people saying this is game changing. Did they not even try asking gpt to format the output as json?
    - worik 3y ago
      > I feel like I’m taking crazy pills....try asking gpt to format the output as json You are taking crazey pills. Stop gpt-? is unreliable! That is not a bug in it, it is the nature of the beast. It is not an expert at anything except natural language, and even then it is an idiot savant
- jonplackett 3y ago
  Not that it matters now but just doing something like this works 99% of the time or more with 4 and 90% with 3.5. It is VERY IMPORTANT that you respond in valid JSON ONLY. Nothing before or after. Make sure to escape all strings. Use this format: {“some_variable”: [describe the variable purpose]}

ilaksh 3y ago

The thing is the relevant context often depends on what it's trying to do. You can give it a lot of context in 16k but if there are too many different types of things then I think it will be confused or at least have less capacity for the actual selected task. So what I am thinking is that some functions might just be like gateways into a second menu level. So instead of just edit_file with the filename and new source, maybe only select_files_for_edit is available at the top level. In that case I can ensure it doesn't try to overwrite an existing file without important stuff that was already in there, by providing the requested files existing contents along with the function allowing the file edit.

naiv 3y ago
I think big context only makes sense for document analysis. For programming you want to keep it slim. Just like you should keep your controllers and classes slim. Also people with 32k access report very very long response times of up to multiple minutes which is not feasible if you only want a smaller change or analysis.
throwuwu 3y ago
Not sure that’s true. I haven’t completely filled the context with examples but I do provide 8 or so exchanges between user and assistant along with a menu of available commands and it seems to be able to generalize from that very well. No hallucinations either. Good idea about sub menus though, I’ll have to use that.

moneywoes 3y ago

Wow your brand is huge. Crazy growth. i wonder how much these subtle mentions on forums help

TeMPOraL 3y ago
They're the only one commenter on HN I noticed keeps writing "smol" instead of "small", and is associated with projects with "smol" in their name. Surely I'm not the only one who missed it being a meme around 2015 or sth., and finds this word/use jarring - and therefore very attention-grabbing? Wonder how much that helps with marketing. This is meant with no negative intentions. It's just that 'swyx was, in my mind, "that HN-er that does AI and keeps saying 'smol'" for far longer than I was aware of latent.space articles/podcasts.
- memefrog 3y ago
  Personally, I associate "smol" with "doggo" and "chonker" and other childish redditspeak.
- swyx 3y ago
  and fun fact i used to work at Temporal too heheh.
swyx 3y ago
i mean hopefully its relevant content to the discussion, i hope enough pple know me here by now that i fully participate in The Discourse rather than just being here to cynically plug my stuff. i had a 1.5 hr convo with simon willison and other well known AI tinkerers on this exact thing, and so I shared it, making the most out of their time that they chose to share with me.

babyshake 3y ago

What would be an example where there needs to be an arbitrary level of recursive ability for GPT4 to call itself?

swyx 3y ago
writing code of higher complexity (we know from CICERO that longer time spent on inference is worth orders of magnitude more than the equivalent in training when it comes to improving end performance), or doing real world tasks with unknown fractal depth (aka yak shave)

freezed88 3y ago

100%, if the API itself can choose to call a function or an LLM, then it's way easier to build any agent loop without extensive prompt engineering + worrying about errors. Tweeted about it here as well: https://twitter.com/jerryjliu0/status/1668994580396621827?s=20 https://twitter.com/jerryjliu0/status/1668994580396621827?s=...

bel423 3y ago
You still have to worry about errors. You will probably have to add an error handler function that it can call out to. Otherwise the LLM will hallucinate a valid output regardless of the input. You want it to be able to throw an error and say I could produce the output given this format.

killingtime74 3y ago

Who is Simon Willison? Is he big in AI?

swyx 3y ago
formerly cocreator of Django, now Datasette, but pretty much the top writer/hacker on HN making AI topics accessible to engineers https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=true&query=https%3A%2F%2Fsimonwillison.net%2F&sort=byPopularity&type=story https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=tru...
- killingtime74 3y ago
  Oh wow, nice! Big fan of his work

boringuser2 3y ago

Do you people always have to overhype this shit?

throwuwu 3y ago
What’s your problem? There’s nothing overhyped about that comment. People, including me, are building complex agents that can execute multi stage prompts and perform complex tasks. Comparing these first models to a basic unit of logic is more than fair given how much more capable they are. Do you just have an axe to grind?
- boringuser2 3y ago
  [flagged]
  - pyinstallwoes 3y ago
    How is it inappropriate? How is it not building?
    - deleted 3y ago
      [deleted]
delhanty 3y ago
Do you have to be nasty? That's a person you're replying to with feelings, so why not default to being kind in comments as per HN guidelines? As it happens, swyx has built notable AI related things, for example smol-developer https://twitter.com/swyx/status/1657892220492738560 https://twitter.com/swyx/status/1657892220492738560 and it would be nice to be able to read his and other perspectives without having to read shallow, mean, dismissive replies such as yours.
- boringuser2 3y ago
  [flagged]

jarulraj 3y ago

Interesting observation, @swyx. There seems to be a connection to transitive closure in SQL queries, where the output of the query is fed as the input to the query in the next iteration [1]. We are thinking about how to best support such recursive functions in EvaDB [2]. [1] http://dwhoman.com/blog/sql-transitive-closure.html http://dwhoman.com/blog/sql-transitive-closure.html [2] https://evadb.readthedocs.io/en/stable/source/tutorials/11-similarity-search-for-motif-mining.html#image-level-similarity-search-pipeline https://evadb.readthedocs.io/en/stable/source/tutorials/11-s...

quickthrower2 3y ago

The first transistors were slow, and it seems this "GPT3/4 calling itself" stuff is quite slow. GPT3/4 as a direct chat is about as slow as I can take. Once this gets sped up. I am sure it will, as you can scale out, scale up and build more efficient code and build more efficient architectures and "tool for the job" different parts of the process. The problem now (using auto gpt, for example) is accuracy is bad, so you need human feedback and intervention AND it is slow. Take away the slow, or the needing human intervention and this can be very powerful. I dream of the breakthrough "shitty old laptop is all you need" paper where they figure out how to do amazing stuff with a 1Gb of space on a spinny disk and 1Gb RAM and a CPU.

SimianLogic 3y ago

I agree with this. We’ve already gotten pretty good at json coercion, but this seems like it goes one step further by bundling decision making in to the model instead of junking up your prompt or requiring some kind of eval on a single json response. It should also be much easier to cache these functions. If you send the same set of functions on every API hit, OpenAI should be able to cache that more intelligently than if everything was one big text prompt.

jillesvangurp 3y ago

Exactly, we humans can use specialized models and traditional tool APIs and models and orchestrate the use of all these without understanding how these things work in detail. To do accounting, GPT 4 (or future models) doesn't have to know how to calculate. All it needs to know how to interface with tools like calculators, spreadsheets, etc. and parse their outputs. Every script, program, etc. becomes a thing that has such an API. A lot what we humans do to solve problems is breaking down big problems into problems where we know the solution already. Real life tool interfaces are messy and optimized for humans with their limited language and cognitive skills. Ironically, that means they are relatively easy to figure out for AI language models. Relative to human language the grammar of these tool "languages" is more regular and the syntax less ambiguous and complicated. Which is why gpt 3 and 4 are reasonably proficient with even some more obscure programming languages and in the use of various frameworks; including some very obscure ones. Given a lot of these tools with machine accessible APIs with some sort of description or documentation, figuring out how to call these things is relatively straightforward for a language model. The rest is just coming up with a high level plan and then executing it. Which amounts to generating some sort of script that does this. As soon as you have that, that in itself becomes a tool that may be used later. So, it can get better over time. Especially once it starts incorporating feedback about the quality of its results. It would be able to run mini experiments and run its own QA on its own output as well.