In their current iteration the models are very neutered. It’s been demonstrated that GPT models are fairly good at choosing when to perform a task. Obviously lo

In their current iteration the models are very neutered. It’s been demonstrated that GPT models are fairly good at choosing when to perform a task. Obviously lots of APIs and machinery is needed to actually perform tasks, but the heavy lifting “intelligence” portion can be almost entirely performed by our existing models. Some basic text based APIs that would quickly improve LLM utility: Calculators Database storage and retrieval Web access (already kind of done by bing) Shell scripting Thinking further into the future of multimodal models, it’s not hard to imagine this sort of thing could be extended to include image based APIs. Imagine a LLM looking at your gui and clicking on things. The sky’s the limit at that point. Checkout toolformer, they’ve got this mostly working with a much smaller model than gpt3.5. https://arxiv.org/abs/2302.04761 https://arxiv.org/abs/2302.04761