They are, but not chatGPT, at least not yet. In one paper they create a so called <work> token, such as <work>22+44</work> and get 66 inserted after the work bl

They are, but not chatGPT, at least not yet. In one paper they create a so called <work> token, such as <work>22+44</work> and get 66 inserted after the work block automatically. It can also run Python commands and write functions and use them. For example they ask what is the current BTC price and the model writes code to load the price from a web API. When it gets an error message it can try to fix the code. Language models would benefit from having a <search> token as well. Some models have demonstrated amazing things - with a large search index you can get good performance on many tasks with a 20x smaller model. No need to burn all the trivia in the weights of the network. Just use a search engine to help it.