3 ms·

> Can somebody help me to understand how a LLM has achieved this capability. It's worth clarifying what is being accomplished here. iOS is handling speech reco

> Can somebody help me to understand how a LLM has achieved this capability. It's worth clarifying what is being accomplished here. iOS is handling speech recognition, and Shortcuts is handling task execution with an unspecified and presumably long user script. What GPT does here is convert text instructions into JSON formatted slot filling[1] responses. It's somewhat amazing that GPT is emitting valid JSON, but I guess it's seen enough JSON in the training set to understand the grammar, and we shouldn't be too surprised it can learn regular grammars if it can learn multiple human languages. Slot filling is a well studied topic, and with the very limited vocabulary of slots, it doesn't have as many options to go wrong as commercial voice assistants. I would be way more amazed if this were able to generate Shortcuts code directly, but I don't think that's allowed. > some of its output (eg future time stamps) will not be present in its training data. Even with several billion parameters that seems impossible Maybe this is a feature of attention, which lets each token look back to modify its own prediction, and special token segmentation[2] for dates? [1]: http://nlpprogress.com/english/intent_detection_slot_filling.html http://nlpprogress.com/english/intent_detection_slot_filling... [2]: https://github.com/google/sentencepiece https://github.com/google/sentencepiece

iinnPP 4y ago
I have also been having ChatGPT respond in JSON and it works incredibly well. Adding information into the middle of existing JSON output is easy as well. I find that ChatGPT allows you to "generate" random(terribly so) and unique IDs for references. It used to work a lot better, and was seemingly alterered just before the Dec 15 build was replaced. With this you can create the structure for replies and have chatGPT respond in an easy to read way. When you walk the Assistant through each method to fill the variables, the result is an ability to have Assistant then run the same steps on any new topic while accurately refollowing the steps. I have used this method to hold values for things such as image descriptions with "coordinates", behavioural chemistry, and moving "limbs" with limitations baked in using variables. In my image descriptions I would have 20-30 items capable of being interacted with. Each would be assigned a list of viable tasks and each item on the list would go through special instructions. The interesting part is that the team running the Assistant layer has blocked the bot from doing some of the final steps of my tests (recently). My first test ended at a closed door with an override preventing the bot from using the door. Assistant got through the door anyway, successfully, with 6 different options to choose from. By correctly identifying the word use as the problem to focus on The largest advantage I can see is the ability to locate invalid information very quickly for targeted retraining.

iinnPP 4y ago

I have also been having ChatGPT respond in JSON and it works incredibly well. Adding information into the middle of existing JSON output is easy as well. I find that ChatGPT allows you to "generate" random(terribly so) and unique IDs for references. It used to work a lot better, and was seemingly alterered just before the Dec 15 build was replaced. With this you can create the structure for replies and have chatGPT respond in an easy to read way. When you walk the Assistant through each method to fill the variables, the result is an ability to have Assistant then run the same steps on any new topic while accurately refollowing the steps. I have used this method to hold values for things such as image descriptions with "coordinates", behavioural chemistry, and moving "limbs" with limitations baked in using variables. In my image descriptions I would have 20-30 items capable of being interacted with. Each would be assigned a list of viable tasks and each item on the list would go through special instructions. The interesting part is that the team running the Assistant layer has blocked the bot from doing some of the final steps of my tests (recently). My first test ended at a closed door with an override preventing the bot from using the door. Assistant got through the door anyway, successfully, with 6 different options to choose from. By correctly identifying the word use as the problem to focus on The largest advantage I can see is the ability to locate invalid information very quickly for targeted retraining.