Cute, there are a number of people and labs working on integrating GPT models with robotics. I think you'd need to train a model on prompt-action pairs to get t

Cute, there are a number of people and labs working on integrating GPT models with robotics. I think you'd need to train a model on prompt-action pairs to get the best results but you could potentially do what they did in the Voyager paper https://arxiv.org/abs/2305.16291 https://arxiv.org/abs/2305.16291 and have a pre-trained LLM just write the raw programs to control the robot and then save and re-use them with function calling. https://www.microsoft.com/en-us/research/group/autonomous-systems-group-robotics/articles/chatgpt-for-robotics/ https://www.microsoft.com/en-us/research/group/autonomous-sy... https://www.microsoft.com/en-us/research/group/applied-robotics-research/articles/gpt-models-meet-robotic-applications-long-step-robot-control-in-various-environments/ https://www.microsoft.com/en-us/research/group/applied-robot... https://blogs.nvidia.com/blog/eureka-robotics-research/ https://blogs.nvidia.com/blog/eureka-robotics-research/ https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to-make-a-robot-that-cleans-on-command https://www.vice.com/en/article/93kb43/princeton-uses-gpt-to... https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-near/?guccounter=1 https://techcrunch.com/2023/11/10/ai-robotics-gpt-moment-is-...