5 ms·

ROScribe: An LLM-based robotic platform

77 points by RoboticTech 3y ago

nathanfig 3y ago
Upvoting this because I love innovation in this space, and I use both ROS and LLMs daily. Kudos on the work here, I can see value for developers trying to break into ROS in particular. That being said, and I might just need to try it out to understand, but it's not clear to my why this tool would be better than just talking to ChatGPT directly. Particularly GPT-4. There's the guided workflow aspect, but even that I could probably emulate by copying the description from this article and asking GPT-4 to do the same. It might just require more manual copying and pasting.
- RoboticTech 3y ago
  Thank you. One of main features of ROScribe (which is still under development) is the use of RAG (Retrieval Augmented Generation) for robot software generation. In RAG, the code, rather than being generated by the LLM, is pulled from ROS index by LLM (LLM keeps all documentations and meta data of ROS index in a vector data base to figure out what piece of code is most suitable for the task at hand.) We are working on this part and we plan to publish a conference paper on it. So the ultimate solution is a mixture of codes pulled from online repositories (ROS index), internal data (companies own IPs), and the code that LLM generates. LLM is handling all of three segments, but only codes as minimum as needed.
  - nathanfig 3y ago
    That is cool, and especially useful since LLMs often get details like URLs to repositories wrong, so I could see how this system would increase the accuracy of the output.
chfritz 3y ago
The title of this post is wrong and misleading. ROScribe is not a platform, let alone a robotic platform. It's a tool for generating code for a robotic platform (ROS). I feel the need to clarify this because there is a ton of people who don't understand the difference between a user interface for a system (which ROScribe is for ROS code) and the system itself, and I think that this confusion is cause for much of the hype. What's important to ask is: Does the new technology, in this case LLMs, solve some hard problems in the application area (here robotics), or is it mostly "syntactic sugar", a nice-to-have.
- actionfromafar 3y ago
  I agree - I also think it's a lost cause. Everything will be "reinvented" with AI now, including terminology. Think about "an operating system for the cloud". What's that? Could be a hypervisor. Or a Javascript framework. Or Office 365, or ChromeOS. Who even knows at this point?
- RoboticTech 3y ago
  Maybe today it feels like less of a platform, but we plan to make it a platform by integrating more components into it. ROScribe is an AI-native robot software solution now; other than software generation (which is far more than code generation; it includes software specification, code retrieval from online repos, code retrieval from internal IPs, as well as code generation), we want to facilitate simulation (gazebo,...) and the actual robot build (order components online, etc.). That's why we called it platform. We are an early stage startup; hopefully our product gets better. I appreciate your feedback.
- nathanfig 3y ago
  Ehh, I think platform has been a pretty squishy term for a long time already so it's not one I usually dispute, even if I'm inclined to agree that this project is more of a workflow at this point. ROS itself is not an Operating System as the name implies; middleware is probably more accurate.
deleted 3y ago
[deleted]
synergy20 3y ago
great to see this,always interested in ros
- RoboticTech 3y ago
  Thank you. This is the main focus of our startup. So we will make this tool better and better. You can find more documentations and demos on our github.
moffkalast 3y ago
Was this developed in part by the OSRF? If not you may have a trademark problem on your hands regarding the naming.[0] As nice to have as name puns are, I contacted them for permission of a similar kind of pun name for a package earlier this year and unfortunately they couldn't approve it. Probably for the best since the name I came up with as a replacement was much better in the end lol. But you'll probably need to change this to something like "Scribe for ROS" or similar or you'll get a notice to cease and desist eventually. ROS-Mobile almost did before it got discontinued. [0] https://www.ros.org/imgs/TrademarkRulesAndGuidelines2022.pdf https://www.ros.org/imgs/TrademarkRulesAndGuidelines2022.pdf
- RoboticTech 3y ago
  Thanks for letting me know. I still feel like it shouldn't be a problem to have a name like that, especially for an open source project. I will double check.
  - moffkalast 3y ago
    Yeah I figured it would be cool as long as it's non-commercial, but I was wrong too. I guess they're kinda forced to protect the trademark by the established guidelines or they'll lose it, but they also don't want to discourage people from developing stuff so they're not super heavy handed about it.
- RoboticTech 3y ago
  May I ask what was the name of your package that wasn't approved?
  - moffkalast 3y ago
    Ah sure, it was OutdooROS i.e. a web visualizer for outdoor ROS robots. Later renamed to Vizanti. The two examples I cited in my appeal were MAVROS and ROS Mobile, and yeah turns out the former is actually maintained by OSRF and the latter was in breach of guidelines.

77 points by RoboticTech 3y ago

Upvoting this because I love innovation in this space, and I use both ROS and LLMs daily. Kudos on the work here, I can see value for developers trying to break into ROS in particular. That being said, and I might just need to try it out to understand, but it's not clear to my why this tool would be better than just talking to ChatGPT directly. Particularly GPT-4. There's the guided workflow aspect, but even that I could probably emulate by copying the description from this article and asking GPT-4 to do the same. It might just require more manual copying and pasting.

RoboticTech 3y ago
Thank you. One of main features of ROScribe (which is still under development) is the use of RAG (Retrieval Augmented Generation) for robot software generation. In RAG, the code, rather than being generated by the LLM, is pulled from ROS index by LLM (LLM keeps all documentations and meta data of ROS index in a vector data base to figure out what piece of code is most suitable for the task at hand.) We are working on this part and we plan to publish a conference paper on it. So the ultimate solution is a mixture of codes pulled from online repositories (ROS index), internal data (companies own IPs), and the code that LLM generates. LLM is handling all of three segments, but only codes as minimum as needed.
- nathanfig 3y ago
  That is cool, and especially useful since LLMs often get details like URLs to repositories wrong, so I could see how this system would increase the accuracy of the output.

chfritz 3y ago

The title of this post is wrong and misleading. ROScribe is not a platform, let alone a robotic platform. It's a tool for generating code for a robotic platform (ROS). I feel the need to clarify this because there is a ton of people who don't understand the difference between a user interface for a system (which ROScribe is for ROS code) and the system itself, and I think that this confusion is cause for much of the hype. What's important to ask is: Does the new technology, in this case LLMs, solve some hard problems in the application area (here robotics), or is it mostly "syntactic sugar", a nice-to-have.

actionfromafar 3y ago
I agree - I also think it's a lost cause. Everything will be "reinvented" with AI now, including terminology. Think about "an operating system for the cloud". What's that? Could be a hypervisor. Or a Javascript framework. Or Office 365, or ChromeOS. Who even knows at this point?
RoboticTech 3y ago
Maybe today it feels like less of a platform, but we plan to make it a platform by integrating more components into it. ROScribe is an AI-native robot software solution now; other than software generation (which is far more than code generation; it includes software specification, code retrieval from online repos, code retrieval from internal IPs, as well as code generation), we want to facilitate simulation (gazebo,...) and the actual robot build (order components online, etc.). That's why we called it platform. We are an early stage startup; hopefully our product gets better. I appreciate your feedback.
nathanfig 3y ago
Ehh, I think platform has been a pretty squishy term for a long time already so it's not one I usually dispute, even if I'm inclined to agree that this project is more of a workflow at this point. ROS itself is not an Operating System as the name implies; middleware is probably more accurate.

deleted 3y ago

[deleted]

synergy20 3y ago

great to see this,always interested in ros

RoboticTech 3y ago
Thank you. This is the main focus of our startup. So we will make this tool better and better. You can find more documentations and demos on our github.

moffkalast 3y ago

Was this developed in part by the OSRF? If not you may have a trademark problem on your hands regarding the naming.[0] As nice to have as name puns are, I contacted them for permission of a similar kind of pun name for a package earlier this year and unfortunately they couldn't approve it. Probably for the best since the name I came up with as a replacement was much better in the end lol. But you'll probably need to change this to something like "Scribe for ROS" or similar or you'll get a notice to cease and desist eventually. ROS-Mobile almost did before it got discontinued. [0] https://www.ros.org/imgs/TrademarkRulesAndGuidelines2022.pdf https://www.ros.org/imgs/TrademarkRulesAndGuidelines2022.pdf

RoboticTech 3y ago
Thanks for letting me know. I still feel like it shouldn't be a problem to have a name like that, especially for an open source project. I will double check.
- moffkalast 3y ago
  Yeah I figured it would be cool as long as it's non-commercial, but I was wrong too. I guess they're kinda forced to protect the trademark by the established guidelines or they'll lose it, but they also don't want to discourage people from developing stuff so they're not super heavy handed about it.
RoboticTech 3y ago
May I ask what was the name of your package that wasn't approved?
- moffkalast 3y ago
  Ah sure, it was OutdooROS i.e. a web visualizer for outdoor ROS robots. Later renamed to Vizanti. The two examples I cited in my appeal were MAVROS and ROS Mobile, and yeah turns out the former is actually maintained by OSRF and the latter was in breach of guidelines.