4 ms·
This is a very intriguing application of physical AI. It is astounding to see demos of how simple, human instructions can produce complex machine actions. What
by k-warburton 1y ago
This is a very intriguing application of physical AI. It is astounding to see demos of how simple, human instructions can produce complex machine actions. What can we do to improve the safety of protecting assets or people in the case of misuse? I see some code contributions regarding permission controls, but are there other steps we can take to ensure this technology "understands" when the physical motion being requested is not appropriate because it might endanger people or expensive hardware that is difficult to replace?
- sdallagasperina 1y agoGreat point, I believe safety has to be layered. The real challenge is deciding which agent is responsible for judging whether a command is safe to execute. For instance, MCP could enforce permissions, rate limits, and safe defaults, while the ROS stack could add motion constraints, watchdogs, and velocity/force caps, all backed by physical interlocks as the final safeguard.
- sdallagasperina 1y agoI see the LLM not as the one giving direct commands, but as suggesting a path. An arbitration layer should always check whether that suggestion is safe, and if it isn’t, the system should fall back to a deterministic, well-tested behavior. That way you get flexibility without ever compromising safety.