4 ms·
This seems perfect to hook up to my 'LLMs can control robots over MCP' system. The idea is that LLMs are great at writing code, so let's lean in to that. I'll g
by colinator 6mo ago
This seems perfect to hook up to my 'LLMs can control robots over MCP' system. The idea is that LLMs are great at writing code, so let's lean in to that. I'll give it a try! I just got a bigger robot, we'll see how it does...
https://colinator.github.io/Ariel/post1.html https://colinator.github.io/Ariel/post1.html
- Glemllksdf 6mo agoReally unfortunate that I forgot what YT video i saw about this just 2 weeks ago. It was about Googles PaLM-E evolution and progress. It basically has two models one which controls the robot, the other is a llm and they are combined together in some attention layer.
- johntb86 6mo agoMust have been https://www.youtube.com/watch?v=2mrGMMmrVNE https://www.youtube.com/watch?v=2mrGMMmrVNE from Welch Labs.
- colinator 6mo agoThat video is pretty good, thanks for finding it. I'm basically betting that an earlier, abandoned approach described in the video, "Code as Policy", will beat everything else. It requires no training data, and generalizes instantly to all robots.
- Glemllksdf 6mo agoYes :-)
- fennecfoxy 6mo agoI would look into VLAs/VLMs if you haven't already.