4 ms·
> No such LLMs exist, because they are trained to predict the next word not (WebCamState, RobotArmState, NextWord) Seems we might not be that far away given th
by magicalhippo 2y ago
> No such LLMs exist, because they are trained to predict the next word not (WebCamState, RobotArmState, NextWord)
Seems we might not be that far away given the work on action tokens[1] and such.
NVIDIA divoted a lot of the CES presentation to this kind of stuff[2].
[1]: https://arxiv.org/abs/2403.19578 https://arxiv.org/abs/2403.19578 (semi-random example)
[2]: https://www.nvidia.com/en-us/ai/cosmos/ https://www.nvidia.com/en-us/ai/cosmos/