3 ms·
The only thing the LLM is missing is an self actuation loop. We put a camera on a multimodal LLM, it interprets the visual world and sees before it a bunch of
by pixl97 3y ago
The only thing the LLM is missing is an self actuation loop.
We put a camera on a multimodal LLM, it interprets the visual world and sees before it a bunch of blocks. It looks at the task list it has that says "pick up red blocks, put them in blue bin". The visual components identifies the red blocks and the textual component issues commands to its drive unit which calculates the best path and how to use it's manipulators to pick up the blocks.
This is a very basic chain of existence. We have world identification, we have actuation on motivation, we have interaction with the environment. We can do this now. These goals have already been achieved. Companies are already testing more complex models with much more general instructions such as "Pick up everything you think is trash" or "Organize this room" to see the emergent behaviors that come out of the models.
You are seemingly a few years behind what has already been done, and why people are starting to get concerned.