5 ms·
This is getting pretty close to how I think we get to the general purpose humanoid robot. This is how I see it playing out: - You have your Boston Dynamics sty
by rcarr 3y ago
This is getting pretty close to how I think we get to the general purpose humanoid robot. This is how I see it playing out:
- You have your Boston Dynamics style humanoid robot at the job site, lets say it's a bricklayer for the purposes of this example.
- You have a human somewhere offsite in an open room with an omnidirectional treadmill floor, and cameras and depth sensors positioned all around the room. They're wearing a Hollywood style motion capture suit and have a VR headset on so they can see what the humanoid robot sees through their cameras.
- The human then acts as they would on site, walking up to the pile of bricks, picking them up, placing them etc. The robot moves in real time on the job site, mimicking whatever action the human performs. I don't know if you'll need props to do this properly or if the muscle memory from years on the job will be enough for the humans to get the motions right.
- You log all the data. You then have someone watch through the video stream, labelling each action that is being performed.
- You run it all through a machine learning algorithm, until you get to the point where you can just send the architectural plan to the robot and essentially say "Build this wall for me".
- tshaddox 3y agoAnd yet, as far as I know we don't even have mature implementations of this even for machines with much courser movement whose movement are controlled by a loose physical mapping to the human operator's movements. Things like excavators with dual joystick controls.
- ramesh31 3y agoThis is precisely how Comma and Tesla do self driving.
- wegfawefgawefg 3y agocomma does raw input to output behaviour cloning like you propose. interestingly, tesla does not. via there last public ml update, theyre self driving architecture still involves a path generator and evaluator combo, and a lot of seperately designed sensor fusion steps. parking is done via ppo reinforcement learning i think.
- Lemmi 3y agoIn short term perhaps but not long-term. I believe they will send out a team to digitize the job site. The team will then create a digital twin. The architect will map everything into this twin. The computer system will simulate the build steps. Robots will be brought onto the job site and get a finetuned model (if necessary) and will build it automatically.
- mortureb 3y agoNo, long term we somehow get something magical like LLMs that swoop in and magically pull off these tasks.
- eastbound 3y agoNone of that. We get LLM on the jobsite and it tells worker how it’s done. Often nothing as cheap as humans in terms of energy consumption. I doubt LLMs will beat that energy consumption.
- deleted 3y ago[deleted]
- SanderNL 3y agoHumans are .. complicated. You cannot just compensate a human’s caloric expense (although many companies would love that). Anywhere a human gets involved shit gets expensive. Interestingly robots are pretty human-intensive, from design to programming and maintenance. Only at scale does the “human factor” go down and things become affordable.
- danielbln 3y agoAlso, humans are squishy, you need safety requirements to keep them intact (seeing as human repair is incredibly costly and takes ages to complete, often with no guarantee that repair completes and performance levels go back to nominal), humans need sleep and can't work around the clock, and they have a mind of their own, which can make them difficult to deal with.
- 0xbeefcab 3y agoThe first 3/4 points were all almost exactly from the 2008 movie Sleep Dealer to the point where I thought you were referencing that
- deleted 3y ago[deleted]
- ramesh31 3y agoYes, it's called reinforcement learning.
- pests 3y agoI once read a novel or short story with this concept. The robots were piloted (on-site) for training and then let loose on their own. I forget the plot besides that though or who may have wrote it.