4 ms·

Research Notes on Humanoid Robots

25 points by liuliu 5y ago

TaylorAlexander 5y ago
I am a robotics engineer. Everyone wanted to mock Tesla for their announcement. And sure, it may well be vaporware. But I do think they have the resources to buy/build something that works, and moreover their Dojo AI system would possibly be well suited for training a humanoid. This is a major piece which other companies do not have. Finally they have the manufacturing capability to spin up a manufacturing line. However it is weird for them to make the announcement without any hardware to show. It’s kind of like me announcing I’m going to climb Mount Everest. I could, probably, but there is a lot to be done between then and now. I think Tesla could do it, if they really believe there is enough profit in it to make a massive investment. They could purchase Agility Robotics and be well on their way. So I think it’s possible, but only with significant investment. Time will tell what really happens.
- sovreign 5y ago
  I agree, time will tell. At least AI day was promoted more as a recruiting event rather than a showcase or product announcement. Agility Robotics really has a good chunk of the hardware figured out with Digit. Gonna be interesting to see how this plays out (and how quickly).
  - TaylorAlexander 5y ago
    Yep. Tesla has a fast path to success if they can buy Agility Robotics. Then they can slap their AI hardware on Digit, start training navigation in Dojo, rev the Digit design to integrate that AI hardware, and the hardware engineers will be spinning up production in parallel to the AI team training navigation. In one year they could be showing off a Tesla robot walking around the Tesla factory moving boxes (Agility already has this demo). I also just think Digit plausibly looks like a Tesla product (as opposed to Boston Dynamics Atlas).
- lootsauce 5y ago
  ok as someone in the field if they were to produce an humanoid how the hell would that work because I try to follow robotics and to produce something with the capabilities of a human is not something you see ANYONE doing well. Robotics actuators are all motors, gears/belts and to replicate human level functionality would require some serious advancements. Or maybe I'm unaware of some existing tech out there? These are not just nobody bothered to design one yet types of issues, they are fundamental physics problems you don't get to wave away with good showmanship.
  - TaylorAlexander 5y ago
    The robot I am most personally impressed with is Agility Robotics Digit. It is a real human like form factor (small human not huge!). If I recall correctly it is an energy efficient walking system too. https://www.youtube.com/watch?v=Jycdks836bY https://www.youtube.com/watch?v=Jycdks836bY This long continuous shot of walking from 2019 is nice too. They have improved their software a lot but the hardware was already in good shape. https://www.youtube.com/watch?v=_YrlR1iNVcQ https://www.youtube.com/watch?v=_YrlR1iNVcQ Meanwhile Boston Dynamics is doing quite well. https://www.youtube.com/watch?v=tF4DML7FIWk https://www.youtube.com/watch?v=tF4DML7FIWk Best path to success is Tesla buys Agility Robotics, revs the design for Digit to integrate the Tesla AI system, and spins up a manufacturing line. They could focus immediately on the high level AI problem rather than worrying about design. Could slap their AI system on digit while mechanical teams rev the new design so they get moving quickly. Disclosure: I do not work for any of these companies or any related company. I am making a farming robot (see my profile).
joe_the_user 5y ago
I think the Darpa Humanoid Robot Challenge illustrates how far humanoid robots are from a reality [1]. Some reasons I'd speculate for this: 1) The human gait, bipedal motion, is based on a maintain a dynamic balance with the ground - fall forward, catch yourself with one leg and use that catch to fall forward again, catching yourself with your other leg. Most robots "Frankestein walk" - keep a static, put out a leg, touch the ground, transfer the weight - this is inelegant, inefficient and often fails. So far, achieving something like this has been extreme computationally intensive. 2) Human interactions with the world depend on using this dynamic balance to accomplish other things. Without mastery dynamic balance, a two-legged device is inherent precarious and inept. 3) The value of a humanoid robot is in being able to be a drop-in replacement for a human, in a limited capacity. But the problem of robot-human interactions isn't really solved for either self-driving cars or wheeled robots navigating hospitals. 4) Human "hardware" is extraordinarily sensitive, robust and versatile. Vast improvements in robots would be needed to compete. And then you have the software problem. [1] https://www.popsci.com/darpa-robotics-challenge-was-bust-why-darpa-needs-try-again/ https://www.popsci.com/darpa-robotics-challenge-was-bust-why...
- liuliu 5y ago
  2015 is long time ago, hence the purpose of the article. From my research, I think at least 3 things changed: 1. We are getting much better at sensing the environment and construct realistic representations of these, mostly due to the advances in deep learning and interests in augmented reality. Best of all, many of these solutions are open-source; 2. Comparing to the onboard computations available today, I won't call dynamic walking a "computational expensive" task any more. We have Jetson AGX Xavier, which can do teraflops of floating-point computation; 3. We are better at battery and electric actuators. I think that today's humanoid, even made by universities, can probably perform much better in the same Darpa challenge with reasonable budget (1m or so).
  - TaylorAlexander 5y ago
    And if it's Tesla we're talking about, they have extremely powerful custom AI chips already in production in their vehicles, more powerful than AGX Xavier (I have one of those!). Tesla also has what is in my opinion a very good AI system (Dojo) for training machines designed to operate in the real world. I think it could work for a humanoid. And I pointed out in a neighbor comment but both Boston Dynamics and Agility Robotics use dynamic walks, not static walking. I agree they would do much better on the Darpa challenge too. This video from Agility shows some tasks similar to the Darpa challenge: https://www.youtube.com/watch?v=Jycdks836bY https://www.youtube.com/watch?v=Jycdks836bY Tesla is IMO in a good position to do something with humanoids especially if they buy Agility Robotics and slap their AI system into it. We know they can do the manufacturing and Musk does love to buy companies for their tech.
lootsauce 5y ago
IMHO most people fantasize about robots solving general use problems, as in the case of a humanoid, but in reality they will probably be ridiculously impractical compared to human labor. The problem is much like automation vs a manual process. Yes we can certainly automate this or that workflow just give us the specification of what to do, oh did you remember every quirk and nuance. Now to make this work without constant human interaction we actually need to know a lot about the problem, the problem space, the specific requirements of each task because the automation process does not have anything close to common sense reasoning capability. Without that every possible nuance must be explicitly detailed and our brains are not good at that. This is what much of software development is, even when given detailed specs, you find out that there are plenty of requirements yet to be discovered. This is a huge problem in AI. It's a complete non-starter of a design problem when making a robot. You really need to know what the hell the thing is going to be used for before you draw 1 line in the cad program. The fantasy of the humanoid model is to say look, humans can do a lot there's your design specs go make something that can do that. Good luck, first of all how much energy can you pack in a humanoid robot after affording for actuators and the support structures? How long will the thing last on a single charge? How much and what kind of work can it really do? How do you tell it to do some task? What can it really accomplish? Ok now look at the cost of an employee for one of the mundane jobs, their capabilities are greater, they have true intelligence so you don't need to specify arcane level of detail for requirements because they have common sense. Lets not even limit ourselves to humans, any general purpose robot will never be able to compete with a task specific robot and when you think about it, the mundane use cases are often the kind where you do the same thing over and over, aka a limited set of specialized tasks. Well design a robot to do just that and it will be cheaper and more effective than a humanoid robot. Carpet cleaning is one example, a roomba works just fine, no need for a cybernetic humanoid to accomplish that mundane task and the cost differential is huge.
- liuliu 5y ago
  My research notes does try to pose some of your concerns as challenges (rather than pose them as impossibility): > Yes we can certainly automate this or ... See "4. How does a humanoid robot operate?": https://liuliu.me/eyes/research-notes-on-humanoid-robots/#4-how-does-a-humanoid-robot-operate https://liuliu.me/eyes/research-notes-on-humanoid-robots/#4-... > Lets not even limit ourselves to humans, any general purpose robot will never be able to compete with a task specific robot ... See last two paragraphs of "Why": https://liuliu.me/eyes/research-notes-on-humanoid-robots/#why https://liuliu.me/eyes/research-notes-on-humanoid-robots/#wh... Otherwise: > Good luck, first of all how much energy can you pack in a humanoid robot after affording for actuators and the support structures? How long will the thing last on a single charge? Probably somewhere around 30 min to 1 hour on a single charge. If we can fast charge, this probably would be sufficient for home-use or in some commercial settings.
ilaksh 5y ago
I think that 3d printed artificial limbs/bodies with integrated artificial muscles could go a long way towards enabling dexterity and higher strength-to-weight ratios and other performance capabilities. For example if someone could do that with HASEL actuators like the ones manually made by Artimus. Another idea I had for computer vision indoors was to make something like a super fast constructive solid geometry part of the process. Because that's a really high level of scene understanding that could potentially make things more efficient and more accurate. Especially for indoor with a lot of very regular shapes. Not saying those things exist yet or are easy to do.

25 points by liuliu 5y ago

I am a robotics engineer. Everyone wanted to mock Tesla for their announcement. And sure, it may well be vaporware. But I do think they have the resources to buy/build something that works, and moreover their Dojo AI system would possibly be well suited for training a humanoid. This is a major piece which other companies do not have. Finally they have the manufacturing capability to spin up a manufacturing line. However it is weird for them to make the announcement without any hardware to show. It’s kind of like me announcing I’m going to climb Mount Everest. I could, probably, but there is a lot to be done between then and now. I think Tesla could do it, if they really believe there is enough profit in it to make a massive investment. They could purchase Agility Robotics and be well on their way. So I think it’s possible, but only with significant investment. Time will tell what really happens.

sovreign 5y ago
I agree, time will tell. At least AI day was promoted more as a recruiting event rather than a showcase or product announcement. Agility Robotics really has a good chunk of the hardware figured out with Digit. Gonna be interesting to see how this plays out (and how quickly).
- TaylorAlexander 5y ago
  Yep. Tesla has a fast path to success if they can buy Agility Robotics. Then they can slap their AI hardware on Digit, start training navigation in Dojo, rev the Digit design to integrate that AI hardware, and the hardware engineers will be spinning up production in parallel to the AI team training navigation. In one year they could be showing off a Tesla robot walking around the Tesla factory moving boxes (Agility already has this demo). I also just think Digit plausibly looks like a Tesla product (as opposed to Boston Dynamics Atlas).
lootsauce 5y ago
ok as someone in the field if they were to produce an humanoid how the hell would that work because I try to follow robotics and to produce something with the capabilities of a human is not something you see ANYONE doing well. Robotics actuators are all motors, gears/belts and to replicate human level functionality would require some serious advancements. Or maybe I'm unaware of some existing tech out there? These are not just nobody bothered to design one yet types of issues, they are fundamental physics problems you don't get to wave away with good showmanship.
- TaylorAlexander 5y ago
  The robot I am most personally impressed with is Agility Robotics Digit. It is a real human like form factor (small human not huge!). If I recall correctly it is an energy efficient walking system too. https://www.youtube.com/watch?v=Jycdks836bY https://www.youtube.com/watch?v=Jycdks836bY This long continuous shot of walking from 2019 is nice too. They have improved their software a lot but the hardware was already in good shape. https://www.youtube.com/watch?v=_YrlR1iNVcQ https://www.youtube.com/watch?v=_YrlR1iNVcQ Meanwhile Boston Dynamics is doing quite well. https://www.youtube.com/watch?v=tF4DML7FIWk https://www.youtube.com/watch?v=tF4DML7FIWk Best path to success is Tesla buys Agility Robotics, revs the design for Digit to integrate the Tesla AI system, and spins up a manufacturing line. They could focus immediately on the high level AI problem rather than worrying about design. Could slap their AI system on digit while mechanical teams rev the new design so they get moving quickly. Disclosure: I do not work for any of these companies or any related company. I am making a farming robot (see my profile).

joe_the_user 5y ago

I think the Darpa Humanoid Robot Challenge illustrates how far humanoid robots are from a reality [1]. Some reasons I'd speculate for this: 1) The human gait, bipedal motion, is based on a maintain a dynamic balance with the ground - fall forward, catch yourself with one leg and use that catch to fall forward again, catching yourself with your other leg. Most robots "Frankestein walk" - keep a static, put out a leg, touch the ground, transfer the weight - this is inelegant, inefficient and often fails. So far, achieving something like this has been extreme computationally intensive. 2) Human interactions with the world depend on using this dynamic balance to accomplish other things. Without mastery dynamic balance, a two-legged device is inherent precarious and inept. 3) The value of a humanoid robot is in being able to be a drop-in replacement for a human, in a limited capacity. But the problem of robot-human interactions isn't really solved for either self-driving cars or wheeled robots navigating hospitals. 4) Human "hardware" is extraordinarily sensitive, robust and versatile. Vast improvements in robots would be needed to compete. And then you have the software problem. [1] https://www.popsci.com/darpa-robotics-challenge-was-bust-why-darpa-needs-try-again/ https://www.popsci.com/darpa-robotics-challenge-was-bust-why...

liuliu 5y ago
2015 is long time ago, hence the purpose of the article. From my research, I think at least 3 things changed: 1. We are getting much better at sensing the environment and construct realistic representations of these, mostly due to the advances in deep learning and interests in augmented reality. Best of all, many of these solutions are open-source; 2. Comparing to the onboard computations available today, I won't call dynamic walking a "computational expensive" task any more. We have Jetson AGX Xavier, which can do teraflops of floating-point computation; 3. We are better at battery and electric actuators. I think that today's humanoid, even made by universities, can probably perform much better in the same Darpa challenge with reasonable budget (1m or so).
- TaylorAlexander 5y ago
  And if it's Tesla we're talking about, they have extremely powerful custom AI chips already in production in their vehicles, more powerful than AGX Xavier (I have one of those!). Tesla also has what is in my opinion a very good AI system (Dojo) for training machines designed to operate in the real world. I think it could work for a humanoid. And I pointed out in a neighbor comment but both Boston Dynamics and Agility Robotics use dynamic walks, not static walking. I agree they would do much better on the Darpa challenge too. This video from Agility shows some tasks similar to the Darpa challenge: https://www.youtube.com/watch?v=Jycdks836bY https://www.youtube.com/watch?v=Jycdks836bY Tesla is IMO in a good position to do something with humanoids especially if they buy Agility Robotics and slap their AI system into it. We know they can do the manufacturing and Musk does love to buy companies for their tech.

lootsauce 5y ago

IMHO most people fantasize about robots solving general use problems, as in the case of a humanoid, but in reality they will probably be ridiculously impractical compared to human labor. The problem is much like automation vs a manual process. Yes we can certainly automate this or that workflow just give us the specification of what to do, oh did you remember every quirk and nuance. Now to make this work without constant human interaction we actually need to know a lot about the problem, the problem space, the specific requirements of each task because the automation process does not have anything close to common sense reasoning capability. Without that every possible nuance must be explicitly detailed and our brains are not good at that. This is what much of software development is, even when given detailed specs, you find out that there are plenty of requirements yet to be discovered. This is a huge problem in AI. It's a complete non-starter of a design problem when making a robot. You really need to know what the hell the thing is going to be used for before you draw 1 line in the cad program. The fantasy of the humanoid model is to say look, humans can do a lot there's your design specs go make something that can do that. Good luck, first of all how much energy can you pack in a humanoid robot after affording for actuators and the support structures? How long will the thing last on a single charge? How much and what kind of work can it really do? How do you tell it to do some task? What can it really accomplish? Ok now look at the cost of an employee for one of the mundane jobs, their capabilities are greater, they have true intelligence so you don't need to specify arcane level of detail for requirements because they have common sense. Lets not even limit ourselves to humans, any general purpose robot will never be able to compete with a task specific robot and when you think about it, the mundane use cases are often the kind where you do the same thing over and over, aka a limited set of specialized tasks. Well design a robot to do just that and it will be cheaper and more effective than a humanoid robot. Carpet cleaning is one example, a roomba works just fine, no need for a cybernetic humanoid to accomplish that mundane task and the cost differential is huge.

liuliu 5y ago
My research notes does try to pose some of your concerns as challenges (rather than pose them as impossibility): > Yes we can certainly automate this or ... See "4. How does a humanoid robot operate?": https://liuliu.me/eyes/research-notes-on-humanoid-robots/#4-how-does-a-humanoid-robot-operate https://liuliu.me/eyes/research-notes-on-humanoid-robots/#4-... > Lets not even limit ourselves to humans, any general purpose robot will never be able to compete with a task specific robot ... See last two paragraphs of "Why": https://liuliu.me/eyes/research-notes-on-humanoid-robots/#why https://liuliu.me/eyes/research-notes-on-humanoid-robots/#wh... Otherwise: > Good luck, first of all how much energy can you pack in a humanoid robot after affording for actuators and the support structures? How long will the thing last on a single charge? Probably somewhere around 30 min to 1 hour on a single charge. If we can fast charge, this probably would be sufficient for home-use or in some commercial settings.

ilaksh 5y ago

I think that 3d printed artificial limbs/bodies with integrated artificial muscles could go a long way towards enabling dexterity and higher strength-to-weight ratios and other performance capabilities. For example if someone could do that with HASEL actuators like the ones manually made by Artimus. Another idea I had for computer vision indoors was to make something like a super fast constructive solid geometry part of the process. Because that's a really high level of scene understanding that could potentially make things more efficient and more accurate. Especially for indoor with a lot of very regular shapes. Not saying those things exist yet or are easy to do.