14 ms·

Learning Dexterity

470 points by gdb 8y ago

runesoerensen 8y ago
Very cool. There's also a Times article about Dactyl: https://www.nytimes.com/interactive/2018/07/30/technology/robot-hands.html https://www.nytimes.com/interactive/2018/07/30/technology/ro...
hellofunk 8y ago
Holy cow, the robots are definitely coming. We really are at the ground floor of a technology that is going to change humanity, I am certain of that. Changes greater than any changes we've seen before.
- aerovistae 8y ago
  indeed. the world 200 years from now will be as unrecognizably different as today from 1818.
  - chrisweekly 8y ago
    That seems unlikely. One way or another, I'd expect 2218 to be much, much more different from today than is 1818.
    - logfromblammo 8y ago
      The history of innovation compounds. One cannot improve on what hasn't been invented yet, but afterward, it can happen at any time afterward, even if the unimproved original has since been rendered obsolete. If you think of it in terms of "200 years ago" versus "200 years from now" you might think there would be equal magnitudes of progress. But now is 200 years since 1818, and 2218 is 400 years from 1818 AND 200 years from now. The future has more to build on, and taller shoulders to stand on. The center of history, at the exact moment where as much has changed since the idea "maybe we could just plant some seeds between the Tigris and Euphrates, and maybe not wander around so much," and from then until the present, may be shorter than 50 years ago, and getting closer every year. Thousands of years on one side of the balance, and tens of years on the other. With that in mind, today may be as different from 1818 as 2038 is from today. 2218 would be just inconceivable.
      - chrisweekly 8y ago
        Exactly.
- deviationblue 8y ago
  Well, if you're thinking sentient, AI beings, then no-- they are still a long way off. Unless we can give any meaning behind why a robot should do something, for example, have and use this kind of dexterity, it's all mechanical tricks. Cool tricks, nonetheless.
  - jabgrabdthrow 8y ago
    They don’t have to be fully sentient just more sentient than the bottom 10% of people, a goal function like energy storage with a counterfacuals algo will probably yield surprisingly familiar emergent behaviors
  - criddell 8y ago
    I agree with the person you replied to but I'm not thinking about intelligent machines. I'm just thinking about the automation of everything. Look how close we are to self driving cars without needing a sentient robot behind the wheel. Mechanical tricks are going to eliminate the jobs of a lot of people.
    - hk__2 8y ago
      > Look how close we are to self driving cars without needing a sentient robot behind the wheel. Mechanical tricks are going to eliminate the jobs of a lot of people. Let’s say we’re close to self-driving cars, i.e. it’ll happen in 10 years or so. How much will it cost? How much the maintainance will cost? How many years will be needed until everybody owns a self-driving car? Unless more than a handful people have that kind of car you won’t kill a lot of jobs.
    - kochikame 8y ago
      With this kind of dexterity, I would say Las Vegas croupiers should be getting a little worried and thinking about re-skilling. David Copperfield should be OK a while longer though
pmohun 8y ago
Fascinating: "We observed that for precision grasps, such as the Tip Pinch grasp, Dactyl uses the thumb and little finger. Humans tend to use the thumb and either the index or middle finger instead. However, the robot hand’s little finger is more flexible due to an extra degree of freedom, which may explain why Dactyl prefers it. This means that Dactyl can rediscover grasps found in humans, but adapt them to better fit the limitations and abilities of its own body." The learning of "emergent" behavior, specifically when it creates improvements to natural human motion is one of the main reasons why this type of work is so important. Similar to the way that we imitate design from nature (e.g. wings, suction cups), we can now accelerate development by observing how the bots perform the task in a variety of environments
- sgillen 8y ago
  It's a really cool phenomena, it also means we have to make our simulations better. These sorts of RL algorithms are so good at finding "exploits" in the physics engine they are in that help them "cheat" sometimes, compared to what the researcher wanted.
  - TaylorAlexander 8y ago
    I agree, but FWIW in the linked work they did randomize some of the engine parameters during training to avoid fitting too much to a specific set of assumptions. Certainly though more accurate simulations, as long as they were still fast to compute, would be very useful!
    - sgillen 8y ago
      yeah, I think a big reason why they needed to do randomization in the first place is because of inaccuracies in the simulation compared to the real world. And the extra randomization required almost two orders of magnitude more training time!
  - jashmenn 8y ago
    otoh, maybe finding "exploits" is all we ever do. I don't grip with my thumb and little finger because it hurts the back of my hand, but that isn't a problem for Dactyl.
0x8BADF00D 8y ago
> Rapid used 6144 CPU cores and 8 GPUs to train our policy, collecting about one hundred years of experience in 50 hours. That seemed an order of magnitude higher than I expected. Is training usually this computationally expensive?
- GlenTheMachine 8y ago
  The notable thing about this is how little computation it needed. We have a long way to go, but this is a big step in the right direction.
- LeanderK 8y ago
  haha I just read it and thought that's a magnitude less than I expected. Pretty often it is. A lot of papers from high profile institutions have a lot of computing power availiable. First, it seems to be a lot and be really expensive, but think of it in man-hours. It quickly diminishes.
  - visarga 8y ago
    Not to mention that evolution has had millions of years of optimizing this stuff.
- cyberpunk0 8y ago
  Not always but a lot of machine learning techniques now are effectively just brute forcing and highly expensive band wasteful to compute. Machine learning is getting close but I don't believe we will get there with our current methods
- sgillen 8y ago
  Well if you look at the plot in the "Learning progress" section, you'll see that they did require almost two orders of magnitude more training time due to the randomizations they were adding to the simulation. Without these the policy isn't very robust but also takes a lot less time to train.
- sigi45 8y ago
  Yeah but it doesn't matter; One model to rule them all.
tevlon 8y ago
This is a perfect example of how AI is taking over the world by storm. I don't know how people don't realize that there will be no jobs left for billions of people. Yes, billions. Not Millions. I don't quite get the "New Jobs will be created" fallacy. Let me explain: What is job? An Abstract way of looking at it: A job is something that requires a set skills to accomplish a task. What most politicians don't get: Researchers like OpenAi teach machines SKILLS not jobs. A little thought experiment: Let's say humans are capable of 100 skills. Skills can be anything from: driving, seeing, hearing, reading, walking, carrying, drawing etc. Usually, a low paying job requires little to no traning. For example: Someone in a warehouse that picks the stuff you have ordered. The skill that are required are: walking, picking and using a device. A High paying jobs usually requires more skills and/or experience. We train machines to see better, hear better, sort faster etc. Any new job will require some sort of skills out the set of skills that can be trained. But the moment you create this job, it will be automated, because a machine can do it better and faster. We need to adress this now, otherwise i don't see a bright future for the generations to come.
- greggyb 8y ago
  This is a response to you and all the various siblings, nieces and nephews. AI does not remove scarcity. Comparative advantage is a real thing. It is well studied and well understood. The common argument seems to conflate AI, automation, robotics, and similar things with the removal of scarcity. In the presence of scarcity, comparative advantage tells us that we're unlikely to see the vast majority of the world's population with nothing to do.
  - evv 8y ago
    Comparative advantage is exactly why so many people will loose their jobs. If a small number of people, with the aid of machines, can do the equivalent work of thousands of workers, how will the free market support the higher cost of human labour in comparison to the robots? Its not that people will have nothing to do. But for a huge number of people, there will be no way to get paid as much as it costs them to live.
dclowd9901 8y ago
The hand itself is an incredible piece of machinery.
tomxor 8y ago
I guess someone has to be the negative one: I can't help feeling it's route to the correct face looks entirely accidental (and I don't mean that in a good way)... I'm sure it's "learned" some methods, but they don't look that efficient, reliable, purposeful or controlled. In a more noisy and dynamic environment I'd expect them to fail. Granted is possible these could be more due to training conditions than an inherent limitation of the underlying model.
- toxik 8y ago
  Agreed, it looks really uncoordinated. A lot of reinforcement learning algorithms have this problem, in my experience.
- superfx 8y ago
  It looks that way because they're moving rapidly from one face configuration to another. But there's no way that's happening by random. I would guess that even just holding the cube constant in a dynamic grip is quite difficult.
- poppingtonic 8y ago
  Maybe if they randomized to n<=20 n-gons. I'd love to see Dactyl tackle a dodecahedron.
- dgreensp 8y ago
  I agree it looks sloppy, but that doesn’t mean it isn’t reliable. All it has to do in any given moment is make progress towards the goal of having the cube in the proper orientation, on average. It may be that it can do that very reliably even with noisy inputs and outputs.
kingbirdy 8y ago
> Learning to rotate an object in simulation without randomizations requires about 3 years of simulated experience It's interesting to me that this is about the same amount of time it takes humans to develop similar levels of motor control. I don't know enough about AI or neuroscience to say whether it's likely to be a coincidence or not, though.
- Ajedi32 8y ago
  Interesting observation. I suspect it's probably coincidence though. Other tasks which humans are able to learn (such as [playing Dota][1]) have taken OpenAI much longer to master. OpenAI Five spends 180 years of training per day, per hero in order to learn Dota, and it still isn't at the level of professional players (though that may change soon). Though I suppose you could argue that Dota benefits more from high-level reasoning, whereas basic motor control is a more intuitive skill. (And therefore better suited for this type of AI.) [1]: https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/
  - mortenjorck 8y ago
    Also, the time referenced in the article is presumably three years of non-stop training – given that an infant has a calendar packed with other things like sleeping, crying, and other non-motor activities, total human time logged on learning fine motor control is probably half that, if not less.
- andreyk 8y ago
  Basically coincidence; the models here are far simpler and do not at all mimic the human nervous system.
  - red75prime 8y ago
    I don't doubt that it is coincidence. But it's interesting that vertebrates' brains seems to be using specialized structure (cerebellum) for motor coordination. And here we have far simpler artificial system without any evolutionary priors.
Giho 8y ago
They should set up an accelerometer and gyroscope in each fingertip instead of pressure sensors. Could then maybe control without a camera.
- sgillen 8y ago
  They already have a good estimate of the finger tip pose from the angular measurements of all the internal degrees of freedom (presumably from angular encoders?). And I'm not sure any IMU small enough to fit into the fingertip will be accurate enough to provide really useful additional pose information.
sidcool 8y ago
I immediately thought of the robotic arm from Terminator 2. It's pretty cool.
preparedzebra 8y ago
This is a great example of why AI innovation is not moving at the pace we are told to believe. This is using the same basic algorithms we've known about for decades, just more compute and differently formulated problems. We need a paradigm shift!
- deleted 8y ago
  [deleted]
madeuptempacct 8y ago
Let's be honest - the only thing we care about is "are the programming jobs safe?!" Well, are they? P.S. I am trying to help a newer dev atm, and I realize I always have only one question for them while basically doing their work "What. the. hell. are. the. business. requirements?" Suddenly, this makes me feel much more like a business analyst than a code monkey, though being a decent code monkey is definitely a pre-req.
- crwalker 8y ago
  How apropos, if our own jobs are the first to go.
- hk__2 8y ago
  > Let's be honest - the only thing we care about is "are the programming jobs safe?!" Well, are they? As a sofware developer myself, I’d love to ever see a world where my work is not needed anymore. Wouldn’t that be awesome to have worked so hard toward automation that you can automate your own job?
habosa 8y ago
Any comments on why it seems to basically not use the middle finger at all?
- ryanmercer 8y ago
  It is polite. ;P
- pmulv 8y ago
  Wouldn't it just be a product of the model not finding that finger useful in the simulations?
- Menerve 8y ago
  I find it strangely correlated with the way the camera is set up. If it uses the middle finger then the camera might not see correctly the cube's face. You can see it using it at the last resort. But I don't see why this would matter in the simulation phase.
Symmetry 8y ago
That's very impressive. Robotic grasping is getting pretty good[1] but in-hand manipulation is a whole 'nother kettle of fish and this is really exciting. [1] He said, tooting his employer's horn.
andreyk 8y ago
Link to paper ( why no Arxiv :/ ): https://d4mucfpksywv.cloudfront.net/research-covers/learning-dexterity/learning-dexterity-paper.pdf https://d4mucfpksywv.cloudfront.net/research-covers/learning... TLDR (quick-ish skim, feel free to correct) they train a deep neural network to control a robot hand to choose desired joints state changes (binned into 11 discrete values; eg rotate this joint by 10 degrees) for a 20-joint hand given low-level (non-visual; so, current and desired 3D orientation of the object and exact numeric state of the joints) input of the state of a particular object and the hand. They also train a network to extract the 3D pose of a given object given RGB input. All this training is done in simulation with a ton of computation, and they use a technique called domain randomization (changing colors and textures and so friction coefficient and so on) to make these learned models pretty much work in the real world despite being trained only in simulation. It's pretty cool work, but if I may pull my reviewer hat on not that interesting in terms of new ideas - still, it's cool OpenAI is continuing to demonstrate what can be achieved today with established RL techniques and nice distributed compute.
- Eridrus 8y ago
  It's pretty amusing/amazing how well domain randomization works, but it seems like they train the pose detector in the real world and not in simulation: "To transfer to the real world, we predict the object pose from 3 real camera feeds with the CNN, measure the robot fingertip locations using a 3D motion capture system, and give both of these to the control policy to produce an action for the robot."
aeleos 8y ago
I am continually impressed by OpenAI, whenever we think that something is too difficult for our currently understanding of AI. With their Dota AI and this they have shown that more can be done with a lot less than previously thought.
- andreyk 8y ago
  Not to be too negative, it's cool work, but I'd argue unlike the OpenAI result it is not so surprising this was doable with the techniques they used ; see eg this paper from Google http://www.roboticsproceedings.org/rss14/p10.pdf http://www.roboticsproceedings.org/rss14/p10.pdf and this one from Stanford/DeepMind http://www.roboticsproceedings.org/rss14/p09.pdf http://www.roboticsproceedings.org/rss14/p09.pdf . Yes there is the additional aspects of an object in hand, but fundamentally the techniques are the same. Of course these works are cited in related works of paper as they should be; perhaps the OpenAI blog should also provide more context on where this stands wrt prior work, as many non-researchers may read this is may be quite misleading...
  - j2kun 8y ago
    OpenAI has not exactly had the best reputation with their press releases.
dokem 8y ago
Does anyone know a good graduate program/route for this kind of work? My undergrad was CS with some experience in (dumb) robotics and mechanical design but no ML. I am interested in applying ML/CV to physical systems like this however am I bit weary of going back to a CS program. I have seen some Mechanical programs with an emphasis on control that let you 'build your own degree'. If I could take a mix of ML/CV, control systems, kinematics I would be happy. Just looking for some input from people in this field.
- gdb 8y ago
  (I work at OpenAI.) Worth noting: it's a well-supported route to join OpenAI without any special graduate training. Many of our teams (including our robotics team!) hire experienced software engineers, teaching them whatever ML they need to know, or our Fellows program lets people do a more formal curriculum (https://blog.openai.com/openai-fellows/ https://blog.openai.com/openai-fellows/). We also have a number of software engineers who focus on what looks like traditional software engineering: see for example https://www.youtube.com/watch?v=UdIPveR__jw https://www.youtube.com/watch?v=UdIPveR__jw. See our open positions here: http://openai.com/jobs http://openai.com/jobs!
- andreyk 8y ago
  I am a 2nd year Phd at Stanford and you could definitely do such work here! Also at CMU, Georgia Tech, Berkeley, U Washington, and others. You can enter PhD via EE/MechE or CS - once you are focused on research it does not matter much. The FAIR/Google Residency programs may also be of interest.
ryanmercer 8y ago
Terrifyingly amazing.
westmeal 8y ago
Can't wait for my robo maid to roll a joint for me.
- sctb 8y ago
  Please stop posting low-effort one-liners here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html
kylek 8y ago
I'd like to see it roll a coin on its knuckles. Or maybe some card tricks.
Animats 8y ago
Nice. Take a look at position 44, where it seems to get stuck, with no move to make forward progress, and two fingers straight out. Did it lack image recognition to tell it what block rotation was needed? It doesn't seem to work by discovering strategies for rotating the block one face at a time, then combining those. It's solving the problem as a whole. That has both good and bad implications.
YeGoblynQueenne 8y ago
>> We’ve trained a human-like robot hand to manipulate physical objects with unprecedented dexterity. To be precise, the "physical objects" appear to invariably be cubes of the same dimensions. Not arbitrary "physical objects". Which is probably the best that can be done by training only in a simulated environment.
bambax 8y ago
> a human-like robot hand But why?? Why should robots' hands resemble human hands? They could have any number of fingers, or tentacles, or magnets, why should they be like human hands?? It seems "AI" really means "as close as possible to human behavior", even if we're not really that clever in said behavior. Also, human intelligence being at least debatable, it's not obvious that the obsessive imitation of humans is the best way to attain "AI".
- jorgemf 8y ago
  Because most objects are made for human hands. So it is better to have a robot with human-like hands (or body) than to change the shape of all the things we have already created. So future robots can use our human-made stuff.
- bytematic 8y ago
  We have built this world for human hands so that has become the best shape overall. It does depend on the situation but I'm guessing for many, an improved human shape is best.
  - mattigames 8y ago
    Yep, backwards compatibility, not just hands but bodies overall, if we ever make a robot that can drive existing cars it will pretty much resemble a human body, and our cities were created for human bodies/interactions (walking up stairs, etc)
- TaylorAlexander 8y ago
  Human hands are incredible manipulators. Really, compared to what’s in robotics today they’re fabulous. There’s some other promising developments in non human hand style manipulators, but human hands are still really good. It’s really good practice for your AI system to try it on a human hand. If you can make it work with a human hand you can probably make it work for other manipulators, but it’s a great place to start!
- deleted 8y ago
  [deleted]
shady-lady 8y ago
Has the pricing come down on these robotic hands? Anybody have ballpark cost for the Shadow Dexterous Hand - 100k, 300k?
- elsewhen 8y ago
  Pricing on robotic hands: http://www.androidworld.com/prod76.htm http://www.androidworld.com/prod76.htm
techVentureStar 8y ago
Nice work. I see a future of robots ruling homo sapiens vividly.
- kochikame 8y ago
  Dexterity is their secret weapon!
chubot 8y ago
Honest question: In the video, it looks like it works, but performs worse than about 90% of humans at the task of rotating a cube. On the other hand, Alpha Go or even a rudimentary chess program does better than 99.99% of all humans. So is it fair to say that deep learning is fundamentally missing something that humans do? Or that chess and Go are "easy" problems in some sense? (It seems like with "unlimited" training hours it could eventually be better than a human? Or is that a hardware issue?)
- yters 8y ago
  Moravec's paradox.
- Jarwain 8y ago
  Simulating the manipulation of a cube, and physics in general, is more complex than simulating a board game.
- olingern 8y ago
  I would say that the number of permutations (while many) in chess and go are finite, while this has to adapt to anything "at hand." addendum: there are most definitely a greater number of environmental factors (x, y, z axes) involved in solving such problems.
- dgreensp 8y ago
  In 2015, it was commonly thought that it would still be decades before a computer could beat a top human player at Go. Now, you are calling it “easy,” because it’s been done. The first chess program was written by Alan Turing on paper between 1948 and 1950. He didn’t have a computer to run it, but he could still play a game with it by stepping through the algorithm by hand. In 1997, Deep Blue beat Kasparov, using traditional algorithms and not deep learning. Clearly there are differences between these problems and dexterity. Chess, for example, can be described relatively simply using logic, and there is no dynamic or physical element; a rudimentary player can be written using pencil and paper; a winning player just needs enough compute power, apparently. More importantly, there is a technology curve. You are asking about the ultimate limits of a technique moments after its first success puts it at the low end of the spectrum of human ability. Give it a decade or two. I am just shocked the video was real-time and not sped up like so many of these videos are (eg watch a robot arm fold a shirt in thirty seconds when you play it at 5x speed).
Albert500 8y ago
WE HAVE 100% REFUND POLICY IF WE ARE UNABLE TO COMPLETE THE JOB.GET RESULTS IN 24 HOURS TIME FOR ALL KIND OF EMAIL HACK.Change School Grades? Hack Banks? Erase Criminal Records? Hack Websites? Hack Database? Hack Drivers license? Hack Call Log? Hack Visichat and Flash chat Rooms? Hack FTP User and Pass? Hack Facebook,Whatssap,Twitter,Instagram,Webcam etc.Hack VB Forum? Hack WordPress Blog? Hack CC any Country? Hack Money Booker Account? Hack Liberty Reverse Account? Hack PayPal Account? Root Server? By Pass Google Phone Verification? Install Red on Linux Server? Hash Crack? DDOS Server? Job completion in 24hours TEXT/whatsapp: +1 786 708 9974 EMAIL: theredhackergroup@gmail com

470 points by gdb 8y ago

Very cool. There's also a Times article about Dactyl: https://www.nytimes.com/interactive/2018/07/30/technology/robot-hands.html https://www.nytimes.com/interactive/2018/07/30/technology/ro...

hellofunk 8y ago

Holy cow, the robots are definitely coming. We really are at the ground floor of a technology that is going to change humanity, I am certain of that. Changes greater than any changes we've seen before.

aerovistae 8y ago
indeed. the world 200 years from now will be as unrecognizably different as today from 1818.
- chrisweekly 8y ago
  That seems unlikely. One way or another, I'd expect 2218 to be much, much more different from today than is 1818.
  - logfromblammo 8y ago
    The history of innovation compounds. One cannot improve on what hasn't been invented yet, but afterward, it can happen at any time afterward, even if the unimproved original has since been rendered obsolete. If you think of it in terms of "200 years ago" versus "200 years from now" you might think there would be equal magnitudes of progress. But now is 200 years since 1818, and 2218 is 400 years from 1818 AND 200 years from now. The future has more to build on, and taller shoulders to stand on. The center of history, at the exact moment where as much has changed since the idea "maybe we could just plant some seeds between the Tigris and Euphrates, and maybe not wander around so much," and from then until the present, may be shorter than 50 years ago, and getting closer every year. Thousands of years on one side of the balance, and tens of years on the other. With that in mind, today may be as different from 1818 as 2038 is from today. 2218 would be just inconceivable.
    - chrisweekly 8y ago
      Exactly.
deviationblue 8y ago
Well, if you're thinking sentient, AI beings, then no-- they are still a long way off. Unless we can give any meaning behind why a robot should do something, for example, have and use this kind of dexterity, it's all mechanical tricks. Cool tricks, nonetheless.
- jabgrabdthrow 8y ago
  They don’t have to be fully sentient just more sentient than the bottom 10% of people, a goal function like energy storage with a counterfacuals algo will probably yield surprisingly familiar emergent behaviors
- criddell 8y ago
  I agree with the person you replied to but I'm not thinking about intelligent machines. I'm just thinking about the automation of everything. Look how close we are to self driving cars without needing a sentient robot behind the wheel. Mechanical tricks are going to eliminate the jobs of a lot of people.
  - hk__2 8y ago
    > Look how close we are to self driving cars without needing a sentient robot behind the wheel. Mechanical tricks are going to eliminate the jobs of a lot of people. Let’s say we’re close to self-driving cars, i.e. it’ll happen in 10 years or so. How much will it cost? How much the maintainance will cost? How many years will be needed until everybody owns a self-driving car? Unless more than a handful people have that kind of car you won’t kill a lot of jobs.
  - kochikame 8y ago
    With this kind of dexterity, I would say Las Vegas croupiers should be getting a little worried and thinking about re-skilling. David Copperfield should be OK a while longer though

pmohun 8y ago

Fascinating: "We observed that for precision grasps, such as the Tip Pinch grasp, Dactyl uses the thumb and little finger. Humans tend to use the thumb and either the index or middle finger instead. However, the robot hand’s little finger is more flexible due to an extra degree of freedom, which may explain why Dactyl prefers it. This means that Dactyl can rediscover grasps found in humans, but adapt them to better fit the limitations and abilities of its own body." The learning of "emergent" behavior, specifically when it creates improvements to natural human motion is one of the main reasons why this type of work is so important. Similar to the way that we imitate design from nature (e.g. wings, suction cups), we can now accelerate development by observing how the bots perform the task in a variety of environments

sgillen 8y ago
It's a really cool phenomena, it also means we have to make our simulations better. These sorts of RL algorithms are so good at finding "exploits" in the physics engine they are in that help them "cheat" sometimes, compared to what the researcher wanted.
- TaylorAlexander 8y ago
  I agree, but FWIW in the linked work they did randomize some of the engine parameters during training to avoid fitting too much to a specific set of assumptions. Certainly though more accurate simulations, as long as they were still fast to compute, would be very useful!
  - sgillen 8y ago
    yeah, I think a big reason why they needed to do randomization in the first place is because of inaccuracies in the simulation compared to the real world. And the extra randomization required almost two orders of magnitude more training time!
- jashmenn 8y ago
  otoh, maybe finding "exploits" is all we ever do. I don't grip with my thumb and little finger because it hurts the back of my hand, but that isn't a problem for Dactyl.

0x8BADF00D 8y ago

> Rapid used 6144 CPU cores and 8 GPUs to train our policy, collecting about one hundred years of experience in 50 hours. That seemed an order of magnitude higher than I expected. Is training usually this computationally expensive?

GlenTheMachine 8y ago
The notable thing about this is how little computation it needed. We have a long way to go, but this is a big step in the right direction.
LeanderK 8y ago
haha I just read it and thought that's a magnitude less than I expected. Pretty often it is. A lot of papers from high profile institutions have a lot of computing power availiable. First, it seems to be a lot and be really expensive, but think of it in man-hours. It quickly diminishes.
- visarga 8y ago
  Not to mention that evolution has had millions of years of optimizing this stuff.
cyberpunk0 8y ago
Not always but a lot of machine learning techniques now are effectively just brute forcing and highly expensive band wasteful to compute. Machine learning is getting close but I don't believe we will get there with our current methods
sgillen 8y ago
Well if you look at the plot in the "Learning progress" section, you'll see that they did require almost two orders of magnitude more training time due to the randomizations they were adding to the simulation. Without these the policy isn't very robust but also takes a lot less time to train.
sigi45 8y ago
Yeah but it doesn't matter; One model to rule them all.

tevlon 8y ago

This is a perfect example of how AI is taking over the world by storm. I don't know how people don't realize that there will be no jobs left for billions of people. Yes, billions. Not Millions. I don't quite get the "New Jobs will be created" fallacy. Let me explain: What is job? An Abstract way of looking at it: A job is something that requires a set skills to accomplish a task. What most politicians don't get: Researchers like OpenAi teach machines SKILLS not jobs. A little thought experiment: Let's say humans are capable of 100 skills. Skills can be anything from: driving, seeing, hearing, reading, walking, carrying, drawing etc. Usually, a low paying job requires little to no traning. For example: Someone in a warehouse that picks the stuff you have ordered. The skill that are required are: walking, picking and using a device. A High paying jobs usually requires more skills and/or experience. We train machines to see better, hear better, sort faster etc. Any new job will require some sort of skills out the set of skills that can be trained. But the moment you create this job, it will be automated, because a machine can do it better and faster. We need to adress this now, otherwise i don't see a bright future for the generations to come.

greggyb 8y ago
This is a response to you and all the various siblings, nieces and nephews. AI does not remove scarcity. Comparative advantage is a real thing. It is well studied and well understood. The common argument seems to conflate AI, automation, robotics, and similar things with the removal of scarcity. In the presence of scarcity, comparative advantage tells us that we're unlikely to see the vast majority of the world's population with nothing to do.
- evv 8y ago
  Comparative advantage is exactly why so many people will loose their jobs. If a small number of people, with the aid of machines, can do the equivalent work of thousands of workers, how will the free market support the higher cost of human labour in comparison to the robots? Its not that people will have nothing to do. But for a huge number of people, there will be no way to get paid as much as it costs them to live.

dclowd9901 8y ago

The hand itself is an incredible piece of machinery.

tomxor 8y ago

I guess someone has to be the negative one: I can't help feeling it's route to the correct face looks entirely accidental (and I don't mean that in a good way)... I'm sure it's "learned" some methods, but they don't look that efficient, reliable, purposeful or controlled. In a more noisy and dynamic environment I'd expect them to fail. Granted is possible these could be more due to training conditions than an inherent limitation of the underlying model.

toxik 8y ago
Agreed, it looks really uncoordinated. A lot of reinforcement learning algorithms have this problem, in my experience.
superfx 8y ago
It looks that way because they're moving rapidly from one face configuration to another. But there's no way that's happening by random. I would guess that even just holding the cube constant in a dynamic grip is quite difficult.
poppingtonic 8y ago
Maybe if they randomized to n<=20 n-gons. I'd love to see Dactyl tackle a dodecahedron.
dgreensp 8y ago
I agree it looks sloppy, but that doesn’t mean it isn’t reliable. All it has to do in any given moment is make progress towards the goal of having the cube in the proper orientation, on average. It may be that it can do that very reliably even with noisy inputs and outputs.

kingbirdy 8y ago

> Learning to rotate an object in simulation without randomizations requires about 3 years of simulated experience It's interesting to me that this is about the same amount of time it takes humans to develop similar levels of motor control. I don't know enough about AI or neuroscience to say whether it's likely to be a coincidence or not, though.

Ajedi32 8y ago
Interesting observation. I suspect it's probably coincidence though. Other tasks which humans are able to learn (such as [playing Dota][1]) have taken OpenAI much longer to master. OpenAI Five spends 180 years of training per day, per hero in order to learn Dota, and it still isn't at the level of professional players (though that may change soon). Though I suppose you could argue that Dota benefits more from high-level reasoning, whereas basic motor control is a more intuitive skill. (And therefore better suited for this type of AI.) [1]: https://blog.openai.com/openai-five/ https://blog.openai.com/openai-five/
- mortenjorck 8y ago
  Also, the time referenced in the article is presumably three years of non-stop training – given that an infant has a calendar packed with other things like sleeping, crying, and other non-motor activities, total human time logged on learning fine motor control is probably half that, if not less.
andreyk 8y ago
Basically coincidence; the models here are far simpler and do not at all mimic the human nervous system.
- red75prime 8y ago
  I don't doubt that it is coincidence. But it's interesting that vertebrates' brains seems to be using specialized structure (cerebellum) for motor coordination. And here we have far simpler artificial system without any evolutionary priors.

Giho 8y ago

They should set up an accelerometer and gyroscope in each fingertip instead of pressure sensors. Could then maybe control without a camera.

sgillen 8y ago
They already have a good estimate of the finger tip pose from the angular measurements of all the internal degrees of freedom (presumably from angular encoders?). And I'm not sure any IMU small enough to fit into the fingertip will be accurate enough to provide really useful additional pose information.

sidcool 8y ago

I immediately thought of the robotic arm from Terminator 2. It's pretty cool.

preparedzebra 8y ago

This is a great example of why AI innovation is not moving at the pace we are told to believe. This is using the same basic algorithms we've known about for decades, just more compute and differently formulated problems. We need a paradigm shift!

deleted 8y ago
[deleted]

madeuptempacct 8y ago

Let's be honest - the only thing we care about is "are the programming jobs safe?!" Well, are they? P.S. I am trying to help a newer dev atm, and I realize I always have only one question for them while basically doing their work "What. the. hell. are. the. business. requirements?" Suddenly, this makes me feel much more like a business analyst than a code monkey, though being a decent code monkey is definitely a pre-req.

crwalker 8y ago
How apropos, if our own jobs are the first to go.
hk__2 8y ago
> Let's be honest - the only thing we care about is "are the programming jobs safe?!" Well, are they? As a sofware developer myself, I’d love to ever see a world where my work is not needed anymore. Wouldn’t that be awesome to have worked so hard toward automation that you can automate your own job?

habosa 8y ago

Any comments on why it seems to basically not use the middle finger at all?

ryanmercer 8y ago
It is polite. ;P
pmulv 8y ago
Wouldn't it just be a product of the model not finding that finger useful in the simulations?
Menerve 8y ago
I find it strangely correlated with the way the camera is set up. If it uses the middle finger then the camera might not see correctly the cube's face. You can see it using it at the last resort. But I don't see why this would matter in the simulation phase.

Symmetry 8y ago

That's very impressive. Robotic grasping is getting pretty good[1] but in-hand manipulation is a whole 'nother kettle of fish and this is really exciting. [1] He said, tooting his employer's horn.

andreyk 8y ago

Link to paper ( why no Arxiv :/ ): https://d4mucfpksywv.cloudfront.net/research-covers/learning-dexterity/learning-dexterity-paper.pdf https://d4mucfpksywv.cloudfront.net/research-covers/learning... TLDR (quick-ish skim, feel free to correct) they train a deep neural network to control a robot hand to choose desired joints state changes (binned into 11 discrete values; eg rotate this joint by 10 degrees) for a 20-joint hand given low-level (non-visual; so, current and desired 3D orientation of the object and exact numeric state of the joints) input of the state of a particular object and the hand. They also train a network to extract the 3D pose of a given object given RGB input. All this training is done in simulation with a ton of computation, and they use a technique called domain randomization (changing colors and textures and so friction coefficient and so on) to make these learned models pretty much work in the real world despite being trained only in simulation. It's pretty cool work, but if I may pull my reviewer hat on not that interesting in terms of new ideas - still, it's cool OpenAI is continuing to demonstrate what can be achieved today with established RL techniques and nice distributed compute.

Eridrus 8y ago
It's pretty amusing/amazing how well domain randomization works, but it seems like they train the pose detector in the real world and not in simulation: "To transfer to the real world, we predict the object pose from 3 real camera feeds with the CNN, measure the robot fingertip locations using a 3D motion capture system, and give both of these to the control policy to produce an action for the robot."

aeleos 8y ago

I am continually impressed by OpenAI, whenever we think that something is too difficult for our currently understanding of AI. With their Dota AI and this they have shown that more can be done with a lot less than previously thought.

andreyk 8y ago
Not to be too negative, it's cool work, but I'd argue unlike the OpenAI result it is not so surprising this was doable with the techniques they used ; see eg this paper from Google http://www.roboticsproceedings.org/rss14/p10.pdf http://www.roboticsproceedings.org/rss14/p10.pdf and this one from Stanford/DeepMind http://www.roboticsproceedings.org/rss14/p09.pdf http://www.roboticsproceedings.org/rss14/p09.pdf . Yes there is the additional aspects of an object in hand, but fundamentally the techniques are the same. Of course these works are cited in related works of paper as they should be; perhaps the OpenAI blog should also provide more context on where this stands wrt prior work, as many non-researchers may read this is may be quite misleading...
- j2kun 8y ago
  OpenAI has not exactly had the best reputation with their press releases.

dokem 8y ago

Does anyone know a good graduate program/route for this kind of work? My undergrad was CS with some experience in (dumb) robotics and mechanical design but no ML. I am interested in applying ML/CV to physical systems like this however am I bit weary of going back to a CS program. I have seen some Mechanical programs with an emphasis on control that let you 'build your own degree'. If I could take a mix of ML/CV, control systems, kinematics I would be happy. Just looking for some input from people in this field.

gdb 8y ago
(I work at OpenAI.) Worth noting: it's a well-supported route to join OpenAI without any special graduate training. Many of our teams (including our robotics team!) hire experienced software engineers, teaching them whatever ML they need to know, or our Fellows program lets people do a more formal curriculum (https://blog.openai.com/openai-fellows/ https://blog.openai.com/openai-fellows/). We also have a number of software engineers who focus on what looks like traditional software engineering: see for example https://www.youtube.com/watch?v=UdIPveR__jw https://www.youtube.com/watch?v=UdIPveR__jw. See our open positions here: http://openai.com/jobs http://openai.com/jobs!
andreyk 8y ago
I am a 2nd year Phd at Stanford and you could definitely do such work here! Also at CMU, Georgia Tech, Berkeley, U Washington, and others. You can enter PhD via EE/MechE or CS - once you are focused on research it does not matter much. The FAIR/Google Residency programs may also be of interest.

ryanmercer 8y ago

Terrifyingly amazing.

westmeal 8y ago

Can't wait for my robo maid to roll a joint for me.

sctb 8y ago
Please stop posting low-effort one-liners here. https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html

kylek 8y ago

I'd like to see it roll a coin on its knuckles. Or maybe some card tricks.

Animats 8y ago

Nice. Take a look at position 44, where it seems to get stuck, with no move to make forward progress, and two fingers straight out. Did it lack image recognition to tell it what block rotation was needed? It doesn't seem to work by discovering strategies for rotating the block one face at a time, then combining those. It's solving the problem as a whole. That has both good and bad implications.

YeGoblynQueenne 8y ago

>> We’ve trained a human-like robot hand to manipulate physical objects with unprecedented dexterity. To be precise, the "physical objects" appear to invariably be cubes of the same dimensions. Not arbitrary "physical objects". Which is probably the best that can be done by training only in a simulated environment.

bambax 8y ago

> a human-like robot hand But why?? Why should robots' hands resemble human hands? They could have any number of fingers, or tentacles, or magnets, why should they be like human hands?? It seems "AI" really means "as close as possible to human behavior", even if we're not really that clever in said behavior. Also, human intelligence being at least debatable, it's not obvious that the obsessive imitation of humans is the best way to attain "AI".

jorgemf 8y ago
Because most objects are made for human hands. So it is better to have a robot with human-like hands (or body) than to change the shape of all the things we have already created. So future robots can use our human-made stuff.
bytematic 8y ago
We have built this world for human hands so that has become the best shape overall. It does depend on the situation but I'm guessing for many, an improved human shape is best.
- mattigames 8y ago
  Yep, backwards compatibility, not just hands but bodies overall, if we ever make a robot that can drive existing cars it will pretty much resemble a human body, and our cities were created for human bodies/interactions (walking up stairs, etc)
TaylorAlexander 8y ago
Human hands are incredible manipulators. Really, compared to what’s in robotics today they’re fabulous. There’s some other promising developments in non human hand style manipulators, but human hands are still really good. It’s really good practice for your AI system to try it on a human hand. If you can make it work with a human hand you can probably make it work for other manipulators, but it’s a great place to start!
deleted 8y ago
[deleted]

shady-lady 8y ago

Has the pricing come down on these robotic hands? Anybody have ballpark cost for the Shadow Dexterous Hand - 100k, 300k?

elsewhen 8y ago
Pricing on robotic hands: http://www.androidworld.com/prod76.htm http://www.androidworld.com/prod76.htm

techVentureStar 8y ago

Nice work. I see a future of robots ruling homo sapiens vividly.

kochikame 8y ago
Dexterity is their secret weapon!

chubot 8y ago

Honest question: In the video, it looks like it works, but performs worse than about 90% of humans at the task of rotating a cube. On the other hand, Alpha Go or even a rudimentary chess program does better than 99.99% of all humans. So is it fair to say that deep learning is fundamentally missing something that humans do? Or that chess and Go are "easy" problems in some sense? (It seems like with "unlimited" training hours it could eventually be better than a human? Or is that a hardware issue?)

yters 8y ago
Moravec's paradox.
Jarwain 8y ago
Simulating the manipulation of a cube, and physics in general, is more complex than simulating a board game.
olingern 8y ago
I would say that the number of permutations (while many) in chess and go are finite, while this has to adapt to anything "at hand." addendum: there are most definitely a greater number of environmental factors (x, y, z axes) involved in solving such problems.
dgreensp 8y ago
In 2015, it was commonly thought that it would still be decades before a computer could beat a top human player at Go. Now, you are calling it “easy,” because it’s been done. The first chess program was written by Alan Turing on paper between 1948 and 1950. He didn’t have a computer to run it, but he could still play a game with it by stepping through the algorithm by hand. In 1997, Deep Blue beat Kasparov, using traditional algorithms and not deep learning. Clearly there are differences between these problems and dexterity. Chess, for example, can be described relatively simply using logic, and there is no dynamic or physical element; a rudimentary player can be written using pencil and paper; a winning player just needs enough compute power, apparently. More importantly, there is a technology curve. You are asking about the ultimate limits of a technique moments after its first success puts it at the low end of the spectrum of human ability. Give it a decade or two. I am just shocked the video was real-time and not sped up like so many of these videos are (eg watch a robot arm fold a shirt in thirty seconds when you play it at 5x speed).

Albert500 8y ago

WE HAVE 100% REFUND POLICY IF WE ARE UNABLE TO COMPLETE THE JOB.GET RESULTS IN 24 HOURS TIME FOR ALL KIND OF EMAIL HACK.Change School Grades? Hack Banks? Erase Criminal Records? Hack Websites? Hack Database? Hack Drivers license? Hack Call Log? Hack Visichat and Flash chat Rooms? Hack FTP User and Pass? Hack Facebook,Whatssap,Twitter,Instagram,Webcam etc.Hack VB Forum? Hack WordPress Blog? Hack CC any Country? Hack Money Booker Account? Hack Liberty Reverse Account? Hack PayPal Account? Root Server? By Pass Google Phone Verification? Install Red on Linux Server? Hash Crack? DDOS Server? Job completion in 24hours TEXT/whatsapp: +1 786 708 9974 EMAIL: theredhackergroup@gmail com