11 ms·
Robots that learn
- gallerdude 9y agoImagine how revolutionary a robot would be that could simply walk around, pick things up, and set things down. "Herbert: grab my car keys." "Herbert: set the dinner table." "Herbert: put the mail in the mailbox." Obviously, cost will be a big deal, but I think we'll get there.
- shahbaby 9y agoOnce we reach that point, the robot will be running on a far more advanced algorithm, closer to the one you're using. Consequences will likely be as profound and unpredictable as the internet turned out to be.
- mikeash 9y agoIf the internet is any guide, such a robot would find itself being used mostly for petting, feeding, and playing with cats.
- xj9 9y agowhat happens when the bot gets tired of being your slave?
- gboudrias 9y agoProbably best not to code that in.
- Houshalter 9y agoIt's a general artificial intelligence. You don't need to code it to do anything! It will learn to conquer puny humans all on it's own.
- namaemuta 9y ago"Tired" is a human feeling. A robot would never be tired of doing anything.
- xj9 9y agook, what happens when the bot decides it isn't in its best interest to continue to be your slave?
- robotresearcher 9y ago> Once we reach that point, the robot will be running on a far more advanced algorithm, closer to the one you're using. I'm using one algorithm? That's a rather controversial claim. And it's advanced? How so?
- IanCal 9y agoIs it a particularly controversial claim? You can describe any collection of algorithms as a single, more complicated, one. And it seems reasonable to say that humans are quite advanced given we can't yet replicate some of their reasoning capabilities.
- sillysaurus3 9y agoThe central problem with this idea is noise and creepiness. Nobody wants a noisy robot, and roboticists tend to produce creepy ones. Both problems might be solvable, but I think robotics tends to focus on the technical problems at the exclusion of the aesthetics. It's definitely a startup idea.
- bitdivision 9y agoDo you mean acoustic noise? Both noise and aesthetics are far simpler problems to solve than autonomous learning. I don't think there's much of a market for realistic, quiet robots until the central problem has been solved.
- grzm 9y agoWhat you're describing are separate concerns that I'm sure will be addressed before they're brought to market. Currently they're focussed on getting the robot to be able to learn from example. Spending a lot of time on aesthetics at the expense of getting the learning to work doesn't make sense at this point. You're exactly right that they're focussing on the technical problems: that's what needs to be solved. Once it works as expected, then they can focus on improving the aesthetics. A startup that focussed on making robots that look good but aren't effective at accomplishing tasks would likely have a very limited market.
- sillysaurus3 9y agoYeah, fair enough. I was thinking that BigDog was an example of this that works today, but the military passed on it because it was too noisy. But it didn't really do anything but move around.
- enraged_camel 9y agoThat's because the aesthetics are relatively easy to improve once the difficult technical problems are solved. We're still in the research phase, so polish isn't important (unless perhaps if giving a demo to an outside investor who isn't tech-savvy).
- Herodotus38 9y ago
- gnaritas 9y ago> "Herbert: grab my car keys." "Herbert: set the dinner table." "Herbert: put the mail in the mailbox." Doing that requires far more than simply being able to pick things up and put things down and walk around. To do that requires nearly human level AI that can "understand".
- Houshalter 9y agoNot really. You just need some basic natural language processing to recognize what task needs to be done. There's some stuff like this already: https://www.youtube.com/watch?v=54HpAmzaIbs https://www.youtube.com/watch?v=54HpAmzaIbs
- gnaritas 9y agoNo, really. Natural language processing isn't going to allow it to see and recognize the thing, nor navigate to the thing; it'll take vastly more complication than the OP implied to do something as simple as pick up something when instructed.
- fooker 9y ago"Grab my son from his play room."
- d--b 9y agoYes but to get to that point, Herbert needs to know the physics of the world, cause the mail may be under the kid's football which would drop on the floor of the mail is pulled and bounce around and break some things. And the robots also needs to know about making the right decision: is it better to let the ball break the China vase than making a move that would hit the dog in the face even not very hard? In order to get there, you need a robot that knows a lot of stuff.
- tdb7893 9y agoI think for the example with the China you would just have it do it's best to avoid things and if it breaks stuff then it breaks stuff. Unless you're robot breaks a ton of stuff in normal operation I don't see how being unable to choose between different things to break being a large issue
- j2kun 9y agoHaving to anticipate how the robot will screw up and reorganize your life based around that is a big issue. It's a big enough reason for people to decide they need to give up a dog they love, and people love dogs a lot more than they love robots.
- anonymfus 9y agoBut benefits of having dogs are mostly psychological and so dog owners are a minority. Since robots will have economical benefits people will use them even if it will make their lives worse.
- ulber 9y agoThis is placing a much higher standard on robots than humans are held to: to demonstrate that robots have to make hard decisions you presented a scenario where a human making either decision would have socially acceptable justifications available. But that just may be the key: maybe robots don't have to have superhuman decision making as long as they can produce humanly acceptable justifications for their decisions. That might still be hard, but at least you don't have to do that in a split second.
- ragebol 9y agoThis is actually getting there, at a competition called RoboCup@Home [0] Some videos from that competition: - https://www.youtube.com/watch?v=3jmsFmXzyzk https://www.youtube.com/watch?v=3jmsFmXzyzk - https://www.youtube.com/watch?v=Csd9USPxpDs https://www.youtube.com/watch?v=Csd9USPxpDs [0] https://github.com/RoboCupAtHome/RuleBook https://github.com/RoboCupAtHome/RuleBook
- deepnotderp 9y agoMore awesome stuff from OpenAI!
- 11thEarlOfMar 9y agoNow. Make me a peanut butter and jelly sandwich.
- jayjay71 9y agoYou should google bistrobot
- rl3 9y agoIt's not long before a huge chunk of food service jobs will be automated away by robotics, and that will likely include sandwich making.
- H1Supreme 9y agoThat's all I want a homebot for. "Homebot, make me some Gordon Ramsey level Beef Wellington". Beep boop beep, "Ready in 40 minutes".
- ldp01 9y agoTo be honest I think automated sandwich/burger making has been possible for a long time already. But fast food level labour is just so cheap that it never caught on.
- Practicality 9y agoI still want one that folds my laundry.
- wonderous 9y agoReminds me of this robot that was trained in a simulation, then in real life, "Autonomous Drifting using Machine Learning": https://m.youtube.com/watch?v=opsmd5yuBF0 https://m.youtube.com/watch?v=opsmd5yuBF0
- zardo 9y agoLooks like "Blocks World" with learning. Can you out stack SHRDLU?
- tomdre 9y agoWhat are the benefits of using VR vs simulation or precollected training data? Maybe I'm missing something...
- deleted 9y ago[deleted]
- komaromy 9y agovs simulation, it lets you get trajectories from the way people actually move without having to make insanely complicated models. vs. precollected training data, maybe the recording apparatus is less expensive/complex?
- npew 9y agoWe actually use both. The data used to train the neural networks is all generated through a scripted policy simulation. We then use a single human demonstration in VR to show the robot how to carry out a particular task. VR lets a human do a demonstration quite naturally, and would also be an effective way to collect training data for more complex tasks, where we’re not be able to create a scripted policy.
- tomdre 9y agoThanks for your reply. So essentially VR doesn't affect the training phase per se, but simplifies the life of the trainer.
- zardo 9y agoIt's demonstrating one-shot learning. If you want the robot to do something that you can do, just doing it while the robot watches is a lot easier than building simulations. It's not easy to get a million training examples, if you can learn something useful from just one example, that's great.
- tomdre 9y agoNot sure this makes sense because if you can learn from one example, then you can also learn from one single simulation (assuming you have a model).
- porter 9y agocan anyone recommend a good intelligent robot arm kit for building myself?
- dbcurtis 9y agoDepends on your goals and budget. I've seen a lot of cheap hobby servo based arms. Precision is awful, payload is miniscule. OTOH, you will learn why precision is important :) and you will learn the software of arm motion planning. The next step up in cheap arms is to build around Bioloid/Dynamixel style servos, which will increase the budget significantly, but you will end up with useable but coarse precision. Meaningful payloads will still cause servos to overheat but at least those brands will shut down rather than smoke -- usually. If you want to do serious research you will need a serious budget. Arms are hard. This is not to say you can't have a lot of fun and learn a lot with something less. A friend build a smart task lamp around Bioloids -- the goals was a lamp that would autonomously aim a work light where he was soldering by following the soldering iron tip using CV and a cheap web cam. This is totally within the payload and precision limits of hobby servos, and the software can run on a RasPi.
- cr0sh 9y agoBased on reviews I have read, this arm is probably the best you can get on a budget: http://dobot.cc/dobot-magician/product-overview.html http://dobot.cc/dobot-magician/product-overview.html Now - granted, that's a lot of money for a small robot arm (actually, it isn't - price out a TeachMover or Rhino arm), but take a look at it - it's not just a robot arm. Probably it's best additional use is as a 3D printer - so if you have given thought to buying one of those, too - well, here you can have both. You can also purchase one on Amazon, if you prefer that method. It was also reviewed in March in Servo Magazine: http://www.servomagazine.com/index.php/magazine/issue/2017/03 http://www.servomagazine.com/index.php/magazine/issue/2017/0... Now granted, this arm isn't a kit - but usually, if you want a kit, you'll sacrifice precision, and if you want precision, you'll get it (usually) pre-built. Note that the lower the mass on the end of the arm, the more precise it will be and move (and be able to lift more). If the servos or motors are mostly near the base of the arm, that's going to be the best placement; the popular "pallet stacking" style parallel arm kits you see sold out there typically have this arrangement, with only a servo on the end for the gripper (one of these actually isn't bad, if you pair it up with quality metal-gear dual-ball bearing servos). Another option would be to 3D print a robot arm; there are plenty of examples and files out there for that, if you already have a 3D printer or access to one. You might look around on Ebay or similar for an older "vintage" desktop robot arm from the 1980s (and old TeachMover or similar arm is ideal), and try to repair or refurbish it (there are a few sellers that do this as well, and their work can be stellar - but you'll pay the price for it, too). For instance, I got an old Rhino XR-1 arm from Ebay for a couple hundred dollars - it's controlled using a simple serial port (RS-232) protocol. Most of those early arms work the same way. Another option might be to try to replicate one of those early robot arms, or build your own arm from parts - pringles cans can make a great starting point! Finally - and only do this if you are really serious - you can find on Ebay used industrial robot arms for sale; look for one that is "lab grade", as they'll usually be smaller in size and cleaner (most of the arms on there - while a great buy considering what they cost new - are so big you'll need a truck and pallet jack to move them, plus 440v to run 'em - not to mention all the hydraulic mess). The downside will probably be on the interfacing end; you might have to do something custom there (ie - rip out the old controllers and hack your own in place).
- briga 9y agoImagine if you went back 50 years and told Terry Winograd or Marvin Minsky that in 50 years we'd still be trying to figure out how to get robots to stack blocks on top of each other. They'd think you were nuts.
- asperous 9y agoMore accurately it's "Robots learning unprogrammed tasks by observing behavior" they'd be pretty impressed. Programming robots to stack blocks directly is pretty easy.
- erikpukinskis 9y agoWhy is that impressive?
- ArchReaper 9y agoAre you asking why AI is impressive?
- erikpukinskis 9y agoI'm asking why it's so impressive that it learns by watching a person. Specifically, why we think Terry Winograd would be impressed that 50 years of AI research led to this. Didn't Deep Blue do the same thing as part of its chess training in the 1990s? Social/observational/modeling learning has been studied in psychological research for over 50 years: https://en.wikipedia.org/wiki/Social_learning_theory https://en.wikipedia.org/wiki/Social_learning_theory Isn't basically every recommender system, all of the vision systems, and the machine translation we all use... isn't all of that just observational learning? I'm not saying it's not interesting. It's a cool toy. But Elon Musk gets an idea for a neat machine learning toy, using an idea that predates AI as a field, and Terry Winograd is supposed to be impressed by this? The point is, AI has only been narrowly successful. Narrow success is rad. I'm not hating. But none of the promises of broad intelligence have really progressed. Siri is not a particularly meaningful advance beyond SHRDLU. Instead of just stacking blocks, she stacks text on Messages.app and pushes buttons in Weather.app.
- hackpert 9y agoThis transition from end-to-end differentiable 'black box' systems to multiple networks dedicated to certain tasks working in conjunction is interesting and very probably the idea that will keep this field going. We might not fully understand end to end systems in as much detail as we'd like to, but this abstraction layer enables us to at least know what part is doing what, empirically.
- Candles123 9y agoIs this 'learning' or just 'copying'?
- kyleschiller 9y agoIt seems like learning is just generalized copying. In this case, the arm still has to be shown the specific task, but is able to generalize across different block locations, textures, lighting etc.
- kreutz 9y agowhat is the difference?
- backpropaganda 9y agoAnd I was enlightened.
- tdb7893 9y agoI think saying 'just' copying really belies just how difficult and useful it is to be able to do
- hackjam600 9y ago+ University grades changing + Facebook hack + Email interception hack + Email accounts hack + Grade Changes hack + Website crashed hack + Word Press Blogs hack + Retrieval of lost file/documents + Erase criminal records hack + Databases hack + Sales of Dumps cards of all kinds + Untraceable Ip + Bank accounts hack + Individual computers hack + Websites hack + Control devices remotely hack + Burner Numbers hack + Verified Paypal Accounts hack + Any social media account hack + Android & iPhone Hack + server crashed hack + Text message interception hack + Twitters hack + Skype hack + Credit cards hacker + We can drop money into bank accounts. + credit score hack + blank credit card sale + Hack and use Credit Card to shop online + Monitor any phone and email address + Tap into anybody's call and monitor their conversation CONTACT:hackjam600@gmail.com
- blazespin 9y agoThis, IMHO, is the real revolution of the one-two punch of AI/VR. Training robots in VR and then using them to produce in a real time environment that maps well to the VR env. Humans interact in the VR env (with oculus/vive/etc) to train / configure the robots and assembly line. Once complete, the factory line is built IRL.
- lefnire 9y agoWith current VR content, these robots will be damn fine archers.
- a_d 9y agoIs the vision network learning continuously or it has been trained with many configurations of the blocks and gives a continuous output? The post says that the imitation network takes the input from the vision network and processes it to infer the intent of the task. Isn't the "intent" always "to stack"? Or can the imitation also be just re-arranging blocks in another configuration? This part is interesting, if I understood it well.. > "But how does the imitation network know how to generalize? The network learns this from the distribution of training examples. It is trained on dozens of different tasks with thousands of demonstrations for each task. Each training example is a pair of demonstrations that perform the same task. The network is given the entirety of the first demonstration and a single observation from the second demonstration. We then use supervised learning to predict what action the demonstrator took at that observation. In order to predict the action effectively, the robot must learn how to infer the relevant portion of the task from the first demonstration." Does this mean that the imitation network has been trained on stacking, un-stacking, throwing...and other such tasks, and then it identifies that "stacking" is what is being done in order to imitate it? Is there an ELI5 for what the 2 NNs are actually learning?
- npew 9y agoThe vision network is trained before-hand on lots of different configurations in simulation and then used to infer the block locations in the image from the camera. So it’s not learning continously. The imitation network takes the block locations predicted by the vision network, together with the demonstration trajectory in VR, and imitates the task shown in the demonstration. So, it learns to look through the demonstration to decide what action to take next given the current state (i.e. location of blocks and gripper). To keep the setup simple, we only trained the imitation network on stacking tasks (so no unstacking, throwing, etc). In future work, we want to make the setup and tasks much more general.
- a_d 9y agoThanks for the explanation. Can you also explain the significance of "one-shot imitation learning" generally (beyond the context of this experiment)?
- jorgemf 9y agoThe real value of this is the algorithm of "one-shot imitation learning" (the paper is here: https://arxiv.org/pdf/1703.07326.pdf https://arxiv.org/pdf/1703.07326.pdf ). The title about robots it is only to catch the attention of the media. The domain is simple to show the idea, but it can be applied to more complex domains once you know how to define it (which it is usually really complex). The blocksworld domain is used because it has been used in automatic planning for decades, and it is well know in research. It feels trivial to use only 6 blocks, but when you want to create an automatic plan of the steps to reach the final position it is not that simple for the computers.
- karpathy 9y ago+1 that the meta-learning spin on this approach is the really interesting part. The normal approach would be as follows: "you want to stack 6 blocks on one another? great, let me collect 1,000 examples of doing that in VR, and I'll train my policy on this and see how that works" instead, we change the question: "you want to stack 6 blocks on one another? great, that's one possible thing out of thousands you might want to do. so lets create a dataset of 1,000 examples of tuples: one 'query' demonstration, and a second demonstration as the target behavior to train the network on, when it sees the query. The training data is now 1,000 tuples of (query_demo, target_demo)), trained again with supervised learning." Once this is trained, we can sub in (in theory) any arbitrary desired demonstration, and the network will learn how to "extract" what is intended, and uses the demonstration as a crutch that is being imitated. It's a bit of a change of mindset, but a very powerful one, much more general one, and much more exciting one.
- speby 9y agokarpathy, I see you are at Stanford for Deep learning and NLP... I'm working on a project for audio/sound classification and have been sniffing around for some folks who may have encountered a similar set of feature points for audio data in deep learning. Would you be open to connecting? If so, let me know an email or other way to contact you and I'll reach out.
- 9y ago
- Houshalter 9y agoThe fact that NNs can generalize from simulated data to real data is very interesting. We can generate tons of simulated data of stuff like driving in simulators or even video games like GTA. And do experiments that would be way to dangerous or costly to perform in real life. It's not good when the first iteration of a reinforcement learner crashes the car at 60 miles per hour! You can then add tons of randomization to the simulation to make sure it doesn't overfit to the particulars of the simulated data. Like random filters on the input, moving the camera around and vibrating it, making cars and pedestrians behave unrealistically erratically, etc, or having sensors fail. If it can learn to handle these extreme situations in the simulators, hopefully it would be generalize even to rare scenarios that occur in real driving.
- pplonski86 9y agoDoes Tesla use this in their factory?
- tshadwell 9y ago'we've taught this robot to move small uniform blocks and we're going to make it perform arbitrary complex tasks on a variety of objects' sounds a lot like 'i'm trying to draw the mona lisa and I got her eyebrow down really good'
- nojvek 9y agoI have to say openai website is very well designed. Love those gradients.
- Houshalter 9y agoGradients are their speciality.
- known 9y ago"Show me the code" --Linus
- hagakure0c 9y agoWaiting for the first IKEA assembly robot that will follow you home, setup your new furniture and then return to base.
- deleted 9y ago[deleted]
- deleted 9y ago[deleted]