19 ms·

Open Problems in Robotics

436 points by haltingproblem 6y ago

tsimionescu 6y ago
> I’ll point out that the humble housefly has no problem understanding the concept of “shit in front of you; avoid,” Surely in the case of the house fly, it would be more like 'dinner is served' than 'avoid'! But in all seriousness, this is a good list to remind us how vastly far away from human-level AI we are.
- jayd16 6y ago
  Flys move reflexively don't they? They bump into walls and windows all the time.
heyitsguay 6y ago
I know that there's a lot of successful work in specific controlled environments (company X's factory floor), and that a lot of environment understanding/SLAM is broadly unsolved in arbitrary uncontrolled environments, but what about specific uncontrolled environments? What if i want a beer serving robot to learn a consistent, high-accuracy model of just my house, the way it is, and I'm willing to put in some technical work? Can i do any better than completely hand crafting a 3D object model or similar?
- cpgxiii 6y ago
  If you have high-quality 3D sensors (or in some cases, good-enough 2D cameras) and usable IMU/odometry, you can assemble quite reasonable 3D models for many small environments. The biggest challenges in a home environment are that many items we want in our houses (stainless steel appliances, chairs with thin legs, glass anything, reflective floor materials) are almost pathologically bad from a computer vision standpoint and are very hard to see and avoid in an uncontrolled household setting. Vision-based navigation against a known environment is doable with reasonable reliability, but adding fixed markers or positioning systems goes a long way towards robustness.
- redis_mlc 6y ago
  Willow Garage sold a robot that did that. Only $440,000 each. It was a vanity project of the Google founders/early employees, but they contributed to ROS (the Robot Operating System.) https://en.wikipedia.org/wiki/Willow_Garage https://en.wikipedia.org/wiki/Willow_Garage https://en.wikipedia.org/wiki/Robot_Operating_System https://en.wikipedia.org/wiki/Robot_Operating_System
helltone 6y ago
One problem I encountered while working on robotics is that many commonly used algorithms yield approximate solutions with no error bounds. They work 99.99‰ of the time. This is fine from a computer science or math point of view, but very scary from an engineering perspective, specifically when there are humans nearby. A big part of me struggles to accept the suitability of algorithms coming from gaming engines or machine learning etc for real world heavy duty robots. The lack of rigour in the field is astonishing.
- fluffy87 6y ago
  This is the scariest part of using machine learning as an engineer on any practical application as well. Without an error bound, ML can’t be in charge of anything that could put human lives at risk. This is also why I don’t understand all the hype about FSD / L5 autonomous driving. We don’t even know yet if such error bounds even exist, so we don’t even know if machine learning is even the right tool for FSD yet. All certification entities for control systems that put human lives at risk in aviation, automotive, etc. require those error bounds. So it actually doesn’t really matter if Tesla comes up with a “maybe L5” system, without right error bounds, their cars won’t be certified as L5 and drivers will need to keep hands on the steering wheel.
  - jaaron 6y ago
    What do you think the error bounds are for a human? I know it sounds like a flippant question, but for certain applications, if we can get a model that's better than human, then it doesn't need to be perfect. And they way we currently do this in all sorts of ways is to pair a human with a computer so that they each do what they're best at. It doesn't have to be about full automation.
    - deleted 6y ago
      [deleted]
    - de_watcher 6y ago
      Human is like your ancient software that was here since forever and somewhat worked fine. So everyone is used to it. (the difference from an actual software is that humans are based on some crazy nanotech from the future that nobody can completely control)
monkeydust 6y ago
Honestly, late yesterday evening after work I was looking at floor full of toys that my little kids were playing with in yet another lockdown day thinking I wish there was a robot that I could build or buy to tidy this up. Did some research found this research project, promising but from 2018 and looks like it didn't go anywhere. https://youtu.be/geub-Nuu-Vw https://youtu.be/geub-Nuu-Vw So now I'm thinking what about the build option.
- mvn9 6y ago
  Build it. There is a global market of parents who will buy it. But on the other hand, why not accept the toys on the floor? You are fighting entropy for no reason. You sleep at night, you will work tomorrow during the day, and when you look again, the toys are in an equally dispersed state. Why not let them stay in that state for days until you need to hover?
  - tuatoru 6y ago
    > Why not let them stay in that state for days until you need to ho[o]ver? Risk of personal injury. (The Lego-on-the-stairs scenario.) Also, some people just like a calm visual field at home.
    - mvn9 6y ago
      All very reasonable arguments. What I don't understand is cleaning up in the evening. There is no visual field to perceive if you are asleep.
      - tuatoru 6y ago
        It ain't the sleeping in bed, it's the crusty eyelids in the morning. :-)
      - fxtentacle 6y ago
        It's to make sure that when you walk around half-asleep early in the morning trying to get a diaper, that you don't step on a pointy lego brick.
cpgxiii 6y ago
In the area of Motion Planning (my own area of research), the most that can be said is that practical solutions exist for a tiny subset of cases, workable methods exist for a larger subset, expensive methods exist for a still larger subset, and everthing else might as well be impossible. - If you've got a low-dimensional problem, say 2D or 3D, without uncertainty (or at least bounded enough to pad obstacles and ignore it), search-based planners like A* and its derivatives work. Add uncertainty, complex non-holonomic constraints, limited horizons, etc and it becomes much harder. - If you've got a higher dimensional problem, say a 6/7-DoF arm, even multi-armed or humanoid robots, and you don't have uncertainty and dynamics (or can ignore them), sampling-based planners like RRTs and PRMs and their derivatives will often practically work. Actually useful guarantees of finding a solution or trying to find an optimal solution in useful time are still very much unsolved. - If your problem is basically open, and something approximating the "straight-line" from A to B is in the same local minima as a solution, trajectory optimization will work for a lot of problems. Motion planning is very much non-convex, though, so it's very easy to go from a problem solvable with optimization to a problem that isn't. - Planning with non-rigid objects, significant uncertainty (effectively continuous MDPs or POMDPs), and/or complex dynamics are all very unsolved problems. For motion planning problems in the gray area of "practically solvable", the art is figuring out how to simplify the problem as much as possible to make it tractable - highly optimized collision checking (generally speaking the performance bottelneck), combining search/sampling-based + optimization methods to get an initial solution from a global method and then refining it towards a local minima with optimization, or using special hardware or sensors to bound dynamics and uncertainty so it can be ignored.
- alwahi 6y ago
  sorry for sidetracking your answer but what actually does convex mean in the context of optimization. I remember looking at a book called convex optimization. Your statement that > Motion planning is very much non-convex, suggests to me that you are very much talking about the same thing. I understand convexity as in a shape. Why is convex good and concave bad in terms of optimization? I don't want you to dumb down the answer too much as I am a trained Mechanical Engineer but then my major isn't math. Hope you understand :)
modeless 6y ago
The objections to neural nets as the solution to all the problems on his list are the same as the old objections to neural nets as the solution to computer vision, speech recognition, translation, playing Go, etc. The objections will fall in the face of overwhelming evidence that neural nets simply work better than other approaches to these types of problems. For a long time software was the reason robots didn't work, but that time is ending. Hardware will be the bottleneck soon if it isn't already. No general purpose robot will ever be successful in the mass market using electric motors and gearboxes at each joint. We need simpler, cheaper, lighter, more robust, more reliable, backdrivable, force-sensing actuators.
- tuatoru 6y ago
  > The objections will fall in the face of overwhelming evidence that neural nets simply work better Perhaps? But progress is stalling[1]. We need theoretical foundations. [1] https://www.sciencemag.org/news/2020/05/eye-catching-advances-some-ai-fields-are-not-real https://www.sciencemag.org/news/2020/05/eye-catching-advance...
  - nmca 6y ago
    Wrg to your reference: One wonders why the "modest tweaks" weren't used in the original paper - indeed because they weren't known or weren't possible. The way you should interpet these meta-analysis is "researchers are motivated to attribute improvements in performance to 'the interesting bit' and they will run baselines without applying similar tweaking effort to them" The progress in the field can in some sense be indicated by the huge improvement in the 2006 method due to the "tweaks" discoverd since.
- iamgopal 6y ago
  Any research/potential example of such actuators ?
  - vsskanth 6y ago
    Probably electro active polymers
krisoft 6y ago
Ugh. This is making a hash of it. What is true: The understanding of laymen about what robots can do is very much out of touch with reality. Making real world robots is very hard. I think it's movies which made people think that any 6 year old can just plop together a C-3PO. What is the hash then? (I omit the points where I lack experience.) 1; Motion planning "Even things like a model of where the robot is, with respect to the surroundings" That's not motion planing. Motion planing starts from when you have a model of yourself and your surrounding. There are theoretical challenges (imagine a flytrap, or maze) but the real world challenges in my experience come from that under the inaccuracies and glitchyness of perception you are expected to make okay-ish decisions. 3; SLAM "there’s always going to be new obstacles (a pair of shoes, a book)" SLAM is about localisation. It gives you a 6dof pose from some fixed coordinate system. It does not deal with tripping hazards. That's obstacle detection. It does indeed deal with "mapping" but only in as much as to get you that pose estimate. "not turn-key to where there would be a SLAM module you can buy for your robot." Sure you can. The "Intel Realsense t265" is for example one such a module. The inside-out tracking of the Oculus Quest is an other (albeit one you will have hard time buying for your robot.) 6; Depth estimation. That one is odd. True, getting full fidelity depth maps out of monocular images is a research problem. But if in reality you want to estimate the distance to your beer bottle you will use stereo images, or a Kinect like depth sensor. It is obviously a hard engineering challenge to make it four 9 robust, but not an unheard of challenge. If his definition of an "open problem" is that there are people writing research papers about it then it will remain an open problem for a while. But if he just wants to depth estimate stuff, then there are already working good ways. Maybe cost prohibitive, maybe not robust enough for his liking. 9; Scene understanding Humanity made insane leaps and bounds on this one. Again there are engineering tradeoffs. How much accuracy you get for how much watts. What kind of training data you need, etc etc. Our systems are nowhere as good as a 5 year old human child, but I think we can handle that "beer bottle obscured by ketchup bottle" challenge if we try.
- cpgxiii 6y ago
  Depth estimation I wish it were as solved as vendors would like to believe it is. If you want a sensor for medium-range applications, say order 0.5m to 10m depth with better than 1cm accuracy, that works indoors and outdoors, with reflective and untextured objects, doesn't interfere with other sensors of its kind, it simply does not exist. Traditional active and passive stereo is OK provided you have texture, ToF so long as the surfaces aren't reflective and you don't have sunlight to worry about, structured light if you can control lighting and can otherwise control/avoid interference from other sensors. There's some promise in learned stereo matching, but collecting enough data and running fast enough inference are big challenges to practical use. Existing depth sensors for manipulation tasks, even indoors in reasonably controlled lighting, are still mostly insufficient given that the objects they struggle to see are often the objects you want to manipulate.
sfvisser 6y ago
Autonomous robots sound cool, but there are so many problems that could benefit from robotic automation that don’t require full autonomy. In controlled environments where we know the entire state of the (local) world including the position of our robotic swarm we can deterministically plan a motion and leave the world in an expected end state. Maybe use a bit CV to make sure our assumptions are right, if not hit the break else continue. Construction, mining, infrastructure, agriculture, manufacturing, logistics all contain problems spaces that could use non- or semi-autonomous robotic automation. Still a difficult problem, but how can we expect full autonomy without controlled robotic environments first? Yes we use robots in manufacturing and a few warehouses, but that’s it..?
- bluGill 6y ago
  As an employee of John deere I can tell you that we are currently shipping most of the automation you suggested. Humans are still in the tractor but they often are not driving them.
palad1n 6y ago
Is there a similar report for open problems in AI or ML?
- tuatoru 6y ago
  I don't know about any report, but an analogous problem (to "get me a beer") might be something like: - "Hey Siri, write up minutes of the meeting we just had and email them to everybody". There are quite a few open problems in that, including analogous navigation of an ill-defined environment which nevertheless has regularities.
fxtentacle 6y ago
Motion planning is also an Open Problem for Humans! Let's say we are standing on a high hill and I point to another hill and say: "Walk over there". Do you expect any human to find a reasonably good path by themselves? I would personally try to use a map. How do military robots solve this? They use satellite images. And in general, this article seems very pessimistic to me. My home-built computer vision pipeline can do localization and mapping, loop detection, object segmentation and depth estimation at levels that are "good enough" for indoor drone flight. So I would assume that someone with a generous serving of financial resources would be able to solve most of those problems, except for 2 issues: 1. You need to memorize how the environment works. That's why newborn kids take stupid decisions, they lack the stochastic priors. Lucky for us, memorizing is AIs strong point. 2. You need to have mechanics that are more forgiving. If I accidentally position my hand the wrong way, I might accidentally squeeze someone a bit, but I won't crush their bones, because the mechanics of my arm are flexible. We'll need better accentuators and elastic casings. And just for the sake of discussion, here's my replies for each problem category. Simultaneous Location and Mapping: There are libraries that work well enough for your robot to localize itself in a building-sized environment with just a single camera. I'd consider this solved. https://github.com/raulmur/ORB_SLAM2 https://github.com/raulmur/ORB_SLAM2 As for the obstacles, also for humans it is mostly guessing if you want to step on that blanket or if there'll be something fragile or slippery inside. Lost Robot Problem: Most SLAM solutions are good enough that you could just regenerate the map from scratch every time that there was a gap in your perception. ORB-SLAM2 also has a loop and merge detection module so that if you reset its tracking and then it walks into a known environment, it can merge the old data into its new state. Depth estimation: It works well enough in practice. https://www.stereolabs.com/zed-2/ https://www.stereolabs.com/zed-2/ Scene understanding: I don't know about you, but when I drive the highway, I sometimes have dead flies on my windshield. Apparently, they aren't that clever after all. Position estimation: It works exceptionally well for VR markers. In general, those solutions tend to be called "Visual Odometry" https://www.youtube.com/watch?v=fh5dLF3dmr0 https://www.youtube.com/watch?v=fh5dLF3dmr0 Affordance discovery: This is mostly a memorization problem, so a perfect candidate for AI.
- rimliu 6y ago
  Alas, your problem #1 is not really about memorization, it is about understanding. Take a human to the house they have never been before and tell them: "make me some tea". Now try that with any robot you want. It is you being optimistic, not the article being optimistic.
hoseja 6y ago
There are some really interesting, if maybe ranty and not-politically-correct comments on that blog.
contingencies 6y ago
Robotics founder here. Popular conceptions of "robots" are unrealistically general. In industry, we do not build robots, we build automation systems. Given the choice, would you prefer an automation system with some environmental assumptions, high speed and perfect repeatability (ie. entire industrial automation world), or no environmental assumptions, crushingly high cost, slow speed and poor reliability (eg. walking robot that accesses your fridge because cute or novel)? It seems both the lay population and academia assumes the latter, but industry demands the former. This is perhaps why there are so many "open problems" - because they're not actually real world problems: they're just academic nice-to-haves.
- arethuza 6y ago
  As an aside, this reminds me of the AI vs AGI debate (which I admit I indulge in being a former academic researcher in good old-fashioned AI) - almost all of the focus in modern "AI" appears to be be focused on solving specific tasks - which is completely understandable.
- mikro2nd 6y ago
  Depends what you're after, I guess. If all you want is a 'slave' that does some repeatable action with specified accuracy, then an automation system is just the thing. If you're after exploring the questions around what it means to be an autonomous, sentient being, then perhaps not so much and you really want to build a robot.
  - azernik 6y ago
    The money (and hence most of the engineering time) is in making people's lives and jobs easier, not in exploring deep philosophical questions.
    - deleted 6y ago
      [deleted]
- bad_alloc 6y ago
  Well, most elements on this list would allow us to move beyond needing so many assumptions. Lots of cost could be saved if I didn't need to perfectly measure all my owrkoffsets or didn't have to spend lots of time on designing my workspace to avoid singularities. Also it would allow us to move from relatively narrow-purpose machines ("move this end effector to this position within +/-0.01mm") to much more general jobs, hence opening more to automation.
paulpauper 6y ago
>Multiaxis singularities -this one blew my mind. Imagine you have a robot arm bolted to the ground. You want to teach the stupid thing to paint a car or something. There are actual singularities possible in the equations of motion; and it is more or less an underconstrained problem. I guess there are workarounds for this at this point, but they all have different tradeoffs. It’s as open a problem as motion planning on a macro scale. This is a math problem applied to robotics, not a unique problem to robotics. If there are workarounds, then isn't the problem solved?> Factories have robotic arms and they seem to do an adequate job making cars and other stuff in spite of singularities.
- person_of_color 6y ago
  I thought quaternions work around gimbal lock?
  - rcxdude 6y ago
    In the case of singularities in robotic systems the issue is actually physical (akin to actual gimbal lock in a gimbal with only 3 axes). In certain positions it's not possible to move the end-effector in certain directions, and near those positions you may need large movements of the whole system to make small movements in the end-effector (which can be especially violent if the robot attempts to pass near the singularity at a constant velocity). The solution, kind of akin to what quternions do, is to have more degrees of freedom than you need (like a 4th ring on a gimbal), but this doesn't eliminate singularities, it just makes them possible to avoid for most of the range of motion (extreme limits may still contain them). Actually coming up with a reasonable solution which avoids them is still not solved in general (though it's solved well enough for practical purposes in a lot of situations). These singularities exist in humans as well, though we're quite well adapted to avoiding them in normal motions. If you've ever found yourself making an awkward movement (especially needing to readjust your grip to continue a motion), then it's likely you've encountered this without realising it.
- jononor 6y ago
  The workarounds are designed to solve particular cases. A robot arm in manufacturing is stationary, has a controlled environment, and often a fixed task (or at least task type). The task is know in advance, and humans are involved in developing the solution for each particular task. In an open environment one cannot rely on those things. When the task is not known up-front. Or the new tasks come too quickly for a human to be involved in solving it. Or with influence from other actors, some possibly uncooperative or adversarial. A lot of 6 axis robots actually work in 3D+3D. That is, they position their arm/tooling first into a work pose, then perform their actually work in a 3D space referenced from that point/orientation. Then poses are chosen such that the space it works in is singularity free. And moving between poses there are then explicit solutions (chosen by human) for dealing with singularities.
Animats 6y ago
- Motion planning: already discussed. - Multiaxis singularities: much less of a problem than it used to be. We don't need closed-form solutions any more; we have enough CPU power at the robot to deal with this. You need some additional constraint, like "minimize jerk" when you have too many degrees of freedom. - Simultaneous Location and Mapping. SLAM for short: Getting much better. Things which explore and return a map are fairly common now. LIDAR helps. So does having a heading gyro with a low drift rate. Available commercially on vacuum cleaners. - Lost Robot Problem: hard, but in practical situations, markers of some kind, visual or RF, help. - Object manipulation and haptic feedback: Look up the DARPA manipulation challenge. Getting a key into a lock is still a hard problem. It's embarrassing how bad this is. Part of the problem is that force-sensing wrists are still far too expensive for no good reason. I once built one out of a 6DOF mouse, which is just a spring-loaded thing with optical sensors. Something I was fooling around with before TechShop went down. I have a little robot arm with an end wrench and a force sensing wrist. The idea was to get it to put the wrench around the bolt head by feel. Nice problem because a 6DOF force sensor gives you all the info you can get while holding an end wrench. - Depth estimation: LIDAR helps. The second Kinect was a boon to robotics. Low-cost 3D LIDAR units are still rare. Amusingly, depth from movement progressed because of the desire to make 3D movies from 2D movies. (Are there still 3D movies being made?) - Position estimation of moving objects: the military spends a lot of time on this, but doesn't publish much. Behind the staring sensor of a modern all-aspect air to air missile is - something that solves that problem. - Affordance discovery: Very little progress. The real problem: solve any of these problems, make very little money. If you're qualified to work on any of these problems, you can go to Google, Apple, etc. and make a lot of money solving easier problems.
- deepGem 6y ago
  "The real problem: solve any of these problems, make very little money" - Just curious why have you come to this conclusion ? Object manipulation has potential products in dishwashing and vegetable chopping - sufficiently large markets, potential billion $ outcomes for a startup which takes the early mover lead. Two robotic hands that can work in co-ordination just as human hands do. Extremely difficult to solve, but money is there.
visarga 6y ago
> We all know emergent systems are super important in all manner of phenomena, but we have no mathematics or models to deal with them. So we end up with useless horse shit like GPT-3. Says the author who confesses his lack of domain knowledge in the intro. So much entitlement. Why didn't he invent something better?
danbmil99 6y ago
TL; DR: Robotics is hard. Nature is impressive. (Source: 2 years working on a humanoid bipedal robot. Now, I am constantly amazed that people can balance all that weight on two spindly little legs. Running is a miracle)
NalNezumi 6y ago
I love the ever prevalent pessimism/cautious optimism in the field of robotics (industry and academics alike). It's a fresh breath of air from the ever over-hyping ML/AI field. On a related note, an open problem I see in practicality is also: How do you manage an robotics company effectively? iRobot seems to be succeeding in this well, and so does some industrial robotics arm companies but the latter is more about industrial automation than the more general "robotics" company out there. Some companies that go very broad general solution seems to be struggling, the application based robotics seems also fail more often than succeed. There have emerged a lot of management methods and theories around software development (Agile etc), but what's the efficient management method for robotics? Having been working at a few robotics company as a junior, It have always been either: 1. Someone with extensive research/engineering experience in a subfield of robotics in management, that can't manage the other subfields 2. Some one with too general knowledge and can't balance between each robotics-subfields needs including production & reliability + cost. and both seems to do pretty bad, while the second one slightly favourable.
carrolldunham 6y ago
A bit shocked to realise how I'd completely forgotten about robotics whereas it had seemed the grandest challenge. Is robotics itself slipping toward being marginal as physical manipulation is becoming less important over time? The sci-fi robots don't even walk these days, they're holograms etc. I'm not a software engineer so it's not just that
wazoox 6y ago
What I notice from this thread (and many similar ones) is that actually AI may replace sooner or later lawyers, some programmers, system administrators, engineers of various ilks; but the sweeper, the toilet cleaner, the janitor, the restaurant maid and the dishwasher, the delivery guy and the auto repair man, all of the relatively "low skilled" jobs are absolutely safe.
mrfusion 6y ago
I’m excited to see gpt-3 or similar technology applied to these problems. A robot can now know lots of common sense. Beer is kept in the fridge. The fridge is in the kitchen. Cans should not be shaken, etc, etc. so if something like gpt-3 can help with a high level plan, and regular computer vision can handle the low level stuff like obstacle avoidance, fridge and beer identification, etc We could have something really interesting. (Get in touch if you want to work on something like that. I think it would be a blast!) Edit. What makes me think gpt-3 has some good common sense. I saw someone ask it “can I do a bench press with a cat?” And it said “no, the cat will bite you.” It’s kind of like we’ve achieved what that common sense database project was going for.
altindag 6y ago
Classic line from this piece This was before the marketing departments of google and other frauds made objective thought about this impossible.
mchusma 6y ago
I work in robotics. The problem for pretty much everything is one of scale and commercialization. Every one of these issues has been addressed, but almost none have a good and bundled solution. It feels like computers in the 70s. All the pieces are there but there aren't mass produced PCs yet somewhat because there aren't the suppliers to make things easier. This is changing fast. For example, SLAM has many things that work, but they require fine tuning, and most contain undocumented features that require reading the source to find. Slamcore is a company working on this, I hope there are more. Teleportation was "possible but hard", but Freedom Robotics has a good solution now that mostly "just works" Robotic bases work well, but they will ship them to you without things plugged in so you have to find the issue and fix it yourself. AWS Deepracer is clearly a prototype for a solid wheeled base. The documentation and build quality is an order of magnitude higher than anything else in the space. My guess is they launch it as a useful base in the next year or so. Depth estimation is pretty good with Intel Realsense now, and the new OpenCV OAK is another attempt here. I think this is more solved and packaged than anything else. ROS2 was only properly released in June 2020, and it is a huge step up from ROS1 for commercial applications. It probably needs another 1-2 years for the community to finalize supporting it. I think for founders finding known robotic solutions and making them into robust commercial products is a great space to work in. The next few years are very interesting in this space, as even 1 year ago everything was much harder and it's rapidly getting easier.
- MereInterest 6y ago
  > Teleportation was "possible but hard", but Freedom Robotics has a good solution now that mostly "just works" For somebody not in robotics, what does "teleportation" mean in this context. I assume that it doesn't mean the Star Trek style beaming that comes up from a Google search.
  - rocketflumes 6y ago
    I think OP meant "teleoperation" - controlling a robot from afar
  - mchusma 6y ago
    *yes, teleoperation (autocorrect).
HALtheWise 6y ago
Everything listed in this article is framed as a software problem, but the beauty of robotics is that many problems can fall to either software, hardware, or electrical solutions. Without further ado, a very incomplete list of hardware and electrical innovations that would push robotics forward. - Cheaper and smaller low-backlash actuators. Motors and associated gearboxes are big, heavy, and expensive, which is a big part of why our robots have singularities in their designs, making the motion planning problem more difficult. - Actuators with good force-speed curves. Muscles have both great torque at zero speed and great speed at zero torque. (weight lifting and throwing a baseball). Only hydraulics come close to matching both numbers, and they're heavy, expensive, and tend to leak oil on the carpet. - Cheaper force/torque sensors. Most robots today don't even have torque sensing on all their actuators, let alone the sort of dense 6dof-sensing full-body surfaces that animal skin provides. - Across-the-board robustness improvements. Robots more mechanically complicated than a quadcopter tend to break a lot. This makes approaches like training ML models directly on hardware difficult. - Reliably low-latency wireless. There's been a lot of hype about cloud robotics, and there's a lot of potential in offboard sensing, but we need a cheap communication system that can deliver <30ms latency for those systems to work reliably. WiFi is great until you pass through a metal doorway and the signal degrades for 200ms. - Cheap and light lidars: true depth sensing with long range that works outside means a lot of hard computer vision problems get easier, and means your robot can be smaller since it needs fewer cameras The thing I love about working in robotics is that we don't need to solve all the software problems and all the hardware problems to make a system that works well in the real world. We get to pick and choose which is easier, and often the solutions in software space depend intimately on the kind of substrate they need to run on.
sudosysgen 6y ago
Well, at least two of these problems have been solved already and just need to be implemented. For example, my cheap mirrorless cameras solves two of the problems on the list. It's phase detection sensors can both do depth estimation and position estimation quite rapidly in order to make the tracking autofocus systems work (autofocusing is really calculating the distance between object and lens). You would really just have to do a bit of integration, it's really a solved problem already.
- HALtheWise 6y ago
  The parent comment was clearly written by someone with no experience in robotics. In fact, every single problem on this list is "solved" in the sense that there are examples of systems that perform it in some limited context. Generalizing those subsystems to handle the difficult edge cases and integrating those subsystems together is 99% of what makes robotics as a whole difficult, and it would be similarly deceptive to say that AI is "solved" because Microsoft Clippy exists.
  - sudosysgen 6y ago
    I do have experience in robotics. There exists robust solutions to depth estimation and position estimation that are deployed in the real world, in difficult applications where they have been generalized to work with any object you can draw a box around. There is a difference between a problem that is solved in theory and a problem that is solved enough so that you can buy polished products that implement a solution and work essentially without failure. I don't think you can compare optical phase detection in the context of position estimation and depth detection to clippy in the context of AI. Phase detection is quite literally a closed form optical solution to the problem of "how far is this object away from me" as long as the object is within a few thousand times the physical aperture of the lens. It's mature enough that you can use it to drive a motor in response to movements or "that bird", "that teapot", "the closest object in that clump of pixels", "the farthest objects in that clump of pixels", as well as calculate the velocity of the aforementioned object in three dimensions. For all intents and purposes, if you have a problem of the order "what is the distance of that object" as well as "how is that distance changing over time", then you can solve it, and indeed it has been solved to very high reliability, using phase detection. That's what it means for a problem to be solved. In other words, I wouldn't call an implementation where you can click anywhere on an image and receive a distance, all of the time, with almost any lens imaginable in any environment where there are enough photons hitting the sensor "performed in a limited context". The technology simply hasn't been used in mainstream robotics, mainly because it's patented and difficult to implement from scratch, but these are all implementation problems not fundamental problems.
lisper 6y ago
> Guys like Rodney Brooks seemed to accept this and built various robots that would learn how to walk using primitive hardware and feedback oriented ideas rather than programmed ideas. There was even a name for this; “Nouvelle AI.” No idea what happened to those ideas; I suppose they were too hard to make progress on, though the early results were impressive looking. The problem was that subsumption didn't scale, but the idea was incorporated into the so-called three-layer architecture: http://www.flownet.com/ron/papers/tla.pdf http://www.flownet.com/ron/papers/tla.pdf (I am the author of that paper. AMA.)
- pilingual 6y ago
  This paper seems to pre-date behavior trees. Any comment on how TLA relates? (I skimmed but will give a closer look when I have more time.) I was curious what Brooks now thinks about subsumption and found this from a few months ago: The approach to controlling robots, the subsumption architecture that it proposed led directly to the Roomba, a robot vacuum cleaner, which with over 30 million sold is the most produced robot ever.¹ Like many, I got a Roomba not long after the pandemic began. I was disappointed by its poor sensorimotor system. Within a few days its IR cover was scuffed up and it was covered in scratches from getting stuck under an office chair. Brooks doesn't say that 30mm Roombas incorporate subsumption, and no doubt after a decade or two of programmers I wonder about the nature of the codebase. The Roomba's behavior is entirely unpredictable, as sometimes it will bump into something full speed and sometimes it will slow down as it approaches. There are a number of other issues too long to mention including the charging contacts and recently it started roaming around with its charger still attached for no apparent reason. ¹ https://rodneybrooks.com/peer-review/ https://rodneybrooks.com/peer-review/
  - lisper 6y ago
    > This paper seems to pre-date behavior trees. Indeed. By a good 20 years :-) > Any comment on how TLA relates? I've been out of the field for a long time so BTs are new to me. All I know about them is from skimming the wikipedia article. But they look to me like a more formal implementation of the TLA sequencing layer.
tgflynn 6y ago
I think one of the problems is that we don't have the right people working on the right problems. Self-driving cars seem to have absorbed most of the world's high-level robotics efforts for the past decade or more. I was always skeptical of that application because my experience has shown that weak-AI works best when there's a human backup and/or the stakes of an error are not too high. That isn't the case with self-driving cars where a lethal mistake can occur in less than a second. I think we would be much better off today if much of the self-driving car effort had been focused on household utility robots and/or business applications. No it wouldn't be "saving lives", but it would be saving countless life-hours spent on mind-numbing tasks that are a major reason for why life is so unpleasant for so many people.
- mdorazio 6y ago
  What household tasks do you think are well-suited to robotics that aren't already addressed? The reason so much focus has gone into autonomous vehicles is because there are trillions of dollars to be made there by the companies that solve it, and many obvious use cases will result in large profits even with imperfect solutions (where "imperfect" means it works only in specific design domains like the southwestern US during the day with no rain).
  - tgflynn 6y ago
    All of them - cleaning, cooking, laundry, etc. Everything that a human needs to do to maintain reasonable living conditions and would hire someone else to do if they were rich. EDIT: I think that if you could come up with a really good solution for all of that most people would be willing to pay about as much as they do for a car.
syntaxing 6y ago
I was really into FIRST robotics when I in HS and would love to make something at home. I thought about a VEX kit but didn't really want an erector set like. Does anyone have a recommendation for something like Misty Robotics [1]. I can only think of the Mindstorm but wanted to do some visual AI with it. [1] https://www.mistyrobotics.com/ https://www.mistyrobotics.com/
r34 6y ago
The only solution to mentioned problems is IMO: we'll have to adapt our environment to the moving machines. At least in early stage, before they will do it autonomously. Language will adapt more by itself (to make machines pass Turing test we need change in language, just as we need change in machines - people will adapt to useful machines)
stillsut 6y ago
The pile of problems in Robotics reminds me of the challenges faced by computer vision before modern methods were developed. To generalize the issue: humans learn and perform vision and navigational tasks "below" the level of language. To use a vivid example, you can use language to teach a child how to hold a pencil and write, how to recognize digits, and how to do two digit addition. But there's some cognitive ability "below" the level language a child needs to be capable of in order to take the hints from language and develop the desired skills. In computer vision, we now have something like this in ConvLayers. In today's robotics we see the hyper-systematization and mathematization of natural concepts like getting an agent to recognize its own bearing. This is what I refer to as "at the level of language", and I think as long as we try to solve these individual navigation problems from mathematical first principles we'll never arrive anywhere near the performance of animal "instinct". We need a new framework that doesn't look like an enumeration of commands within an imperative program.
- rajansaini 6y ago
  That's a very interesting perspective. There have been many articles complaining about the "hype" behind ML, but I too wonder if DNNs could assist with controls, especially when it comes to reacting to sensor data. After all, it's just matrix math.
nojvek 6y ago
What really surprises me is Apple is now 2 trillion dollars with another gajillion sitting in cash. Why are they being an optimization company and not really pushing the edge on robotics and automation? I was excited about Apple car but it seems that project is ded. Why not invest that money in moonshot ideas ? It seems Elon musk is the crazy one with Tesla, Solar City and SpaceX. It blows my mind how their massive rockets reach orbit and land back. Why haven’t we been making breakthroughs like this in robotics?
- cpgxiii 6y ago
  As difficult as it is, control for launching and landing a rocket is vastly easier computationally than even apparently simple robotics problems. Robotics is simply hard, from algorithmic, computational, and engineering perspectives. Only some parts of it can be solved by throwing money and raw compute power at it. Even in areas where progress has been made, you generally need experts to decompose the problem at hand into something that can be tackled with existing techniques,and those experts are expensive and in relative demand.
blauditore 6y ago
> 6. Depth estimation At least indoors, this was already handled pretty well 10 years ago by Kinect, and there are many somewhat robust approaches based on binocular vision. Not sure if this qualifies as "very much an open problem". > 9. Scene understanding The mentioned example is not the best one - anticipation of collisions has been a feature in cars for many years now (usually some warning system, sometimes with automatic emergency breaking). But it's a very constrained problem; more generic scene understanding is of course still very difficult.

> I’ll point out that the humble housefly has no problem understanding the concept of “shit in front of you; avoid,” Surely in the case of the house fly, it would be more like 'dinner is served' than 'avoid'! But in all seriousness, this is a good list to remind us how vastly far away from human-level AI we are.

jayd16 6y ago
Flys move reflexively don't they? They bump into walls and windows all the time.

heyitsguay 6y ago

I know that there's a lot of successful work in specific controlled environments (company X's factory floor), and that a lot of environment understanding/SLAM is broadly unsolved in arbitrary uncontrolled environments, but what about specific uncontrolled environments? What if i want a beer serving robot to learn a consistent, high-accuracy model of just my house, the way it is, and I'm willing to put in some technical work? Can i do any better than completely hand crafting a 3D object model or similar?

cpgxiii 6y ago
If you have high-quality 3D sensors (or in some cases, good-enough 2D cameras) and usable IMU/odometry, you can assemble quite reasonable 3D models for many small environments. The biggest challenges in a home environment are that many items we want in our houses (stainless steel appliances, chairs with thin legs, glass anything, reflective floor materials) are almost pathologically bad from a computer vision standpoint and are very hard to see and avoid in an uncontrolled household setting. Vision-based navigation against a known environment is doable with reasonable reliability, but adding fixed markers or positioning systems goes a long way towards robustness.
redis_mlc 6y ago
Willow Garage sold a robot that did that. Only $440,000 each. It was a vanity project of the Google founders/early employees, but they contributed to ROS (the Robot Operating System.) https://en.wikipedia.org/wiki/Willow_Garage https://en.wikipedia.org/wiki/Willow_Garage https://en.wikipedia.org/wiki/Robot_Operating_System https://en.wikipedia.org/wiki/Robot_Operating_System

helltone 6y ago

One problem I encountered while working on robotics is that many commonly used algorithms yield approximate solutions with no error bounds. They work 99.99‰ of the time. This is fine from a computer science or math point of view, but very scary from an engineering perspective, specifically when there are humans nearby. A big part of me struggles to accept the suitability of algorithms coming from gaming engines or machine learning etc for real world heavy duty robots. The lack of rigour in the field is astonishing.

fluffy87 6y ago
This is the scariest part of using machine learning as an engineer on any practical application as well. Without an error bound, ML can’t be in charge of anything that could put human lives at risk. This is also why I don’t understand all the hype about FSD / L5 autonomous driving. We don’t even know yet if such error bounds even exist, so we don’t even know if machine learning is even the right tool for FSD yet. All certification entities for control systems that put human lives at risk in aviation, automotive, etc. require those error bounds. So it actually doesn’t really matter if Tesla comes up with a “maybe L5” system, without right error bounds, their cars won’t be certified as L5 and drivers will need to keep hands on the steering wheel.
- jaaron 6y ago
  What do you think the error bounds are for a human? I know it sounds like a flippant question, but for certain applications, if we can get a model that's better than human, then it doesn't need to be perfect. And they way we currently do this in all sorts of ways is to pair a human with a computer so that they each do what they're best at. It doesn't have to be about full automation.
  - deleted 6y ago
    [deleted]
  - de_watcher 6y ago
    Human is like your ancient software that was here since forever and somewhat worked fine. So everyone is used to it. (the difference from an actual software is that humans are based on some crazy nanotech from the future that nobody can completely control)

monkeydust 6y ago

Honestly, late yesterday evening after work I was looking at floor full of toys that my little kids were playing with in yet another lockdown day thinking I wish there was a robot that I could build or buy to tidy this up. Did some research found this research project, promising but from 2018 and looks like it didn't go anywhere. https://youtu.be/geub-Nuu-Vw https://youtu.be/geub-Nuu-Vw So now I'm thinking what about the build option.

mvn9 6y ago
Build it. There is a global market of parents who will buy it. But on the other hand, why not accept the toys on the floor? You are fighting entropy for no reason. You sleep at night, you will work tomorrow during the day, and when you look again, the toys are in an equally dispersed state. Why not let them stay in that state for days until you need to hover?
- tuatoru 6y ago
  > Why not let them stay in that state for days until you need to ho[o]ver? Risk of personal injury. (The Lego-on-the-stairs scenario.) Also, some people just like a calm visual field at home.
  - mvn9 6y ago
    All very reasonable arguments. What I don't understand is cleaning up in the evening. There is no visual field to perceive if you are asleep.
    - tuatoru 6y ago
      It ain't the sleeping in bed, it's the crusty eyelids in the morning. :-)
    - fxtentacle 6y ago
      It's to make sure that when you walk around half-asleep early in the morning trying to get a diaper, that you don't step on a pointy lego brick.

cpgxiii 6y ago

In the area of Motion Planning (my own area of research), the most that can be said is that practical solutions exist for a tiny subset of cases, workable methods exist for a larger subset, expensive methods exist for a still larger subset, and everthing else might as well be impossible. - If you've got a low-dimensional problem, say 2D or 3D, without uncertainty (or at least bounded enough to pad obstacles and ignore it), search-based planners like A* and its derivatives work. Add uncertainty, complex non-holonomic constraints, limited horizons, etc and it becomes much harder. - If you've got a higher dimensional problem, say a 6/7-DoF arm, even multi-armed or humanoid robots, and you don't have uncertainty and dynamics (or can ignore them), sampling-based planners like RRTs and PRMs and their derivatives will often practically work. Actually useful guarantees of finding a solution or trying to find an optimal solution in useful time are still very much unsolved. - If your problem is basically open, and something approximating the "straight-line" from A to B is in the same local minima as a solution, trajectory optimization will work for a lot of problems. Motion planning is very much non-convex, though, so it's very easy to go from a problem solvable with optimization to a problem that isn't. - Planning with non-rigid objects, significant uncertainty (effectively continuous MDPs or POMDPs), and/or complex dynamics are all very unsolved problems. For motion planning problems in the gray area of "practically solvable", the art is figuring out how to simplify the problem as much as possible to make it tractable - highly optimized collision checking (generally speaking the performance bottelneck), combining search/sampling-based + optimization methods to get an initial solution from a global method and then refining it towards a local minima with optimization, or using special hardware or sensors to bound dynamics and uncertainty so it can be ignored.

alwahi 6y ago
sorry for sidetracking your answer but what actually does convex mean in the context of optimization. I remember looking at a book called convex optimization. Your statement that > Motion planning is very much non-convex, suggests to me that you are very much talking about the same thing. I understand convexity as in a shape. Why is convex good and concave bad in terms of optimization? I don't want you to dumb down the answer too much as I am a trained Mechanical Engineer but then my major isn't math. Hope you understand :)

modeless 6y ago

The objections to neural nets as the solution to all the problems on his list are the same as the old objections to neural nets as the solution to computer vision, speech recognition, translation, playing Go, etc. The objections will fall in the face of overwhelming evidence that neural nets simply work better than other approaches to these types of problems. For a long time software was the reason robots didn't work, but that time is ending. Hardware will be the bottleneck soon if it isn't already. No general purpose robot will ever be successful in the mass market using electric motors and gearboxes at each joint. We need simpler, cheaper, lighter, more robust, more reliable, backdrivable, force-sensing actuators.

tuatoru 6y ago
> The objections will fall in the face of overwhelming evidence that neural nets simply work better Perhaps? But progress is stalling[1]. We need theoretical foundations. [1] https://www.sciencemag.org/news/2020/05/eye-catching-advances-some-ai-fields-are-not-real https://www.sciencemag.org/news/2020/05/eye-catching-advance...
- nmca 6y ago
  Wrg to your reference: One wonders why the "modest tweaks" weren't used in the original paper - indeed because they weren't known or weren't possible. The way you should interpet these meta-analysis is "researchers are motivated to attribute improvements in performance to 'the interesting bit' and they will run baselines without applying similar tweaking effort to them" The progress in the field can in some sense be indicated by the huge improvement in the 2006 method due to the "tweaks" discoverd since.
iamgopal 6y ago
Any research/potential example of such actuators ?
- vsskanth 6y ago
  Probably electro active polymers

krisoft 6y ago

Ugh. This is making a hash of it. What is true: The understanding of laymen about what robots can do is very much out of touch with reality. Making real world robots is very hard. I think it's movies which made people think that any 6 year old can just plop together a C-3PO. What is the hash then? (I omit the points where I lack experience.) 1; Motion planning "Even things like a model of where the robot is, with respect to the surroundings" That's not motion planing. Motion planing starts from when you have a model of yourself and your surrounding. There are theoretical challenges (imagine a flytrap, or maze) but the real world challenges in my experience come from that under the inaccuracies and glitchyness of perception you are expected to make okay-ish decisions. 3; SLAM "there’s always going to be new obstacles (a pair of shoes, a book)" SLAM is about localisation. It gives you a 6dof pose from some fixed coordinate system. It does not deal with tripping hazards. That's obstacle detection. It does indeed deal with "mapping" but only in as much as to get you that pose estimate. "not turn-key to where there would be a SLAM module you can buy for your robot." Sure you can. The "Intel Realsense t265" is for example one such a module. The inside-out tracking of the Oculus Quest is an other (albeit one you will have hard time buying for your robot.) 6; Depth estimation. That one is odd. True, getting full fidelity depth maps out of monocular images is a research problem. But if in reality you want to estimate the distance to your beer bottle you will use stereo images, or a Kinect like depth sensor. It is obviously a hard engineering challenge to make it four 9 robust, but not an unheard of challenge. If his definition of an "open problem" is that there are people writing research papers about it then it will remain an open problem for a while. But if he just wants to depth estimate stuff, then there are already working good ways. Maybe cost prohibitive, maybe not robust enough for his liking. 9; Scene understanding Humanity made insane leaps and bounds on this one. Again there are engineering tradeoffs. How much accuracy you get for how much watts. What kind of training data you need, etc etc. Our systems are nowhere as good as a 5 year old human child, but I think we can handle that "beer bottle obscured by ketchup bottle" challenge if we try.

cpgxiii 6y ago
Depth estimation I wish it were as solved as vendors would like to believe it is. If you want a sensor for medium-range applications, say order 0.5m to 10m depth with better than 1cm accuracy, that works indoors and outdoors, with reflective and untextured objects, doesn't interfere with other sensors of its kind, it simply does not exist. Traditional active and passive stereo is OK provided you have texture, ToF so long as the surfaces aren't reflective and you don't have sunlight to worry about, structured light if you can control lighting and can otherwise control/avoid interference from other sensors. There's some promise in learned stereo matching, but collecting enough data and running fast enough inference are big challenges to practical use. Existing depth sensors for manipulation tasks, even indoors in reasonably controlled lighting, are still mostly insufficient given that the objects they struggle to see are often the objects you want to manipulate.

sfvisser 6y ago

Autonomous robots sound cool, but there are so many problems that could benefit from robotic automation that don’t require full autonomy. In controlled environments where we know the entire state of the (local) world including the position of our robotic swarm we can deterministically plan a motion and leave the world in an expected end state. Maybe use a bit CV to make sure our assumptions are right, if not hit the break else continue. Construction, mining, infrastructure, agriculture, manufacturing, logistics all contain problems spaces that could use non- or semi-autonomous robotic automation. Still a difficult problem, but how can we expect full autonomy without controlled robotic environments first? Yes we use robots in manufacturing and a few warehouses, but that’s it..?

bluGill 6y ago
As an employee of John deere I can tell you that we are currently shipping most of the automation you suggested. Humans are still in the tractor but they often are not driving them.

palad1n 6y ago

Is there a similar report for open problems in AI or ML?

tuatoru 6y ago
I don't know about any report, but an analogous problem (to "get me a beer") might be something like: - "Hey Siri, write up minutes of the meeting we just had and email them to everybody". There are quite a few open problems in that, including analogous navigation of an ill-defined environment which nevertheless has regularities.

fxtentacle 6y ago

Motion planning is also an Open Problem for Humans! Let's say we are standing on a high hill and I point to another hill and say: "Walk over there". Do you expect any human to find a reasonably good path by themselves? I would personally try to use a map. How do military robots solve this? They use satellite images. And in general, this article seems very pessimistic to me. My home-built computer vision pipeline can do localization and mapping, loop detection, object segmentation and depth estimation at levels that are "good enough" for indoor drone flight. So I would assume that someone with a generous serving of financial resources would be able to solve most of those problems, except for 2 issues: 1. You need to memorize how the environment works. That's why newborn kids take stupid decisions, they lack the stochastic priors. Lucky for us, memorizing is AIs strong point. 2. You need to have mechanics that are more forgiving. If I accidentally position my hand the wrong way, I might accidentally squeeze someone a bit, but I won't crush their bones, because the mechanics of my arm are flexible. We'll need better accentuators and elastic casings. And just for the sake of discussion, here's my replies for each problem category. Simultaneous Location and Mapping: There are libraries that work well enough for your robot to localize itself in a building-sized environment with just a single camera. I'd consider this solved. https://github.com/raulmur/ORB_SLAM2 https://github.com/raulmur/ORB_SLAM2 As for the obstacles, also for humans it is mostly guessing if you want to step on that blanket or if there'll be something fragile or slippery inside. Lost Robot Problem: Most SLAM solutions are good enough that you could just regenerate the map from scratch every time that there was a gap in your perception. ORB-SLAM2 also has a loop and merge detection module so that if you reset its tracking and then it walks into a known environment, it can merge the old data into its new state. Depth estimation: It works well enough in practice. https://www.stereolabs.com/zed-2/ https://www.stereolabs.com/zed-2/ Scene understanding: I don't know about you, but when I drive the highway, I sometimes have dead flies on my windshield. Apparently, they aren't that clever after all. Position estimation: It works exceptionally well for VR markers. In general, those solutions tend to be called "Visual Odometry" https://www.youtube.com/watch?v=fh5dLF3dmr0 https://www.youtube.com/watch?v=fh5dLF3dmr0 Affordance discovery: This is mostly a memorization problem, so a perfect candidate for AI.

rimliu 6y ago
Alas, your problem #1 is not really about memorization, it is about understanding. Take a human to the house they have never been before and tell them: "make me some tea". Now try that with any robot you want. It is you being optimistic, not the article being optimistic.

hoseja 6y ago

There are some really interesting, if maybe ranty and not-politically-correct comments on that blog.

contingencies 6y ago

Robotics founder here. Popular conceptions of "robots" are unrealistically general. In industry, we do not build robots, we build automation systems. Given the choice, would you prefer an automation system with some environmental assumptions, high speed and perfect repeatability (ie. entire industrial automation world), or no environmental assumptions, crushingly high cost, slow speed and poor reliability (eg. walking robot that accesses your fridge because cute or novel)? It seems both the lay population and academia assumes the latter, but industry demands the former. This is perhaps why there are so many "open problems" - because they're not actually real world problems: they're just academic nice-to-haves.

arethuza 6y ago
As an aside, this reminds me of the AI vs AGI debate (which I admit I indulge in being a former academic researcher in good old-fashioned AI) - almost all of the focus in modern "AI" appears to be be focused on solving specific tasks - which is completely understandable.
mikro2nd 6y ago
Depends what you're after, I guess. If all you want is a 'slave' that does some repeatable action with specified accuracy, then an automation system is just the thing. If you're after exploring the questions around what it means to be an autonomous, sentient being, then perhaps not so much and you really want to build a robot.
- azernik 6y ago
  The money (and hence most of the engineering time) is in making people's lives and jobs easier, not in exploring deep philosophical questions.
  - deleted 6y ago
    [deleted]
bad_alloc 6y ago
Well, most elements on this list would allow us to move beyond needing so many assumptions. Lots of cost could be saved if I didn't need to perfectly measure all my owrkoffsets or didn't have to spend lots of time on designing my workspace to avoid singularities. Also it would allow us to move from relatively narrow-purpose machines ("move this end effector to this position within +/-0.01mm") to much more general jobs, hence opening more to automation.

paulpauper 6y ago

>Multiaxis singularities -this one blew my mind. Imagine you have a robot arm bolted to the ground. You want to teach the stupid thing to paint a car or something. There are actual singularities possible in the equations of motion; and it is more or less an underconstrained problem. I guess there are workarounds for this at this point, but they all have different tradeoffs. It’s as open a problem as motion planning on a macro scale. This is a math problem applied to robotics, not a unique problem to robotics. If there are workarounds, then isn't the problem solved?> Factories have robotic arms and they seem to do an adequate job making cars and other stuff in spite of singularities.

person_of_color 6y ago
I thought quaternions work around gimbal lock?
- rcxdude 6y ago
  In the case of singularities in robotic systems the issue is actually physical (akin to actual gimbal lock in a gimbal with only 3 axes). In certain positions it's not possible to move the end-effector in certain directions, and near those positions you may need large movements of the whole system to make small movements in the end-effector (which can be especially violent if the robot attempts to pass near the singularity at a constant velocity). The solution, kind of akin to what quternions do, is to have more degrees of freedom than you need (like a 4th ring on a gimbal), but this doesn't eliminate singularities, it just makes them possible to avoid for most of the range of motion (extreme limits may still contain them). Actually coming up with a reasonable solution which avoids them is still not solved in general (though it's solved well enough for practical purposes in a lot of situations). These singularities exist in humans as well, though we're quite well adapted to avoiding them in normal motions. If you've ever found yourself making an awkward movement (especially needing to readjust your grip to continue a motion), then it's likely you've encountered this without realising it.
jononor 6y ago
The workarounds are designed to solve particular cases. A robot arm in manufacturing is stationary, has a controlled environment, and often a fixed task (or at least task type). The task is know in advance, and humans are involved in developing the solution for each particular task. In an open environment one cannot rely on those things. When the task is not known up-front. Or the new tasks come too quickly for a human to be involved in solving it. Or with influence from other actors, some possibly uncooperative or adversarial. A lot of 6 axis robots actually work in 3D+3D. That is, they position their arm/tooling first into a work pose, then perform their actually work in a 3D space referenced from that point/orientation. Then poses are chosen such that the space it works in is singularity free. And moving between poses there are then explicit solutions (chosen by human) for dealing with singularities.

Animats 6y ago

- Motion planning: already discussed. - Multiaxis singularities: much less of a problem than it used to be. We don't need closed-form solutions any more; we have enough CPU power at the robot to deal with this. You need some additional constraint, like "minimize jerk" when you have too many degrees of freedom. - Simultaneous Location and Mapping. SLAM for short: Getting much better. Things which explore and return a map are fairly common now. LIDAR helps. So does having a heading gyro with a low drift rate. Available commercially on vacuum cleaners. - Lost Robot Problem: hard, but in practical situations, markers of some kind, visual or RF, help. - Object manipulation and haptic feedback: Look up the DARPA manipulation challenge. Getting a key into a lock is still a hard problem. It's embarrassing how bad this is. Part of the problem is that force-sensing wrists are still far too expensive for no good reason. I once built one out of a 6DOF mouse, which is just a spring-loaded thing with optical sensors. Something I was fooling around with before TechShop went down. I have a little robot arm with an end wrench and a force sensing wrist. The idea was to get it to put the wrench around the bolt head by feel. Nice problem because a 6DOF force sensor gives you all the info you can get while holding an end wrench. - Depth estimation: LIDAR helps. The second Kinect was a boon to robotics. Low-cost 3D LIDAR units are still rare. Amusingly, depth from movement progressed because of the desire to make 3D movies from 2D movies. (Are there still 3D movies being made?) - Position estimation of moving objects: the military spends a lot of time on this, but doesn't publish much. Behind the staring sensor of a modern all-aspect air to air missile is - something that solves that problem. - Affordance discovery: Very little progress. The real problem: solve any of these problems, make very little money. If you're qualified to work on any of these problems, you can go to Google, Apple, etc. and make a lot of money solving easier problems.

deepGem 6y ago
"The real problem: solve any of these problems, make very little money" - Just curious why have you come to this conclusion ? Object manipulation has potential products in dishwashing and vegetable chopping - sufficiently large markets, potential billion $ outcomes for a startup which takes the early mover lead. Two robotic hands that can work in co-ordination just as human hands do. Extremely difficult to solve, but money is there.

visarga 6y ago

> We all know emergent systems are super important in all manner of phenomena, but we have no mathematics or models to deal with them. So we end up with useless horse shit like GPT-3. Says the author who confesses his lack of domain knowledge in the intro. So much entitlement. Why didn't he invent something better?

danbmil99 6y ago

TL; DR: Robotics is hard. Nature is impressive. (Source: 2 years working on a humanoid bipedal robot. Now, I am constantly amazed that people can balance all that weight on two spindly little legs. Running is a miracle)

NalNezumi 6y ago

I love the ever prevalent pessimism/cautious optimism in the field of robotics (industry and academics alike). It's a fresh breath of air from the ever over-hyping ML/AI field. On a related note, an open problem I see in practicality is also: How do you manage an robotics company effectively? iRobot seems to be succeeding in this well, and so does some industrial robotics arm companies but the latter is more about industrial automation than the more general "robotics" company out there. Some companies that go very broad general solution seems to be struggling, the application based robotics seems also fail more often than succeed. There have emerged a lot of management methods and theories around software development (Agile etc), but what's the efficient management method for robotics? Having been working at a few robotics company as a junior, It have always been either: 1. Someone with extensive research/engineering experience in a subfield of robotics in management, that can't manage the other subfields 2. Some one with too general knowledge and can't balance between each robotics-subfields needs including production & reliability + cost. and both seems to do pretty bad, while the second one slightly favourable.

carrolldunham 6y ago

A bit shocked to realise how I'd completely forgotten about robotics whereas it had seemed the grandest challenge. Is robotics itself slipping toward being marginal as physical manipulation is becoming less important over time? The sci-fi robots don't even walk these days, they're holograms etc. I'm not a software engineer so it's not just that

wazoox 6y ago

What I notice from this thread (and many similar ones) is that actually AI may replace sooner or later lawyers, some programmers, system administrators, engineers of various ilks; but the sweeper, the toilet cleaner, the janitor, the restaurant maid and the dishwasher, the delivery guy and the auto repair man, all of the relatively "low skilled" jobs are absolutely safe.

mrfusion 6y ago

I’m excited to see gpt-3 or similar technology applied to these problems. A robot can now know lots of common sense. Beer is kept in the fridge. The fridge is in the kitchen. Cans should not be shaken, etc, etc. so if something like gpt-3 can help with a high level plan, and regular computer vision can handle the low level stuff like obstacle avoidance, fridge and beer identification, etc We could have something really interesting. (Get in touch if you want to work on something like that. I think it would be a blast!) Edit. What makes me think gpt-3 has some good common sense. I saw someone ask it “can I do a bench press with a cat?” And it said “no, the cat will bite you.” It’s kind of like we’ve achieved what that common sense database project was going for.

altindag 6y ago

Classic line from this piece This was before the marketing departments of google and other frauds made objective thought about this impossible.

mchusma 6y ago

I work in robotics. The problem for pretty much everything is one of scale and commercialization. Every one of these issues has been addressed, but almost none have a good and bundled solution. It feels like computers in the 70s. All the pieces are there but there aren't mass produced PCs yet somewhat because there aren't the suppliers to make things easier. This is changing fast. For example, SLAM has many things that work, but they require fine tuning, and most contain undocumented features that require reading the source to find. Slamcore is a company working on this, I hope there are more. Teleportation was "possible but hard", but Freedom Robotics has a good solution now that mostly "just works" Robotic bases work well, but they will ship them to you without things plugged in so you have to find the issue and fix it yourself. AWS Deepracer is clearly a prototype for a solid wheeled base. The documentation and build quality is an order of magnitude higher than anything else in the space. My guess is they launch it as a useful base in the next year or so. Depth estimation is pretty good with Intel Realsense now, and the new OpenCV OAK is another attempt here. I think this is more solved and packaged than anything else. ROS2 was only properly released in June 2020, and it is a huge step up from ROS1 for commercial applications. It probably needs another 1-2 years for the community to finalize supporting it. I think for founders finding known robotic solutions and making them into robust commercial products is a great space to work in. The next few years are very interesting in this space, as even 1 year ago everything was much harder and it's rapidly getting easier.

MereInterest 6y ago
> Teleportation was "possible but hard", but Freedom Robotics has a good solution now that mostly "just works" For somebody not in robotics, what does "teleportation" mean in this context. I assume that it doesn't mean the Star Trek style beaming that comes up from a Google search.
- rocketflumes 6y ago
  I think OP meant "teleoperation" - controlling a robot from afar
- mchusma 6y ago
  *yes, teleoperation (autocorrect).

HALtheWise 6y ago

Everything listed in this article is framed as a software problem, but the beauty of robotics is that many problems can fall to either software, hardware, or electrical solutions. Without further ado, a very incomplete list of hardware and electrical innovations that would push robotics forward. - Cheaper and smaller low-backlash actuators. Motors and associated gearboxes are big, heavy, and expensive, which is a big part of why our robots have singularities in their designs, making the motion planning problem more difficult. - Actuators with good force-speed curves. Muscles have both great torque at zero speed and great speed at zero torque. (weight lifting and throwing a baseball). Only hydraulics come close to matching both numbers, and they're heavy, expensive, and tend to leak oil on the carpet. - Cheaper force/torque sensors. Most robots today don't even have torque sensing on all their actuators, let alone the sort of dense 6dof-sensing full-body surfaces that animal skin provides. - Across-the-board robustness improvements. Robots more mechanically complicated than a quadcopter tend to break a lot. This makes approaches like training ML models directly on hardware difficult. - Reliably low-latency wireless. There's been a lot of hype about cloud robotics, and there's a lot of potential in offboard sensing, but we need a cheap communication system that can deliver <30ms latency for those systems to work reliably. WiFi is great until you pass through a metal doorway and the signal degrades for 200ms. - Cheap and light lidars: true depth sensing with long range that works outside means a lot of hard computer vision problems get easier, and means your robot can be smaller since it needs fewer cameras The thing I love about working in robotics is that we don't need to solve all the software problems and all the hardware problems to make a system that works well in the real world. We get to pick and choose which is easier, and often the solutions in software space depend intimately on the kind of substrate they need to run on.

sudosysgen 6y ago

Well, at least two of these problems have been solved already and just need to be implemented. For example, my cheap mirrorless cameras solves two of the problems on the list. It's phase detection sensors can both do depth estimation and position estimation quite rapidly in order to make the tracking autofocus systems work (autofocusing is really calculating the distance between object and lens). You would really just have to do a bit of integration, it's really a solved problem already.

HALtheWise 6y ago
The parent comment was clearly written by someone with no experience in robotics. In fact, every single problem on this list is "solved" in the sense that there are examples of systems that perform it in some limited context. Generalizing those subsystems to handle the difficult edge cases and integrating those subsystems together is 99% of what makes robotics as a whole difficult, and it would be similarly deceptive to say that AI is "solved" because Microsoft Clippy exists.
- sudosysgen 6y ago
  I do have experience in robotics. There exists robust solutions to depth estimation and position estimation that are deployed in the real world, in difficult applications where they have been generalized to work with any object you can draw a box around. There is a difference between a problem that is solved in theory and a problem that is solved enough so that you can buy polished products that implement a solution and work essentially without failure. I don't think you can compare optical phase detection in the context of position estimation and depth detection to clippy in the context of AI. Phase detection is quite literally a closed form optical solution to the problem of "how far is this object away from me" as long as the object is within a few thousand times the physical aperture of the lens. It's mature enough that you can use it to drive a motor in response to movements or "that bird", "that teapot", "the closest object in that clump of pixels", "the farthest objects in that clump of pixels", as well as calculate the velocity of the aforementioned object in three dimensions. For all intents and purposes, if you have a problem of the order "what is the distance of that object" as well as "how is that distance changing over time", then you can solve it, and indeed it has been solved to very high reliability, using phase detection. That's what it means for a problem to be solved. In other words, I wouldn't call an implementation where you can click anywhere on an image and receive a distance, all of the time, with almost any lens imaginable in any environment where there are enough photons hitting the sensor "performed in a limited context". The technology simply hasn't been used in mainstream robotics, mainly because it's patented and difficult to implement from scratch, but these are all implementation problems not fundamental problems.

lisper 6y ago

> Guys like Rodney Brooks seemed to accept this and built various robots that would learn how to walk using primitive hardware and feedback oriented ideas rather than programmed ideas. There was even a name for this; “Nouvelle AI.” No idea what happened to those ideas; I suppose they were too hard to make progress on, though the early results were impressive looking. The problem was that subsumption didn't scale, but the idea was incorporated into the so-called three-layer architecture: http://www.flownet.com/ron/papers/tla.pdf http://www.flownet.com/ron/papers/tla.pdf (I am the author of that paper. AMA.)

pilingual 6y ago
This paper seems to pre-date behavior trees. Any comment on how TLA relates? (I skimmed but will give a closer look when I have more time.) I was curious what Brooks now thinks about subsumption and found this from a few months ago: The approach to controlling robots, the subsumption architecture that it proposed led directly to the Roomba, a robot vacuum cleaner, which with over 30 million sold is the most produced robot ever.¹ Like many, I got a Roomba not long after the pandemic began. I was disappointed by its poor sensorimotor system. Within a few days its IR cover was scuffed up and it was covered in scratches from getting stuck under an office chair. Brooks doesn't say that 30mm Roombas incorporate subsumption, and no doubt after a decade or two of programmers I wonder about the nature of the codebase. The Roomba's behavior is entirely unpredictable, as sometimes it will bump into something full speed and sometimes it will slow down as it approaches. There are a number of other issues too long to mention including the charging contacts and recently it started roaming around with its charger still attached for no apparent reason. ¹ https://rodneybrooks.com/peer-review/ https://rodneybrooks.com/peer-review/
- lisper 6y ago
  > This paper seems to pre-date behavior trees. Indeed. By a good 20 years :-) > Any comment on how TLA relates? I've been out of the field for a long time so BTs are new to me. All I know about them is from skimming the wikipedia article. But they look to me like a more formal implementation of the TLA sequencing layer.

tgflynn 6y ago

I think one of the problems is that we don't have the right people working on the right problems. Self-driving cars seem to have absorbed most of the world's high-level robotics efforts for the past decade or more. I was always skeptical of that application because my experience has shown that weak-AI works best when there's a human backup and/or the stakes of an error are not too high. That isn't the case with self-driving cars where a lethal mistake can occur in less than a second. I think we would be much better off today if much of the self-driving car effort had been focused on household utility robots and/or business applications. No it wouldn't be "saving lives", but it would be saving countless life-hours spent on mind-numbing tasks that are a major reason for why life is so unpleasant for so many people.

mdorazio 6y ago
What household tasks do you think are well-suited to robotics that aren't already addressed? The reason so much focus has gone into autonomous vehicles is because there are trillions of dollars to be made there by the companies that solve it, and many obvious use cases will result in large profits even with imperfect solutions (where "imperfect" means it works only in specific design domains like the southwestern US during the day with no rain).
- tgflynn 6y ago
  All of them - cleaning, cooking, laundry, etc. Everything that a human needs to do to maintain reasonable living conditions and would hire someone else to do if they were rich. EDIT: I think that if you could come up with a really good solution for all of that most people would be willing to pay about as much as they do for a car.

syntaxing 6y ago

I was really into FIRST robotics when I in HS and would love to make something at home. I thought about a VEX kit but didn't really want an erector set like. Does anyone have a recommendation for something like Misty Robotics [1]. I can only think of the Mindstorm but wanted to do some visual AI with it. [1] https://www.mistyrobotics.com/ https://www.mistyrobotics.com/

r34 6y ago

The only solution to mentioned problems is IMO: we'll have to adapt our environment to the moving machines. At least in early stage, before they will do it autonomously. Language will adapt more by itself (to make machines pass Turing test we need change in language, just as we need change in machines - people will adapt to useful machines)

stillsut 6y ago

The pile of problems in Robotics reminds me of the challenges faced by computer vision before modern methods were developed. To generalize the issue: humans learn and perform vision and navigational tasks "below" the level of language. To use a vivid example, you can use language to teach a child how to hold a pencil and write, how to recognize digits, and how to do two digit addition. But there's some cognitive ability "below" the level language a child needs to be capable of in order to take the hints from language and develop the desired skills. In computer vision, we now have something like this in ConvLayers. In today's robotics we see the hyper-systematization and mathematization of natural concepts like getting an agent to recognize its own bearing. This is what I refer to as "at the level of language", and I think as long as we try to solve these individual navigation problems from mathematical first principles we'll never arrive anywhere near the performance of animal "instinct". We need a new framework that doesn't look like an enumeration of commands within an imperative program.

rajansaini 6y ago
That's a very interesting perspective. There have been many articles complaining about the "hype" behind ML, but I too wonder if DNNs could assist with controls, especially when it comes to reacting to sensor data. After all, it's just matrix math.

nojvek 6y ago

What really surprises me is Apple is now 2 trillion dollars with another gajillion sitting in cash. Why are they being an optimization company and not really pushing the edge on robotics and automation? I was excited about Apple car but it seems that project is ded. Why not invest that money in moonshot ideas ? It seems Elon musk is the crazy one with Tesla, Solar City and SpaceX. It blows my mind how their massive rockets reach orbit and land back. Why haven’t we been making breakthroughs like this in robotics?

cpgxiii 6y ago
As difficult as it is, control for launching and landing a rocket is vastly easier computationally than even apparently simple robotics problems. Robotics is simply hard, from algorithmic, computational, and engineering perspectives. Only some parts of it can be solved by throwing money and raw compute power at it. Even in areas where progress has been made, you generally need experts to decompose the problem at hand into something that can be tackled with existing techniques,and those experts are expensive and in relative demand.

blauditore 6y ago

> 6. Depth estimation At least indoors, this was already handled pretty well 10 years ago by Kinect, and there are many somewhat robust approaches based on binocular vision. Not sure if this qualifies as "very much an open problem". > 9. Scene understanding The mentioned example is not the best one - anticipation of collisions has been a feature in cars for many years now (usually some warning system, sometimes with automatic emergency breaking). But it's a very constrained problem; more generic scene understanding is of course still very difficult.