14 ms·

The Mythical Non-Roboticist

199 points by robobenjie 3y ago

jvanderbot 3y ago
I am fond of saying there are only two hard problems in robotics: Perception and Funding. If you have a magical sensor that answers questions about the world, and have a magic box full of near-limitless money, you can easily build any robotic system you want. If perception is "processing data from sensors and users so we can make decisions about it", then there isn't much robotics left. Got a controls problem? forward predict using the magic sensor. Got a planning problem? just sense the world as a few matrices and plug it into an ILP or MDP. What did the user mean? Ask the box. etc etc. Distilling the world into the kind of input our computers require is immesnely difficult, but once that's done "My" problem (being a planning expert) is super easy. I'm often left holding the bag when things go wrong because "my" part is built last (the planning stack), and has the most visible "breaks" (the plan is bad). But it's 90% of the time traceable up to the perception, or a violated assumption about the world. TFA is spot on - it's just not clear how to sense the world to make "programming" robotics a thing. In the way you'd "program" your computer to make lines appear on a screen or packets fly across the internet, we'd love to "program" a robot to pick up an object and put it away, but even a specious attempt to define generally what "object" and "put away" mean is still 100s of PhD theses away.So it's like we invent the entire ecosystem from scratch each time we build a new robot.
- contingencies 3y ago
  Cute quote - added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup :) I would add supply chain, however.
  - jvanderbot 3y ago
    An honor! Pleased to contribute.
  - transitionnel 3y ago
    To solve that: Assumption: Apple's supply chain is gold standard [~max iterative tech envelope push & max known demand] Hypothesis: This is swiftly re-creatable for any [max believable & max useful] product. "Detroit, waiting".
- transitionnel 3y ago
  It's so great to read genuine yet experienced insight like this. Like last night on Twitter I saw an opening for Robotic Behavior Coordinator at Figure. I know for sure, having analyzed this problem with "nothing else" to do for 20 years, I would crush it with humility, and humanity would profit in orders of magnitude. But they are not set up to hand me control of the rounding error of $40M I'd like [and would pay forward], *nor would their teams listen to me, due to human nature and academ-uenza*. Such is our loss. (as you ~say, "reinventing the ecosystem from scratch...")
  - transitionnel 3y ago
    Ah, sorry if I sounded like a douche. Have my Y-C idea now. here we gooooo ..!.. ;)
  - nharada 3y ago
    > humility > humanity would profit in orders of magnitude
    - transitionnel 3y ago
      >> touché :) >> but please believe, I would not risk ostracism on this (my favorite) forum if I were not [approaching] 100% sure.
- yakz 3y ago
  even a specious attempt to define generally what "object" and "put away" mean is still 100s of PhD theses away Is this part still true? There are widely available APIs (and even running at home on consumer level hardware to some extent) that can pick an object out of an image, describe what it might be useful for and where it could go.
  - kaibee 3y ago
    > that can pick an object out of an image You have to do it in real time, from a video feed, and make sure that you're tracking the same unique instance of that object between frames.
    - lukan 3y ago
      Robots could make a short stop or go slower to process an unclear picture, that is probably not the problem - but the image processing itself, is still way too unreliable. Under ideal condition it mostly works, but have some light fog in the picture or strong sunlight and ... usually all fails. Otherwise the Teslas would have indeed full self driving mode, using only cameras.
      - thfuran 3y ago
        >Robots could make a short stop or go slower to process an unclear picture The costs of doing so are hugely dependent application. It is not, for example, an attractive strategy for an image-guided missile, though it's probably fine for an autonomous vacuum cleaner.
    - YeGoblynQueenne 3y ago
      And then you need to grasp it.
  - ska 3y ago
    It's definitely not a solved problem in general, especially in realtime. It's a lot easier to get started on something interesting and maybe even useful than it was even 10 years ago. A lot of the "ah we can just use X API" falls apart pretty fast when you do risk analysis on a real system. Lots of these APIs are do a decent job most of the time under somewhat ideal conditions, beyond that things get hairy.
- ska 3y ago
  Really there are three problem in robotics: Perception, Funding, and Cables :)
  - etrautmann 3y ago
    Connectors imo :)
    - taneq 3y ago
      And fasteners. I swear any automation system is 90% cables, connectors and fasteners by weight.
  - smoldesu 3y ago
    Only one of them is fun to manage.
    - marcosdumay 3y ago
      Perception, right?
  - leoedin 3y ago
    Totally. I worked on the electronics in robot arms for a while and EVERY TIME there was a failure in the field - it was the cables.
- glenngillen 3y ago
  I love this perspective. It’s also made me draw parallels between the experiences with actual people, especially others in my household. With young children who are at the early parts of “doing household chores” of development there is basically constant refinement on what “clean the floor”, “put things away”, etc. _really_ means. I know my wife and I have different definitions on these things too. Our ability to be clear and exhaustive enough upfront on the definitions to have a complete perception and set of assumptions is basically non-existent. We’re all only human! But our willingness to engage in fixing that with humans is also high. If my kids repeatedly miss a section under some chairs when vacuuming we talk about it and know it will improve. When my Roomba does it it sucks and can’t do its job properly. Even thinking about hiring professional trades people to come do handiwork it’s rarely perfect the first time. Not because they’re bad, just because being absolutely precise about things upfront can be so difficult.
- DrDroop 3y ago
  What about transformers for robotics, like ALOHA, they seems to help with learning new tasks.
BWStearns 3y ago
> once they are programming a robot, I feel they become roboticists Yes! I am not a roboticist (or at least a good one in any sense) but I was having a similar discussion regarding enabling non-technical users do data analysis. Once they start doing anything more complicated than `SELECT COUNT(*) FROM blah WHERE foo=otherblah` it's going to get real real quick. You can't just give them some cheap point and click stuff because their questions will immediately overrun the extent of what's practicable. Asking interesting questions of data is roughly as difficult as phrasing the questions in SQL (or any other formal query language) and anyone who can do the first can do the latter easily enough. (or the point and click stuff _is_ really powerful but it's some proprietary non-googleable voodoo that requires a month long training course that costs $5K/week to get a certificate and become middlingly powerful)
- marcosdumay 3y ago
  Yep. The entire article is the low-code fallacy applied to robot programing. It will be the same in any branch of programing you look.
  - feoren 3y ago
    > low-code fallacy I like it that we have a name for this now. Let's keep calling it the "low-code fallacy", because I'm tired of explaining over and over the same idea that semicolons and for loops are not what makes programming hard.
    - notpachet 3y ago
      Exactly. It's that damned JS "class" keyword! ...right?
      - lukan 3y ago
        Actually yes, I think stuff like this makes programming hard. A half ass implementation of "class", not behaving like a class, brings unnecessary confusion. Programming in the real world, is full of these details, you have to know to be productive. 0.1+0.2 = 0.30000000000000004 in many languages is another one. (And semicolons are ugly and I avoid them, wherever I can get away with it, but no, are probably not the reason)
paulsutter 3y ago
Robotics has been completely transformed in the last six months, check out this Figure video from yesterday Robotics is dead. Long live robotics. https://x.com/Figure_robot/status/1767913661253984474?s=20 https://x.com/Figure_robot/status/1767913661253984474?s=20
- transitionnel 3y ago
  That's fake man. But yeah, I checked out those jobs ;) They certainly know what to do. Just not how.
  - sashank_1509 3y ago
    Why do you call it fake?
    - transitionnel 3y ago
      well, CGI as opposed to footage of a physical robot doing those things shown. They forgot the: *for inspirational purposes only
      - sashank_1509 3y ago
        I’m willing to bet a million dollars there was no CGI in figures recent video. What video are you even talking about??
        transitionnel 3y ago
        "I hope you're right ;)" "They have a better chance at proving or disproving this than we do."
      - numpad0 3y ago
        how did I not notice this obvious greenscreen composition!? it is!
- yakz 3y ago
readenough 3y ago
My personal view, as an industrial control systems engineer, is that so much of the world's production software requires teams of software professionals to monitor it and keep it working. When these same software professionals look at systems which physically interact with the real world on a real time basis then a different dynamic comes into play.
- varjag 3y ago
  There's certainly that. Noone cares about that hot stack when your distributed system needs to work 24/7/365 for a couple decades with 0 SREs.
DoctorDabadedoo 3y ago
Loved the TFA. I've been working on robotics pretty much my whole career and people usually miss how complicated it can get even for simple things once you consider what can go wrong AND it's a meeting place for a multitude of areas: hardware, software, mechanical, electrical, machine learning, computer vision, control, driver, database, etc. An issue can hide in between any of those for months before it shows up with bells and whistles. What is sometimes difficult to get across to people is that building robots is not only difficult per se, but the base of comparison is unusually unfair: if you build an e-commerce website you benchmark it against other e-commerce websites, maybe Amazon, maybe ebay; for robots usually the benchmark is against people, the most adaptable and fault tolerant machine that exists, every robot will suck compared to a human doing the same task, but that's what we compare it to every time.
- lukan 3y ago
  "every robot will suck compared to a human doing the same task, but that's what we compare it to every time" What about a factory robot, welding together a part of a car?
  - DoctorDabadedoo 3y ago
    That would fall under the "automation" category: a very specialized customized application of robotics, doing the same set of tasks over and over, this is the kind of application where we can really see the power of robotics, but rest assured that countless hours were spent testing/improving/optimizing/safe guarding these workflows and after every section in an assembly line there will be manual inspection to flag for bad / missing weldings and potential service of the machinery involved.
    - gertlex 3y ago
      Would "single purpose robot" be another reasonable term for welding robots? Just musing. The earlier "when compared to humans" statement definitely sounds pretty accurate to me, worded as "mutli-purpose robots currently always are less robust than humans at the same set of tasks" (or similar)
Animats 3y ago
This is just a phase. The Internet went through this. It was criticized in the early days as requiring "too many PhDs per packet". Eventually, with standardization and automation, we got past that. Now anybody can connect. Rethink Robotics went bust because they couldn't solve this usability problem. It's a problem at a much higher level than the author is talking about. If you're driving your robot with positional data, that's easy to understand, but a huge pain to set up. Usually, you have very rigid tooling and feeders, so that everything is where it is supposed to be. If it's not, you shut down and call for a human. What you'd often like to do is an assembly task like this: - Reach into bin and pull out a part. - Manipulate part until part is in standard orientation. - Place part against assembly so that holes align. - Put in first bolt, leave loose. - Put in other bolts, leave loose. - Tighten all bolts to specified torque. Each of those is a hard but possible robotic task at present. Doing all of those together is even harder. Designing a system where the end user can specify a task at that level of abstraction does not seem to have been done yet. Somebody will probably crack that problem in the next five years.
- samatman 3y ago
  There are only three timeframes for tech forecasts: - one year (someone is building this) - five years (no one knows how to solve this problem but a lot of people are working on it and y'know, eventually you get lucky) - ten years (this isn't forbidden by the laws of physics but it's bloody impossible as far as anyone knows)
  - Animats 3y ago
    It's more that robotics can now mooch off the AI boom. All that money going into adtech and surveillance produces technology that can be used to solve practical problems.
    - hospadar 3y ago
      > All that money going into adtech and surveillance produces technology that can be used to solve practical problems. Problems like "how do we build better automated surveillance robots? it's so inconvenient to have to actually have a human remotely piloting the kill-bots"
jebarker 3y ago
> Design your APIs for someone as smart as you, but less tolerant of stupid bulls*t. This is definitely applicable outside of robotics. For example, I work on a large-scale LLM training framework and tend to think this way when thinking about design decisions.
serf 3y ago
as someone that messes with every low cost robotics thing. this part stuck out as painfully true : “Oh yeah, if you try to move the robot without calling enable() it segfaults. That's a safety feature… I guess? But also if you call it twice, that also segfaults. Just call it exactly once, ever.”
- tonyarkles 3y ago
  Well, hate to tell you this, but it’s generally not much different in a lot of the professional world either. The amount of bullshit I’ve had to deal with to make $20k hardware work mostly reliably still boggles the mind. From the article: Design your APIs for someone as smart as you, but less tolerant of stupid bullshit. One of the most painful parts of doing this professionally is that the people that work at a few of our vendors are incredibly smart and are selling us hardware that we can’t really get anywhere else, but they’re generally Electrical Engineers or Optical Engineers or Physicists and don’t even realize that the APIs they’re providing are bad. You file a bug, they tell you you’re holding it wrong, you point out the footgun in the API, and they come back and ask what that even means. …It’s not until you debug their closed source library using Ghidra and tell them they missed a mutex in a specific function that they start treating you as anything more than a moron. Anyway </rant>
woah 3y ago
Interestingly, a lot of these things are the same challenges that no-code platforms face
AndrewKemendo 3y ago
The goal of computing is, and has always been, controlling the behavior of machines the same way or easier than we do with other agents in the world toward some measurable end So, to what level of granularity do you have to specify a system task in order for it to do the thing you want it to do, at the level of accuracy that you wanted to operate in? That all depends on how accurate you can specify what you want to do which means you have a sense of all of the systems that interact with, and impede the successful task of the set of systems We can build abstraction layers we can build filters, but at some point somebody has to map a set of actions with a set of inputs and outputs, in order to sequentially build this set of tasks, which rolls out into the function of a physical manifestation of some sort Add to that the complexities of mobile actuation complex environments and just the general state of power, computing, routing, etc. and you have a 15 body problem simply to have anything that someone would look at as benefit to humanity Only a couple of disciplines can totally encapsulate all that and none of them are available to study anymore primarily cybernetics, and all of the interactions necessary to fully build a human machine symbiotic system
- transitionnel 3y ago
  > "you have a 15 body problem simply to have anything..." I like that! Although...Physics [so gpu] is enough to do it, when supplied with an optimized way to "know" momentary_[intent/\status] as a reduced ongoing string of equations.
  - jagged-chisel 3y ago
    I’m not going to pretend I know anything about the field, nor do I intend to insult your comment … but this reads to me exactly like the reduction of the subject that the article mentions.
AtlasBarfed 3y ago
So if LLMs are so great, I would think binding black box robotics hardware with apis would lead to a revolution in robotics. That sort of one step implementation seems to be a sweet spot for llm Problem is a lack of available examples for training?
- transitionnel 3y ago
  One would think...all of the above. I think, really, the emperor has had no clothes for quite some time. But -- now that we are here, the optimal path is towards open standards. All historical business acumen points straight to black-box profit bubble. The enormity of what "Useful Robotics" will bring about has got to transcend that.
  - transitionnel 3y ago
    ...as soon as the MBAs realize "no bubble", in real actuality, Means "very many even bigger bubbles", and usher forth to unleash their true potential... we are set!
reason-mr 3y ago
Traditional machine vision developers: I have 10,000 problems. You can’t do this. Neural network people: watch this space, I have a shotgun.
- ansgri 3y ago
  As a former traditional machine vision researcher, currently robotics-adjacent software engineer, I agree, but: they've been brandishing that shotgun for a pretty long time before it became standard issue, for varying reasons. To continue your analogy, I'm pretty sure the (pseudo-)reasoning ability of GPT4-level LLMs could solve a lot of hard robotic problems (perception ambiguity, external agent behavior prediction etc), but now it's a stationary missile silo, and we need this weapon on every APC to be a solution.
atoav 3y ago
As someone who runs a medialab at an art school it is fascinating how many people believe because they understood the general principle of a thing, it is therefore simple to just do it. Many people seem to long for a magical technology that you could just pour over things and they will work out in the ways you wanted, while miraculously sensing the ways you didn't. Those with the edge on the new tech will always be those who have a good understanding of it's limitations, because once a new thing comes around they immediately see the possibilities.
moffkalast 3y ago
> Design your APIs for someone as smart as you, but less tolerant of stupid bullshit. I feel like this has been the problem plaquing the ROS navigation stack since move_base and now nav2. They design the API for people a few standard deviations smarter than everyone else on the planet. Billions of parameters that affect each other in unpredictable ways and you're supposed to read the thesis on each one. Or do what most everyone else does and use the defaults and hope for the best, lmao. You either make an API that the average user will understand or it'll inevitably be used as a black box.
TaylorAlexander 3y ago
I used to work with Benjie at X and he was one of my favorite people. Benjie if you see this it is great to see you doing this kind of writing and I love this article!
superb-owl 3y ago
I was super excited to take a robotics class in college. I'd fallen in love with programming and was excited to take all that magic into the real world. We all had to buy roombas to program. The final exam was getting it to traverse a maze. It seemed so simple! They even gave us the exact dimensions and layout ahead of time. Just hard-code the path, right? Spin the wheels so many rotations, turn 90 degrees, spin some more. Except the real world is messy, and tiny errors add up quickly. One of the wheels hits a bump, or slips a little on the tile, and suddenly you're way off course. Without some kind of feedback loop to self-correct, everything falls apart. My excitement for robotics died quickly. I much prefer the perfectly constrained environment of a CPU.
- defrost 3y ago
  Forty years in university (while I resided in a residential college) I was also excited to work on robotics. We were expected to assist in machining parts, building control libraries from scratch, working out algorithms from scratch for path generation, etc. The goal was to shear a sheep: https://www.youtube.com/watch?v=6ZAh2zv7TMM https://www.youtube.com/watch?v=6ZAh2zv7TMM
- lukan 3y ago
  Oh yes, we were building and programming Lego Mindstorm robots in university. Also with the goal to go through a simple maze and follow a line. Booring simple everyone thought. But the thirst thing we learned, was to not trust our sensors. My expectation was, if the ultrasonic sensor said, there is 1 m to an obstacle, then there is 1 m of free space. Well, not really. Partly because the sensors were really bad and only worked reliable when the obstacle was in a 90 degree angle, partly it is in the nature of sensors to not be perfect. I am still excited for robots though, but haven't worked on one in quite a while.
bitwize 3y ago
s/roboticist/programmer/g and you get an infinite bullshit generator for the world of business and enterprise software, without even firing up ChatGPT! Having worked in both industries, I concur that robotics is much, much messier, as the system has to engage via hardware with the super-messy physical world as opposed to the comparatively modestly messy world of business transactions, data analysis, or whatever. But if we stop trying to solve for "programming for nonprogrammers" and assume that anyone who uses a language or API is a programmer (because once you start programming, that's what you become, irrespective of what's in your job title), we can remove a whole lot of wasted effort from the industry.
a_t48 3y ago
I work as a non-roboticist at a robot company. Most of my job is to enable people with PhDs to do work and to clean up after people who hastily stood up some sort of infrastructure (either actual infra, tooling, or libraries) so that they could go work on the thing they were actually interested in. Occasionally I'll get to work on vaguely actual robot things - drivers, communications frameworks, timing, etc.
creesch 3y ago
> That would be great, right? We should make a software framework so that non-roboticists can program robots. Lol, I work in the field of test automation and this is exactly how no/low code frameworks get pushed as well. And, it rarely does play out in a way that people think it will. In fact, having read the entire article, I feel like a lot of it can be applied more broadly. Basically any time people go "X sure is complex, we should make a simple to use framework for non-X folks to use". Not that it will always fail, but I have seen it happen enough to recognize a pattern.
- deleted 3y ago
  [deleted]
- hitchstory 3y ago
  BTW, odd question - assume somebody built a low/less code test automation framework that didn't fail and was very effective but it also wasn't popular. What could it do to stand out to you? What could it demonstrate in 15-20 seconds for you to think "OK this is different"?
  - creesch 3y ago
    1. Come without vendor lock in. 2. Format is human readable so it can actually be integrated in a proper development cycle (git, etc). 3. Doesn't make it a nightmare to do custom things where needed. It is not impossible. Out the top of my head Robot Framework does fit those criteria. But I'd argue that Robot Framework isn't really low code, but rather a coded framework in a low code trench coat.
    - hitchstory 3y ago
      Thanks, that's very helpful. 1 and 2 make perfect sense and are easy do demonstrate but 3 seems to me to be incredibly difficult. I haven't found an easy way to advertise convincingly to somebody who (quite reasonably) grants you a limited amount of attention that custom things won't be a nightmare. It's the kind of thing you only tend see when you get dug in the weeds and hence people will tend to make assumptions based upon surface details. This is a problem I'm struggling with. I think robot/cucumber could require less code if they were better abstractions (and would be more loved), but I find it hard to illustrate that an abstraction is going to be good or bad, particularly to people with limited attention and particularly to people who don't necessarily have the skills to recognize a good abstraction.
fedeb95 3y ago
This is relevant to many domains, with different degrees of relevance.
deleted 3y ago
[deleted]
jjk166 3y ago
I think the real takeaway from this article, which is applicable to pretty widely applicable, is that a lot of times when you get the requirement "make it simple" the actual requirement is "make it intuitive." Taking away functionality doesn't typically make things any more intuitive, indeed they often make things much less intuitive because a lot of things need to be coupled that a user would not naturally expect to be. Conversely giving the user tons of options but making sure they are distinct, clearly named, their interfaces are consistent, and the defaults are sensible allows someone who understands the problem they want to use the api to solve to jump right in.
markisus 3y ago
This quote hits close to home. “So I got the arm to match the conveyor speed by monitoring the position and pre-empting the motion command, alternating ‘slow’ and ‘fast’ at 10Hz with a duty cycle depending on how we are tracking our target. The motion is pretty jerky but it works.” I’ve been through exactly this scenario in two very popular robotics frameworks. Countless hours were spent by well intentioned framework developers to abstract away the underlying PID loop from the end users. Then countless more hours were spent by the end users to work around this by implementing a PID loop on top of the abstraction.

199 points by robobenjie 3y ago

I am fond of saying there are only two hard problems in robotics: Perception and Funding. If you have a magical sensor that answers questions about the world, and have a magic box full of near-limitless money, you can easily build any robotic system you want. If perception is "processing data from sensors and users so we can make decisions about it", then there isn't much robotics left. Got a controls problem? forward predict using the magic sensor. Got a planning problem? just sense the world as a few matrices and plug it into an ILP or MDP. What did the user mean? Ask the box. etc etc. Distilling the world into the kind of input our computers require is immesnely difficult, but once that's done "My" problem (being a planning expert) is super easy. I'm often left holding the bag when things go wrong because "my" part is built last (the planning stack), and has the most visible "breaks" (the plan is bad). But it's 90% of the time traceable up to the perception, or a violated assumption about the world. TFA is spot on - it's just not clear how to sense the world to make "programming" robotics a thing. In the way you'd "program" your computer to make lines appear on a screen or packets fly across the internet, we'd love to "program" a robot to pick up an object and put it away, but even a specious attempt to define generally what "object" and "put away" mean is still 100s of PhD theses away.So it's like we invent the entire ecosystem from scratch each time we build a new robot.

contingencies 3y ago
Cute quote - added to https://github.com/globalcitizen/taoup https://github.com/globalcitizen/taoup :) I would add supply chain, however.
- jvanderbot 3y ago
  An honor! Pleased to contribute.
- transitionnel 3y ago
  To solve that: Assumption: Apple's supply chain is gold standard [~max iterative tech envelope push & max known demand] Hypothesis: This is swiftly re-creatable for any [max believable & max useful] product. "Detroit, waiting".
transitionnel 3y ago
It's so great to read genuine yet experienced insight like this. Like last night on Twitter I saw an opening for Robotic Behavior Coordinator at Figure. I know for sure, having analyzed this problem with "nothing else" to do for 20 years, I would crush it with humility, and humanity would profit in orders of magnitude. But they are not set up to hand me control of the rounding error of $40M I'd like [and would pay forward], *nor would their teams listen to me, due to human nature and academ-uenza*. Such is our loss. (as you ~say, "reinventing the ecosystem from scratch...")
- transitionnel 3y ago
  Ah, sorry if I sounded like a douche. Have my Y-C idea now. here we gooooo ..!.. ;)
- nharada 3y ago
  > humility > humanity would profit in orders of magnitude
  - transitionnel 3y ago
    >> touché :) >> but please believe, I would not risk ostracism on this (my favorite) forum if I were not [approaching] 100% sure.
yakz 3y ago
even a specious attempt to define generally what "object" and "put away" mean is still 100s of PhD theses away Is this part still true? There are widely available APIs (and even running at home on consumer level hardware to some extent) that can pick an object out of an image, describe what it might be useful for and where it could go.
- kaibee 3y ago
  > that can pick an object out of an image You have to do it in real time, from a video feed, and make sure that you're tracking the same unique instance of that object between frames.
  - lukan 3y ago
    Robots could make a short stop or go slower to process an unclear picture, that is probably not the problem - but the image processing itself, is still way too unreliable. Under ideal condition it mostly works, but have some light fog in the picture or strong sunlight and ... usually all fails. Otherwise the Teslas would have indeed full self driving mode, using only cameras.
    - thfuran 3y ago
      >Robots could make a short stop or go slower to process an unclear picture The costs of doing so are hugely dependent application. It is not, for example, an attractive strategy for an image-guided missile, though it's probably fine for an autonomous vacuum cleaner.
  - YeGoblynQueenne 3y ago
    And then you need to grasp it.
- ska 3y ago
  It's definitely not a solved problem in general, especially in realtime. It's a lot easier to get started on something interesting and maybe even useful than it was even 10 years ago. A lot of the "ah we can just use X API" falls apart pretty fast when you do risk analysis on a real system. Lots of these APIs are do a decent job most of the time under somewhat ideal conditions, beyond that things get hairy.
ska 3y ago
Really there are three problem in robotics: Perception, Funding, and Cables :)
- etrautmann 3y ago
  Connectors imo :)
  - taneq 3y ago
    And fasteners. I swear any automation system is 90% cables, connectors and fasteners by weight.
- smoldesu 3y ago
  Only one of them is fun to manage.
  - marcosdumay 3y ago
    Perception, right?
- leoedin 3y ago
  Totally. I worked on the electronics in robot arms for a while and EVERY TIME there was a failure in the field - it was the cables.
glenngillen 3y ago
I love this perspective. It’s also made me draw parallels between the experiences with actual people, especially others in my household. With young children who are at the early parts of “doing household chores” of development there is basically constant refinement on what “clean the floor”, “put things away”, etc. _really_ means. I know my wife and I have different definitions on these things too. Our ability to be clear and exhaustive enough upfront on the definitions to have a complete perception and set of assumptions is basically non-existent. We’re all only human! But our willingness to engage in fixing that with humans is also high. If my kids repeatedly miss a section under some chairs when vacuuming we talk about it and know it will improve. When my Roomba does it it sucks and can’t do its job properly. Even thinking about hiring professional trades people to come do handiwork it’s rarely perfect the first time. Not because they’re bad, just because being absolutely precise about things upfront can be so difficult.
DrDroop 3y ago
What about transformers for robotics, like ALOHA, they seems to help with learning new tasks.

BWStearns 3y ago

> once they are programming a robot, I feel they become roboticists Yes! I am not a roboticist (or at least a good one in any sense) but I was having a similar discussion regarding enabling non-technical users do data analysis. Once they start doing anything more complicated than `SELECT COUNT(*) FROM blah WHERE foo=otherblah` it's going to get real real quick. You can't just give them some cheap point and click stuff because their questions will immediately overrun the extent of what's practicable. Asking interesting questions of data is roughly as difficult as phrasing the questions in SQL (or any other formal query language) and anyone who can do the first can do the latter easily enough. (or the point and click stuff _is_ really powerful but it's some proprietary non-googleable voodoo that requires a month long training course that costs $5K/week to get a certificate and become middlingly powerful)

marcosdumay 3y ago
Yep. The entire article is the low-code fallacy applied to robot programing. It will be the same in any branch of programing you look.
- feoren 3y ago
  > low-code fallacy I like it that we have a name for this now. Let's keep calling it the "low-code fallacy", because I'm tired of explaining over and over the same idea that semicolons and for loops are not what makes programming hard.
  - notpachet 3y ago
    Exactly. It's that damned JS "class" keyword! ...right?
    - lukan 3y ago
      Actually yes, I think stuff like this makes programming hard. A half ass implementation of "class", not behaving like a class, brings unnecessary confusion. Programming in the real world, is full of these details, you have to know to be productive. 0.1+0.2 = 0.30000000000000004 in many languages is another one. (And semicolons are ugly and I avoid them, wherever I can get away with it, but no, are probably not the reason)

paulsutter 3y ago

Robotics has been completely transformed in the last six months, check out this Figure video from yesterday Robotics is dead. Long live robotics. https://x.com/Figure_robot/status/1767913661253984474?s=20 https://x.com/Figure_robot/status/1767913661253984474?s=20

transitionnel 3y ago
That's fake man. But yeah, I checked out those jobs ;) They certainly know what to do. Just not how.
- sashank_1509 3y ago
  Why do you call it fake?
  - transitionnel 3y ago
    well, CGI as opposed to footage of a physical robot doing those things shown. They forgot the: *for inspirational purposes only
    - sashank_1509 3y ago
      I’m willing to bet a million dollars there was no CGI in figures recent video. What video are you even talking about??
      - transitionnel 3y ago
        "I hope you're right ;)" "They have a better chance at proving or disproving this than we do."
    - numpad0 3y ago
      how did I not notice this obvious greenscreen composition!? it is!
yakz 3y ago

readenough 3y ago

My personal view, as an industrial control systems engineer, is that so much of the world's production software requires teams of software professionals to monitor it and keep it working. When these same software professionals look at systems which physically interact with the real world on a real time basis then a different dynamic comes into play.

varjag 3y ago
There's certainly that. Noone cares about that hot stack when your distributed system needs to work 24/7/365 for a couple decades with 0 SREs.

DoctorDabadedoo 3y ago

Loved the TFA. I've been working on robotics pretty much my whole career and people usually miss how complicated it can get even for simple things once you consider what can go wrong AND it's a meeting place for a multitude of areas: hardware, software, mechanical, electrical, machine learning, computer vision, control, driver, database, etc. An issue can hide in between any of those for months before it shows up with bells and whistles. What is sometimes difficult to get across to people is that building robots is not only difficult per se, but the base of comparison is unusually unfair: if you build an e-commerce website you benchmark it against other e-commerce websites, maybe Amazon, maybe ebay; for robots usually the benchmark is against people, the most adaptable and fault tolerant machine that exists, every robot will suck compared to a human doing the same task, but that's what we compare it to every time.

lukan 3y ago
"every robot will suck compared to a human doing the same task, but that's what we compare it to every time" What about a factory robot, welding together a part of a car?
- DoctorDabadedoo 3y ago
  That would fall under the "automation" category: a very specialized customized application of robotics, doing the same set of tasks over and over, this is the kind of application where we can really see the power of robotics, but rest assured that countless hours were spent testing/improving/optimizing/safe guarding these workflows and after every section in an assembly line there will be manual inspection to flag for bad / missing weldings and potential service of the machinery involved.
  - gertlex 3y ago
    Would "single purpose robot" be another reasonable term for welding robots? Just musing. The earlier "when compared to humans" statement definitely sounds pretty accurate to me, worded as "mutli-purpose robots currently always are less robust than humans at the same set of tasks" (or similar)

Animats 3y ago

This is just a phase. The Internet went through this. It was criticized in the early days as requiring "too many PhDs per packet". Eventually, with standardization and automation, we got past that. Now anybody can connect. Rethink Robotics went bust because they couldn't solve this usability problem. It's a problem at a much higher level than the author is talking about. If you're driving your robot with positional data, that's easy to understand, but a huge pain to set up. Usually, you have very rigid tooling and feeders, so that everything is where it is supposed to be. If it's not, you shut down and call for a human. What you'd often like to do is an assembly task like this: - Reach into bin and pull out a part. - Manipulate part until part is in standard orientation. - Place part against assembly so that holes align. - Put in first bolt, leave loose. - Put in other bolts, leave loose. - Tighten all bolts to specified torque. Each of those is a hard but possible robotic task at present. Doing all of those together is even harder. Designing a system where the end user can specify a task at that level of abstraction does not seem to have been done yet. Somebody will probably crack that problem in the next five years.

samatman 3y ago
There are only three timeframes for tech forecasts: - one year (someone is building this) - five years (no one knows how to solve this problem but a lot of people are working on it and y'know, eventually you get lucky) - ten years (this isn't forbidden by the laws of physics but it's bloody impossible as far as anyone knows)
- Animats 3y ago
  It's more that robotics can now mooch off the AI boom. All that money going into adtech and surveillance produces technology that can be used to solve practical problems.
  - hospadar 3y ago
    > All that money going into adtech and surveillance produces technology that can be used to solve practical problems. Problems like "how do we build better automated surveillance robots? it's so inconvenient to have to actually have a human remotely piloting the kill-bots"

jebarker 3y ago

> Design your APIs for someone as smart as you, but less tolerant of stupid bulls*t. This is definitely applicable outside of robotics. For example, I work on a large-scale LLM training framework and tend to think this way when thinking about design decisions.

serf 3y ago

as someone that messes with every low cost robotics thing. this part stuck out as painfully true : “Oh yeah, if you try to move the robot without calling enable() it segfaults. That's a safety feature… I guess? But also if you call it twice, that also segfaults. Just call it exactly once, ever.”

tonyarkles 3y ago
Well, hate to tell you this, but it’s generally not much different in a lot of the professional world either. The amount of bullshit I’ve had to deal with to make $20k hardware work mostly reliably still boggles the mind. From the article: Design your APIs for someone as smart as you, but less tolerant of stupid bullshit. One of the most painful parts of doing this professionally is that the people that work at a few of our vendors are incredibly smart and are selling us hardware that we can’t really get anywhere else, but they’re generally Electrical Engineers or Optical Engineers or Physicists and don’t even realize that the APIs they’re providing are bad. You file a bug, they tell you you’re holding it wrong, you point out the footgun in the API, and they come back and ask what that even means. …It’s not until you debug their closed source library using Ghidra and tell them they missed a mutex in a specific function that they start treating you as anything more than a moron. Anyway </rant>

woah 3y ago

Interestingly, a lot of these things are the same challenges that no-code platforms face

AndrewKemendo 3y ago

The goal of computing is, and has always been, controlling the behavior of machines the same way or easier than we do with other agents in the world toward some measurable end So, to what level of granularity do you have to specify a system task in order for it to do the thing you want it to do, at the level of accuracy that you wanted to operate in? That all depends on how accurate you can specify what you want to do which means you have a sense of all of the systems that interact with, and impede the successful task of the set of systems We can build abstraction layers we can build filters, but at some point somebody has to map a set of actions with a set of inputs and outputs, in order to sequentially build this set of tasks, which rolls out into the function of a physical manifestation of some sort Add to that the complexities of mobile actuation complex environments and just the general state of power, computing, routing, etc. and you have a 15 body problem simply to have anything that someone would look at as benefit to humanity Only a couple of disciplines can totally encapsulate all that and none of them are available to study anymore primarily cybernetics, and all of the interactions necessary to fully build a human machine symbiotic system

transitionnel 3y ago
> "you have a 15 body problem simply to have anything..." I like that! Although...Physics [so gpu] is enough to do it, when supplied with an optimized way to "know" momentary_[intent/\status] as a reduced ongoing string of equations.
- jagged-chisel 3y ago
  I’m not going to pretend I know anything about the field, nor do I intend to insult your comment … but this reads to me exactly like the reduction of the subject that the article mentions.

AtlasBarfed 3y ago

So if LLMs are so great, I would think binding black box robotics hardware with apis would lead to a revolution in robotics. That sort of one step implementation seems to be a sweet spot for llm Problem is a lack of available examples for training?

transitionnel 3y ago
One would think...all of the above. I think, really, the emperor has had no clothes for quite some time. But -- now that we are here, the optimal path is towards open standards. All historical business acumen points straight to black-box profit bubble. The enormity of what "Useful Robotics" will bring about has got to transcend that.
- transitionnel 3y ago
  ...as soon as the MBAs realize "no bubble", in real actuality, Means "very many even bigger bubbles", and usher forth to unleash their true potential... we are set!

reason-mr 3y ago

Traditional machine vision developers: I have 10,000 problems. You can’t do this. Neural network people: watch this space, I have a shotgun.

ansgri 3y ago
As a former traditional machine vision researcher, currently robotics-adjacent software engineer, I agree, but: they've been brandishing that shotgun for a pretty long time before it became standard issue, for varying reasons. To continue your analogy, I'm pretty sure the (pseudo-)reasoning ability of GPT4-level LLMs could solve a lot of hard robotic problems (perception ambiguity, external agent behavior prediction etc), but now it's a stationary missile silo, and we need this weapon on every APC to be a solution.

atoav 3y ago

As someone who runs a medialab at an art school it is fascinating how many people believe because they understood the general principle of a thing, it is therefore simple to just do it. Many people seem to long for a magical technology that you could just pour over things and they will work out in the ways you wanted, while miraculously sensing the ways you didn't. Those with the edge on the new tech will always be those who have a good understanding of it's limitations, because once a new thing comes around they immediately see the possibilities.

moffkalast 3y ago

> Design your APIs for someone as smart as you, but less tolerant of stupid bullshit. I feel like this has been the problem plaquing the ROS navigation stack since move_base and now nav2. They design the API for people a few standard deviations smarter than everyone else on the planet. Billions of parameters that affect each other in unpredictable ways and you're supposed to read the thesis on each one. Or do what most everyone else does and use the defaults and hope for the best, lmao. You either make an API that the average user will understand or it'll inevitably be used as a black box.

TaylorAlexander 3y ago

I used to work with Benjie at X and he was one of my favorite people. Benjie if you see this it is great to see you doing this kind of writing and I love this article!

superb-owl 3y ago

I was super excited to take a robotics class in college. I'd fallen in love with programming and was excited to take all that magic into the real world. We all had to buy roombas to program. The final exam was getting it to traverse a maze. It seemed so simple! They even gave us the exact dimensions and layout ahead of time. Just hard-code the path, right? Spin the wheels so many rotations, turn 90 degrees, spin some more. Except the real world is messy, and tiny errors add up quickly. One of the wheels hits a bump, or slips a little on the tile, and suddenly you're way off course. Without some kind of feedback loop to self-correct, everything falls apart. My excitement for robotics died quickly. I much prefer the perfectly constrained environment of a CPU.

defrost 3y ago
Forty years in university (while I resided in a residential college) I was also excited to work on robotics. We were expected to assist in machining parts, building control libraries from scratch, working out algorithms from scratch for path generation, etc. The goal was to shear a sheep: https://www.youtube.com/watch?v=6ZAh2zv7TMM https://www.youtube.com/watch?v=6ZAh2zv7TMM
lukan 3y ago
Oh yes, we were building and programming Lego Mindstorm robots in university. Also with the goal to go through a simple maze and follow a line. Booring simple everyone thought. But the thirst thing we learned, was to not trust our sensors. My expectation was, if the ultrasonic sensor said, there is 1 m to an obstacle, then there is 1 m of free space. Well, not really. Partly because the sensors were really bad and only worked reliable when the obstacle was in a 90 degree angle, partly it is in the nature of sensors to not be perfect. I am still excited for robots though, but haven't worked on one in quite a while.

bitwize 3y ago

s/roboticist/programmer/g and you get an infinite bullshit generator for the world of business and enterprise software, without even firing up ChatGPT! Having worked in both industries, I concur that robotics is much, much messier, as the system has to engage via hardware with the super-messy physical world as opposed to the comparatively modestly messy world of business transactions, data analysis, or whatever. But if we stop trying to solve for "programming for nonprogrammers" and assume that anyone who uses a language or API is a programmer (because once you start programming, that's what you become, irrespective of what's in your job title), we can remove a whole lot of wasted effort from the industry.

a_t48 3y ago

I work as a non-roboticist at a robot company. Most of my job is to enable people with PhDs to do work and to clean up after people who hastily stood up some sort of infrastructure (either actual infra, tooling, or libraries) so that they could go work on the thing they were actually interested in. Occasionally I'll get to work on vaguely actual robot things - drivers, communications frameworks, timing, etc.

creesch 3y ago

> That would be great, right? We should make a software framework so that non-roboticists can program robots. Lol, I work in the field of test automation and this is exactly how no/low code frameworks get pushed as well. And, it rarely does play out in a way that people think it will. In fact, having read the entire article, I feel like a lot of it can be applied more broadly. Basically any time people go "X sure is complex, we should make a simple to use framework for non-X folks to use". Not that it will always fail, but I have seen it happen enough to recognize a pattern.

deleted 3y ago
[deleted]
hitchstory 3y ago
BTW, odd question - assume somebody built a low/less code test automation framework that didn't fail and was very effective but it also wasn't popular. What could it do to stand out to you? What could it demonstrate in 15-20 seconds for you to think "OK this is different"?
- creesch 3y ago
  1. Come without vendor lock in. 2. Format is human readable so it can actually be integrated in a proper development cycle (git, etc). 3. Doesn't make it a nightmare to do custom things where needed. It is not impossible. Out the top of my head Robot Framework does fit those criteria. But I'd argue that Robot Framework isn't really low code, but rather a coded framework in a low code trench coat.
  - hitchstory 3y ago
    Thanks, that's very helpful. 1 and 2 make perfect sense and are easy do demonstrate but 3 seems to me to be incredibly difficult. I haven't found an easy way to advertise convincingly to somebody who (quite reasonably) grants you a limited amount of attention that custom things won't be a nightmare. It's the kind of thing you only tend see when you get dug in the weeds and hence people will tend to make assumptions based upon surface details. This is a problem I'm struggling with. I think robot/cucumber could require less code if they were better abstractions (and would be more loved), but I find it hard to illustrate that an abstraction is going to be good or bad, particularly to people with limited attention and particularly to people who don't necessarily have the skills to recognize a good abstraction.

fedeb95 3y ago

This is relevant to many domains, with different degrees of relevance.

deleted 3y ago

[deleted]

jjk166 3y ago

I think the real takeaway from this article, which is applicable to pretty widely applicable, is that a lot of times when you get the requirement "make it simple" the actual requirement is "make it intuitive." Taking away functionality doesn't typically make things any more intuitive, indeed they often make things much less intuitive because a lot of things need to be coupled that a user would not naturally expect to be. Conversely giving the user tons of options but making sure they are distinct, clearly named, their interfaces are consistent, and the defaults are sensible allows someone who understands the problem they want to use the api to solve to jump right in.

markisus 3y ago

This quote hits close to home. “So I got the arm to match the conveyor speed by monitoring the position and pre-empting the motion command, alternating ‘slow’ and ‘fast’ at 10Hz with a duty cycle depending on how we are tracking our target. The motion is pretty jerky but it works.” I’ve been through exactly this scenario in two very popular robotics frameworks. Countless hours were spent by well intentioned framework developers to abstract away the underlying PID loop from the end users. Then countless more hours were spent by the end users to work around this by implementing a PID loop on top of the abstraction.