6 ms·

What is the new breakthrough in robotics that is gpu driven ? There are subsets of the overall problem that can be solved by a gpu (eg object detection) but the

What is the new breakthrough in robotics that is gpu driven ? There are subsets of the overall problem that can be solved by a gpu (eg object detection) but the whole planning and control algo scheme seems to be more or less the same as it has been for the past decades. These typically involve non-convex optimization so not much gpu benefit.

pseudosudoer 2y ago
There are search spaces that are quite large that are used in optimal control. GPUs can be used to drastically accelerate finding a solution. As an example, imagine you are given a height map, a 2D discrete search space overlayed in the height map, 4 legs, and robot dynamics for every configuration of the legs in their constrained workspace. Find the optimal toe placement of the 4 legs. Although a GPU isn't designed exactly to deal with this sort of problem, if it's framed as a reduction problem it still significantly out performs a multi core CPU.
- whatever1 2y ago
  These are sparse systems, factorization is not a strength of the gpu architectures. Typically adding more cpu cores is a better investment rather than trying to parallelize it through gpu. Nvidia has been trying for some time to make progress with cuSparse etc, although not much has been achieved in the space. Maybe they try a completely different approach with reinforcemnt learning and a ton of parallel simulations?
  - deleted 2y ago
    [deleted]
mattlondon 2y ago
> What is the new breakthrough in robotics that is gpu driven ? There are subsets of the overall problem that can be solved by a gpu (eg object detection) but the whole planning and control algo scheme seems to be more or less the same as it has been for the past decades I think the "object detection" goes quite far beyond the classic "objection detection" bounding boxes etc we're used to seeing. So not just a pair of x,y coords for the bounding box for e.g. a mug of coffee in the robot's field of view, but what is the orientation of the mug? where is the handle? If the handle is obscured, can we infer where it might be based on what we understand for what a mug typically looks like and plan our gripper motion towards it (and at 120hz etc)? Is it a solid mug, or a paper cup (affects grip strength/pressure)? Etc etc. Then there is the whole thing about visually show the robot once what you are doing, and it automatically "programs" itself to repeat the tasks in a generalised way etc. Then you could probably spawn 100 startups just on hooking up a LLM to tell a robot what to do in a residential setting (make me a coffee, clear up the kitchen, take out the trash etc) This has all been possible before of course, but could it be done "on device" in a power efficient way? I am guessing they are hoping to sell a billion or two chips + boards to be built directly into things to do so so that your next robotic vacuum or lawn mower or whatever will be able to respond to you yelling at it and not mangle your pets/small children in the process. I eagerly await the day when I have a plug and play robot platform that can tell the difference between my young children and a fox, and attack the fox shitting/shredding something small and fluffy in the garden but ignore the kids
iancmceachern 2y ago
It's the edge computing. Similar to autonomous vehicles, doing complex multi sensor things very quickly. Surgical robotics is a great example, lots of cool use cases coming out in that field.
XenophileJKO 2y ago
I think it is what nobody has answered yet.. virtualized training/testing. I watched a presentation by their research team. This is a HUGE force multiplier. Don't underestimate how much this changes robotic foundational model training.
krasin 2y ago
> What is the new breakthrough in robotics that is gpu driven ? There are subsets of the overall problem that can be solved by a gpu (eg object detection) but the whole planning and control algo scheme seems to be more or less the same as it has been for the past decades. These typically involve non-convex optimization so not much gpu benefit. In the past two years two very important developments appeared around imitation learning and LLMs. Some starting points for this rabbit hole: 1. HuggingFace LeRobot: https://github.com/huggingface/lerobot https://github.com/huggingface/lerobot 2. ALOHA: https://aloha-2.github.io/ https://aloha-2.github.io/ 3. https://robotics-transformer2.github.io/ https://robotics-transformer2.github.io/ 4. https://www.1x.tech/discover/1x-world-model https://www.1x.tech/discover/1x-world-model
- YeGoblynQueenne 2y ago
  We've been here many times before. Imitation learning doesn't generalise and that makes it useless in practice. Aloha is a great example of that. It's great for demos, like the one where their robot "cooked" (not really) one shrimp, but if you wanted to deploy it to real peoples' houses you'd have to train it for every task in every house over a few hours at a time. And "a task" is still at the level of "cook (not really) one shrimp". You want to cook (not really) noodles? It's a new task and you have to train it all over again from scratch. You want it to fold your laundry? OK but you need to train it on each piece of laundry you want it to fold, separately. You want it to put away the dishes? Without exaggeration you'd have to train it to handle each dish separately. You want it to pick up the dishes from the kitchen? Train for that. You want it to pick up the dishes from the living room? Train for that. And so on. It sucks so much with miserable disappointment that it could bring on a new AI winter on its own, if Google was dumb enough to try and make it into a product and market it to people. Robot maids and robot butlers are a long way away. Yeah but you can cook one shrimp (not really) with a few hours of teleoperation training in your kitchen only. Oh wow. We could never cook (not really) one shrimp before. I mean we could but this uses RL and so it's just one step from AGI. It's nonsense on stilts.
  - krasin 2y ago
    I generally agree with your analysis of the current state of art but strongly disagree with the overall conclusion of where it leads us. I believe it will take on the order of 100M hours of training data of doing tasks in real world (so, not just Youtube videos), and much larger models than we have now to make general-purpose robotics working, but I also believe that this will happen. I've saved your comment to my favorites and hope to revisit it in 10 years.
michaelt 2y ago
Two decades ago, I was trying to use classical machine vision to tell the difference between cut and uncut grass, to guide a self-driving lawnmower. I concluded that it couldn't be done with classical machine vision, and that this "neural network" nonsense wasn't going to catch on. Very slow, computationally inefficient, full of weirdos making grandiose claims about "artificial intelligence" without the results to back it up, and they couldn't even explain how their own stuff worked. These days - you want to find the boundary between cut and uncut grass, even though lighting levels can change and cloud cover can change and shadows can change and reflections can change and there's loads of types of grass and grass looks different depending on the angle you look from? Just label some data and chuck a neural network at it, no problemo.
- justmarc 2y ago
  Funnily now with with the advent of GPS+RTK lawnmower robots, fancy AI is not even needed anymore. They follow a very exact, pre-determined patterns and paths, and do a great job.
  - michaelt 2y ago
    Yeah GPS+RTK was what I went with in the end. Didn't work as well as I'd hoped back in those days though, as you could lose carrier lock if you got too close to trees (or indeed buildings), and our target market was golf courses which tend to have a lot of trees. And in those days a dual-frequency RTK+IMU setup was $20k or more, which is expensive for a lawnmower.
    - justmarc 2y ago
      No tool is perfect for every job. That said, the positioning of the RTK unit is crucial. Possibly look for a mower which can work with multiple RTK units, or reposition your existing one for better coverage. I find that even though signals get significantly weaker under trees, mine still works wonderfully in a complex large garden scenario. It will depend on your exact unit/model, as well as their firmware and how it chooses to deal with these scenarios.
- 2y ago
amelius 2y ago
The fun thing with DL is that you don't have to optimize stuff with complicated math. You just train it, and it will generate solutions. Maybe not the perfect solutions, but don't let perfect be the enemy of good.
imtringued 2y ago
Actually, it is quadratic programming that is big in robotics. QP is powerful enough that you can formulate your task, but also fast enough that you can run it in the control loop in real time.
emn13 2y ago
The article lists 2: firstly, simply that ML models are now feasible at a scale they weren't only a few years ago. Secondly, compute power is now better enough that it can now simulate more realistic environments which enables sim-based (pre)training to work better. That second one is potentially particularly alluring to nvidia given how it plays on two of their unique strengths - AI and graphics.
m_ke 2y ago
VLA - Vision Language Action models https://arxiv.org/abs/2406.09246 https://arxiv.org/abs/2406.09246 It turns out you can take a vision language foundational model that has a broad understanding of visual and textual knowledge and fine tune it to output robot actions given a sequence of images and previous actions. This approach beats all previous methods by a wide margin and transfers across tasks.
dartos 2y ago
Unstructured Sensory input driven by these large neural networks, if I had to guess. To be able to visually determine weight, texture, and how durable something is can be done with those systems so long as we have a training set.
mmmore 2y ago
My understanding is that it's in vogue to use deep learning for complex control problems, and the results are fairly impressive. The idea is to train robotic motion end to end with RL. Not an expert so I don't know the strength and weaknesses versus classical approaches. https://blogs.nvidia.com/blog/eureka-robotics-research/ https://blogs.nvidia.com/blog/eureka-robotics-research/ https://arxiv.org/abs/2108.10470 https://arxiv.org/abs/2108.10470
jvanderbot 2y ago
I've been doing this for 10+ years, and have seen GPU-based calculations slowly eat away at the following problems: * Mapping. Nowadays generating a dense grid of costs can be done insanely fast on GPU. There's just no excuse to not use a GPU on every robot so it can build a fast map, unless you move at snail speed. * Computer Vision. Classical depth mapping is best done on a GPU. Classical computer vision object detection has fallen away to the rise in ML-based CV for segmentation. Some (IMHO) overzealous practitioners are trying to eat away at estimation and tracking, which IMHO will recede a little since there was nothing wrong with the estimators (just Bayesian stats) to begin with, it was always the measurements. Still, for detection (and sometimes association), ML on GPU is the way to go and that will very likely not change. It has gotten so good that you can get away without using other sensors and just deploying a vision system (though I don't recommend it, but this is what Tesla does). This is an obvious case for one (or one more) GPU on every robot. * Planning - End to end planning is eating traditional planning now, similar to CV. There are some areas where this is an obvious win (e.g., complex manipulation tasks), and some areas where some overzealous overreach is happening (e.g., simpler planning tasks like routing). But ML on GPUs is here to stay for all planning tasks, especially when estimating costs from complex data, even if a classical planner uses those costs. And I'd be remiss if I didn't mention policy-based planning, which does a huge amount of training to generate essentially a fast lookup table for actions. Deployment of these types of planners often requires a very good estimator to determine what state you are in - and this is a great area for ML, mapping real world messy data to a clean state lookup. I think this can typically be done without a GPU, due to training prior to deployment, but if you have a GPU already (see prior two), you will find this is a good use of it. * Low-level planning / Controls - Shares a small overlap with above, but mostly concerned with fast responses to transient data and stabilizing the system. I've heard, but not seen directly, that learned policies are coming into vogue here. But regardless, it is a common thread that a network can assist with estimating costs and states to allow a traditional controls system to operate more reliably. I doubt this will necessitate a GPU, but like above, will gladly use it if required and available. To add to this, consider that we're generally not talking about discrete, gaming-type GPUs, we're talking about purpose built robotics-targeted embedded systems that speak native CUDA. The Jetson family, in particular.

pseudosudoer 2y ago

There are search spaces that are quite large that are used in optimal control. GPUs can be used to drastically accelerate finding a solution. As an example, imagine you are given a height map, a 2D discrete search space overlayed in the height map, 4 legs, and robot dynamics for every configuration of the legs in their constrained workspace. Find the optimal toe placement of the 4 legs. Although a GPU isn't designed exactly to deal with this sort of problem, if it's framed as a reduction problem it still significantly out performs a multi core CPU.

whatever1 2y ago
These are sparse systems, factorization is not a strength of the gpu architectures. Typically adding more cpu cores is a better investment rather than trying to parallelize it through gpu. Nvidia has been trying for some time to make progress with cuSparse etc, although not much has been achieved in the space. Maybe they try a completely different approach with reinforcemnt learning and a ton of parallel simulations?
- deleted 2y ago
  [deleted]

mattlondon 2y ago

> What is the new breakthrough in robotics that is gpu driven ? There are subsets of the overall problem that can be solved by a gpu (eg object detection) but the whole planning and control algo scheme seems to be more or less the same as it has been for the past decades I think the "object detection" goes quite far beyond the classic "objection detection" bounding boxes etc we're used to seeing. So not just a pair of x,y coords for the bounding box for e.g. a mug of coffee in the robot's field of view, but what is the orientation of the mug? where is the handle? If the handle is obscured, can we infer where it might be based on what we understand for what a mug typically looks like and plan our gripper motion towards it (and at 120hz etc)? Is it a solid mug, or a paper cup (affects grip strength/pressure)? Etc etc. Then there is the whole thing about visually show the robot once what you are doing, and it automatically "programs" itself to repeat the tasks in a generalised way etc. Then you could probably spawn 100 startups just on hooking up a LLM to tell a robot what to do in a residential setting (make me a coffee, clear up the kitchen, take out the trash etc) This has all been possible before of course, but could it be done "on device" in a power efficient way? I am guessing they are hoping to sell a billion or two chips + boards to be built directly into things to do so so that your next robotic vacuum or lawn mower or whatever will be able to respond to you yelling at it and not mangle your pets/small children in the process. I eagerly await the day when I have a plug and play robot platform that can tell the difference between my young children and a fox, and attack the fox shitting/shredding something small and fluffy in the garden but ignore the kids

iancmceachern 2y ago

It's the edge computing. Similar to autonomous vehicles, doing complex multi sensor things very quickly. Surgical robotics is a great example, lots of cool use cases coming out in that field.

XenophileJKO 2y ago

I think it is what nobody has answered yet.. virtualized training/testing. I watched a presentation by their research team. This is a HUGE force multiplier. Don't underestimate how much this changes robotic foundational model training.

krasin 2y ago

YeGoblynQueenne 2y ago
We've been here many times before. Imitation learning doesn't generalise and that makes it useless in practice. Aloha is a great example of that. It's great for demos, like the one where their robot "cooked" (not really) one shrimp, but if you wanted to deploy it to real peoples' houses you'd have to train it for every task in every house over a few hours at a time. And "a task" is still at the level of "cook (not really) one shrimp". You want to cook (not really) noodles? It's a new task and you have to train it all over again from scratch. You want it to fold your laundry? OK but you need to train it on each piece of laundry you want it to fold, separately. You want it to put away the dishes? Without exaggeration you'd have to train it to handle each dish separately. You want it to pick up the dishes from the kitchen? Train for that. You want it to pick up the dishes from the living room? Train for that. And so on. It sucks so much with miserable disappointment that it could bring on a new AI winter on its own, if Google was dumb enough to try and make it into a product and market it to people. Robot maids and robot butlers are a long way away. Yeah but you can cook one shrimp (not really) with a few hours of teleoperation training in your kitchen only. Oh wow. We could never cook (not really) one shrimp before. I mean we could but this uses RL and so it's just one step from AGI. It's nonsense on stilts.
- krasin 2y ago
  I generally agree with your analysis of the current state of art but strongly disagree with the overall conclusion of where it leads us. I believe it will take on the order of 100M hours of training data of doing tasks in real world (so, not just Youtube videos), and much larger models than we have now to make general-purpose robotics working, but I also believe that this will happen. I've saved your comment to my favorites and hope to revisit it in 10 years.

michaelt 2y ago

Two decades ago, I was trying to use classical machine vision to tell the difference between cut and uncut grass, to guide a self-driving lawnmower. I concluded that it couldn't be done with classical machine vision, and that this "neural network" nonsense wasn't going to catch on. Very slow, computationally inefficient, full of weirdos making grandiose claims about "artificial intelligence" without the results to back it up, and they couldn't even explain how their own stuff worked. These days - you want to find the boundary between cut and uncut grass, even though lighting levels can change and cloud cover can change and shadows can change and reflections can change and there's loads of types of grass and grass looks different depending on the angle you look from? Just label some data and chuck a neural network at it, no problemo.

justmarc 2y ago
Funnily now with with the advent of GPS+RTK lawnmower robots, fancy AI is not even needed anymore. They follow a very exact, pre-determined patterns and paths, and do a great job.
- michaelt 2y ago
  Yeah GPS+RTK was what I went with in the end. Didn't work as well as I'd hoped back in those days though, as you could lose carrier lock if you got too close to trees (or indeed buildings), and our target market was golf courses which tend to have a lot of trees. And in those days a dual-frequency RTK+IMU setup was $20k or more, which is expensive for a lawnmower.
  - justmarc 2y ago
    No tool is perfect for every job. That said, the positioning of the RTK unit is crucial. Possibly look for a mower which can work with multiple RTK units, or reposition your existing one for better coverage. I find that even though signals get significantly weaker under trees, mine still works wonderfully in a complex large garden scenario. It will depend on your exact unit/model, as well as their firmware and how it chooses to deal with these scenarios.
2y ago

amelius 2y ago

The fun thing with DL is that you don't have to optimize stuff with complicated math. You just train it, and it will generate solutions. Maybe not the perfect solutions, but don't let perfect be the enemy of good.

imtringued 2y ago

Actually, it is quadratic programming that is big in robotics. QP is powerful enough that you can formulate your task, but also fast enough that you can run it in the control loop in real time.

emn13 2y ago

The article lists 2: firstly, simply that ML models are now feasible at a scale they weren't only a few years ago. Secondly, compute power is now better enough that it can now simulate more realistic environments which enables sim-based (pre)training to work better. That second one is potentially particularly alluring to nvidia given how it plays on two of their unique strengths - AI and graphics.

m_ke 2y ago

VLA - Vision Language Action models https://arxiv.org/abs/2406.09246 https://arxiv.org/abs/2406.09246 It turns out you can take a vision language foundational model that has a broad understanding of visual and textual knowledge and fine tune it to output robot actions given a sequence of images and previous actions. This approach beats all previous methods by a wide margin and transfers across tasks.

dartos 2y ago

Unstructured Sensory input driven by these large neural networks, if I had to guess. To be able to visually determine weight, texture, and how durable something is can be done with those systems so long as we have a training set.

mmmore 2y ago

My understanding is that it's in vogue to use deep learning for complex control problems, and the results are fairly impressive. The idea is to train robotic motion end to end with RL. Not an expert so I don't know the strength and weaknesses versus classical approaches. https://blogs.nvidia.com/blog/eureka-robotics-research/ https://blogs.nvidia.com/blog/eureka-robotics-research/ https://arxiv.org/abs/2108.10470 https://arxiv.org/abs/2108.10470

jvanderbot 2y ago

I've been doing this for 10+ years, and have seen GPU-based calculations slowly eat away at the following problems: * Mapping. Nowadays generating a dense grid of costs can be done insanely fast on GPU. There's just no excuse to not use a GPU on every robot so it can build a fast map, unless you move at snail speed. * Computer Vision. Classical depth mapping is best done on a GPU. Classical computer vision object detection has fallen away to the rise in ML-based CV for segmentation. Some (IMHO) overzealous practitioners are trying to eat away at estimation and tracking, which IMHO will recede a little since there was nothing wrong with the estimators (just Bayesian stats) to begin with, it was always the measurements. Still, for detection (and sometimes association), ML on GPU is the way to go and that will very likely not change. It has gotten so good that you can get away without using other sensors and just deploying a vision system (though I don't recommend it, but this is what Tesla does). This is an obvious case for one (or one more) GPU on every robot. * Planning - End to end planning is eating traditional planning now, similar to CV. There are some areas where this is an obvious win (e.g., complex manipulation tasks), and some areas where some overzealous overreach is happening (e.g., simpler planning tasks like routing). But ML on GPUs is here to stay for all planning tasks, especially when estimating costs from complex data, even if a classical planner uses those costs. And I'd be remiss if I didn't mention policy-based planning, which does a huge amount of training to generate essentially a fast lookup table for actions. Deployment of these types of planners often requires a very good estimator to determine what state you are in - and this is a great area for ML, mapping real world messy data to a clean state lookup. I think this can typically be done without a GPU, due to training prior to deployment, but if you have a GPU already (see prior two), you will find this is a good use of it. * Low-level planning / Controls - Shares a small overlap with above, but mostly concerned with fast responses to transient data and stabilizing the system. I've heard, but not seen directly, that learned policies are coming into vogue here. But regardless, it is a common thread that a network can assist with estimating costs and states to allow a traditional controls system to operate more reliably. I doubt this will necessitate a GPU, but like above, will gladly use it if required and available. To add to this, consider that we're generally not talking about discrete, gaming-type GPUs, we're talking about purpose built robotics-targeted embedded systems that speak native CUDA. The Jetson family, in particular.