5 ms·

What are the use-cases for a GPU-accelerated single board computer?

zucker42 6y ago
One commonly pitched use case is real time neural network calculations on mobile robots.
punnerud 6y ago
Better to understand this as; What is the use-case of RPi combined with machine learning? More specific; What is the use case of a battery enabled robot “brain” with machine learning?
NikolaeVarius 6y ago
Self Driving RC Car Racing
dragontamer 6y ago
Apparently there's a big Software Defined Radio (SDR) community that appreciates the TFlops of floating point compute. cuSignal in particular. Note that embedded radios / modems / codec manipulation is incredibly common. Usually its handled by a DSP (VLIW architecture), but I'd imagine that the GPU architecture has more FLOPs. DSPs probably have less latency, but something like SDR is not very latency sensitive (or more like: the community models latency as a phase-shift and is therefore resilient to many forms of latency) Note: I'm not actually in any of these communities. I'm just curious and someone else told me about this very same question.
- structural 6y ago
  It looks like at least one company has started to produce products that tie together the radio <-> FPGA <-> GPU compute pipeline for SDR using cuSignal and these Jetson modules: https://www.crowdsupply.com/deepwave-digital/air-t https://www.crowdsupply.com/deepwave-digital/air-t It's pretty neat stuff to think about DSP on hundreds of MHz of bandwidth being something that you can do in software these days. I remember when that was firmly in the realm of "don't bother unless you want to design a custom ASIC" and now it's starting to become even hobbyist-accessible.
  - dragontamer 6y ago
    Heck, just being able to find an ADC with hundreds of MHz of bandwidth is kind of a big deal, let alone the computational resources now available to the modern hobbyist. The modern day has brought incredible advancements to the modern hobbyist. Just 10 years ago, you probably still needed to play with OpAmps and Inductors to effectively read a hundreds-of-MHz signal in any real capacity (at the hobbyist scale at least: $1000 or less projects). A PC probably had the compute girth back then to interpret the data, but you still needed a high-speed, and high-quality, ADC to even read the data that quickly.
detaro 6y ago
Fancy 3D visualizations on screens of embedded devices. Image processing on camera feeds (both "traditional" and neural-network-based). Video encoding/transcoding. Complex UIs in high resolutions. (strictly speaking any screen with higher resolution and/or expectations of smooth non-trivial graphics requires some kind of GPU, but I take the question to mean why one would want a powerful one)
jqpabc123 6y ago
As usual, the hardware is way ahead of software.
- nl 6y ago
  This isn't true. I personally can use this now, and I now 5 or 6 people who have been trying to do stuff using the Intel USB accelerator or Google Coral who I expect will switch to this at this price point. Only the MAIX platform has a price advantage over this now, and the software is much less mature.
  - craftinator 6y ago
    The use case between K210 and Jetson systems is quite different. I've used the Maix Dock as a smart controller for a custom CNC system, and it performs beautifully. The Jetson line is more suited to systems that are video heavy (low latency, multiple feeds, mixing, processing, etc). The Maix boards just don't have the RAM to deal with that; I've only been able to handle two incoming video streams at low resolution. That being said, programming the Maix is a piece of cake. The Jetson Nano was very challenging to deal with; the software is unstable in many cases, and the levels of abstraction are all mixed up, and discovering which Nvidia tools to use is an absolute pain (Gstreamer vs DeepStream vs ARGUS vs whatever else Nvidia pumps out this year). Also, I had way better support from the Maix community than from Nvidia. Nvidia's support model seems to be "unless you're spending tens of thousands of dollars, go pound sand, maybe a volunteer on our forums will help you, if anyone knows what's actually going on here." Nvidia just has too much software, too little documentation, and holds onto their secret sauce so tightly that doing anything is a true pain.
    - nl 6y ago
      I haven't had any trouble at all with the NVidia stack. But I do machine learning full time for my day job, so I'm reasonably familiar with it. OTOH, the MAIX has things like the pin assignments wrong[1] and you need a magic (undocumented) flag to make the LCD display colors correctly. There's also different types of models supported with undocumented constraints on each one, and you need to find the correct set of random open source projects to make them work. But it is an exciting platform! [1] https://github.com/sipeed/MaixPy/issues/298 https://github.com/sipeed/MaixPy/issues/298
nl 6y ago
Things like video analysis for security cameras for people who don't want to stream everything to the cloud. It's possible on a single board computer CPU but only with lots of compromises - you can't do multiple streams etc.
- CyberDildonics 6y ago
  Why wouldn't someone just use a PC for that since it doesn't need to be portable?
  - nl 6y ago
    Power usage, fanless, size and price are the usual reasons. If you price a silent PC with a fanless NVidia card it's quite a lot more.
  - antasvara 6y ago
    I'd assume because of cost. This device does exactly what's required with low power draw and is easily replaced if it breaks or more processing power is required.
  - gnarbarian 6y ago
    Convenience. If you have a security camera up 50 ft on a tower 40 miles away connected to the internet via cell modem where do you put the PC? And is the PC able to fit in the power envelope and put up with environmental factors? An sbc is much easier to harden environmentally
paulsutter 6y ago
Local processing and video compression for a raw video camera (especially in autonomy applications). The compression in standard IP cameras introduce not only compression artifacts but also latency and jitter. And system scales better with one low cost GPU right at the camera vs trying to process for many cameras at one server.
ragebol 6y ago
Getting more CUDA buy-in? But more seriously: robotics.
fulafel 6y ago
The RPi was originally a tiny arm bolted to a decent mobile GPU :)
ArtWomb 6y ago
HDMI out and boots Linux, so could function as desktop replacement. The demos I've seen it really shines in small robotics. Think AWS Deep Racer League style competitions for K-12 ;)
- wolrah 6y ago
  > HDMI out and boots Linux, so could function as desktop replacement. As someone who has tried to run a desktop environment on various Pi-like devices for years, with every generation being "the one that will replace your PC" I just laugh. The modern web is nearly unusable with less than 4 GB of RAM and it's really easy to push beyond that. I personally would not recommend that anyone try to use anything with less than 8 GB of RAM as a general purpose desktop computer anymore. You can do it, sure, but there will be significant compromises and limitations along the way. A secondhand x86 PC with 8+ GB of RAM is almost always a better choice for general purpose desktop use. Leave the "hacker boards" for roles that benefit more from their compact size, low power consumption, and/or specialty I/O capabilities. I haven't tried the Pi 8GB, that should at least solve the RAM problem, but the lack of native high-speed storage still likely impacts usability. Here's hoping the major Pi-likes follow suit to offer variants with more memory soon. --- And of course as noted by other comments, another major reason to think of these things more as appliances is the generally spotty support for updating kernels and the like that is unfortunately standard in the ARM world. That entire side of the industry is infected with NDA culture and has no interest in curing it.
Zenst 6y ago
Good retro console emulation like the N64 alone may make this a go to for many.
- colejohnson66 6y ago
  For the price, you might as well do a Mister FPGA build using the DE10-Nano. That’s ~$140 versus the $200 for the Jetson Nano here. And the Mister FPGA, being FPGA based, can avoid most slowdowns software based emulators would have.
  - monocasa 6y ago
    Is there an N64 implementation written in an HDL? cen64 is at least written in an RTL style, but it's not quite there.
  - jamesgeck0 6y ago
    The Jetson Nano is less than half the price of the Mister FPGA.
  - postalrat 6y ago
    Add on sram, expansion board, usb hub, and it gets to be a bit more than $140. I'm hoping someday we can get something like mister on the pano logic boards. You can find 2nd gen ones on ebay for $15 or so and have usb, dvi, audio, and I believe a more powerful fpga. All in a solid metal case.
catsdanxe 6y ago
I want to strap it to a drone and fly around with it.
- riotnrrd 6y ago
  Autonomous drone navigation (forest trail following) using a TX2: https://github.com/NVIDIA-AI-IOT/redtail https://github.com/NVIDIA-AI-IOT/redtail
CydeWeys 6y ago
Same as it's always been: Gaming. I bet even this $59 computer can play games surprisingly well. Obviously not modern AAA titles or anything like that, but there's many decades' worth of games it can play (including ones originally released for non-PC platforms). There's lot of interesting Pi-powered handhelds and mini-arcade setups running emulated games; this seems like it'd be significantly better at that.
- Matsta 6y ago
  ETA Prime already did a emulation, seems to have enough grunt to run some GameCube/Wii games: https://www.youtube.com/watch?v=5xrRxz5633I https://www.youtube.com/watch?v=5xrRxz5633I Compared to him testing on a Raspberry Pi 4: https://www.youtube.com/watch?v=l4TyYU9Xhcs https://www.youtube.com/watch?v=l4TyYU9Xhcs (note both those videos are a old so results could be different now) The video is a year old, but I imagine there would be less bugs now. With this cheaper board coming out, there might be a bit more of development activity.
- shiftpgdn 6y ago
  There is a fairly decent subreddit around single board computer gaming: https://www.reddit.com/r/SBCGaming/ https://www.reddit.com/r/SBCGaming/ .
- thealienthing 6y ago
  Gaming is not the purpose of GPU accelerated single board computers. Though you could reasonably play some games on the Jetson Nano, Nvidia markets the jetson series as embeddable machine learning systems. While there may be complaints of how hot the jetson gets or its performance with Ubuntu, the jetson is not intended to be used as a personal computer. It’s cheaper than dirt machine learning access. I highly recommend it as a way to get started in AI or CUDA with the smallest price barrier possible.
  - CydeWeys 6y ago
    I said it was a use case, not the purpose. But realistically these single board computers are more suitable for gaming than for machine learning. If you don't have a computer at all then I guess you could use a Jetson for this purpose, but a real computer would be much better and is easily within the budgets of most companies and most software engineers. Is this a developing markets play? I've never heard of someone whose only computer is a Pi/Jetson. By the time you add all the necessary peripherals to use it as your only computer, it costs more than a cheap laptop. So the point of a Jetson is .. you only have a cheap laptop with terrible integrated graphics, and for $60 you get a much better GPU with this Jetson, and you remote into it? I think it'll do better in gaming.
gnarbarian 6y ago
Smart security cameras that can perform complex operations at the edge. Like real-time video stitching, object recognition etc.
FrojoS 6y ago
Anything that is not mass produced, needs to have a small size or weight and needs 3D graphics or computer vision. For instance, imagine a bike rental startup that wants to do traffic sign recognition integrated in their vehicles.
fulafel 6y ago
Does this run a stock distribution and a distro provided stock kernel such as Ubuntu? If not, how good are the security updates and for how long does NVidia commit to providing them?

One commonly pitched use case is real time neural network calculations on mobile robots.

Better to understand this as; What is the use-case of RPi combined with machine learning? More specific; What is the use case of a battery enabled robot “brain” with machine learning?

NikolaeVarius 6y ago

Self Driving RC Car Racing

dragontamer 6y ago

Apparently there's a big Software Defined Radio (SDR) community that appreciates the TFlops of floating point compute. cuSignal in particular. Note that embedded radios / modems / codec manipulation is incredibly common. Usually its handled by a DSP (VLIW architecture), but I'd imagine that the GPU architecture has more FLOPs. DSPs probably have less latency, but something like SDR is not very latency sensitive (or more like: the community models latency as a phase-shift and is therefore resilient to many forms of latency) Note: I'm not actually in any of these communities. I'm just curious and someone else told me about this very same question.

structural 6y ago
It looks like at least one company has started to produce products that tie together the radio <-> FPGA <-> GPU compute pipeline for SDR using cuSignal and these Jetson modules: https://www.crowdsupply.com/deepwave-digital/air-t https://www.crowdsupply.com/deepwave-digital/air-t It's pretty neat stuff to think about DSP on hundreds of MHz of bandwidth being something that you can do in software these days. I remember when that was firmly in the realm of "don't bother unless you want to design a custom ASIC" and now it's starting to become even hobbyist-accessible.
- dragontamer 6y ago
  Heck, just being able to find an ADC with hundreds of MHz of bandwidth is kind of a big deal, let alone the computational resources now available to the modern hobbyist. The modern day has brought incredible advancements to the modern hobbyist. Just 10 years ago, you probably still needed to play with OpAmps and Inductors to effectively read a hundreds-of-MHz signal in any real capacity (at the hobbyist scale at least: $1000 or less projects). A PC probably had the compute girth back then to interpret the data, but you still needed a high-speed, and high-quality, ADC to even read the data that quickly.

detaro 6y ago

Fancy 3D visualizations on screens of embedded devices. Image processing on camera feeds (both "traditional" and neural-network-based). Video encoding/transcoding. Complex UIs in high resolutions. (strictly speaking any screen with higher resolution and/or expectations of smooth non-trivial graphics requires some kind of GPU, but I take the question to mean why one would want a powerful one)

jqpabc123 6y ago

As usual, the hardware is way ahead of software.

nl 6y ago
This isn't true. I personally can use this now, and I now 5 or 6 people who have been trying to do stuff using the Intel USB accelerator or Google Coral who I expect will switch to this at this price point. Only the MAIX platform has a price advantage over this now, and the software is much less mature.
- craftinator 6y ago
  The use case between K210 and Jetson systems is quite different. I've used the Maix Dock as a smart controller for a custom CNC system, and it performs beautifully. The Jetson line is more suited to systems that are video heavy (low latency, multiple feeds, mixing, processing, etc). The Maix boards just don't have the RAM to deal with that; I've only been able to handle two incoming video streams at low resolution. That being said, programming the Maix is a piece of cake. The Jetson Nano was very challenging to deal with; the software is unstable in many cases, and the levels of abstraction are all mixed up, and discovering which Nvidia tools to use is an absolute pain (Gstreamer vs DeepStream vs ARGUS vs whatever else Nvidia pumps out this year). Also, I had way better support from the Maix community than from Nvidia. Nvidia's support model seems to be "unless you're spending tens of thousands of dollars, go pound sand, maybe a volunteer on our forums will help you, if anyone knows what's actually going on here." Nvidia just has too much software, too little documentation, and holds onto their secret sauce so tightly that doing anything is a true pain.
  - nl 6y ago
    I haven't had any trouble at all with the NVidia stack. But I do machine learning full time for my day job, so I'm reasonably familiar with it. OTOH, the MAIX has things like the pin assignments wrong[1] and you need a magic (undocumented) flag to make the LCD display colors correctly. There's also different types of models supported with undocumented constraints on each one, and you need to find the correct set of random open source projects to make them work. But it is an exciting platform! [1] https://github.com/sipeed/MaixPy/issues/298 https://github.com/sipeed/MaixPy/issues/298

nl 6y ago

Things like video analysis for security cameras for people who don't want to stream everything to the cloud. It's possible on a single board computer CPU but only with lots of compromises - you can't do multiple streams etc.

CyberDildonics 6y ago
Why wouldn't someone just use a PC for that since it doesn't need to be portable?
- nl 6y ago
  Power usage, fanless, size and price are the usual reasons. If you price a silent PC with a fanless NVidia card it's quite a lot more.
- antasvara 6y ago
  I'd assume because of cost. This device does exactly what's required with low power draw and is easily replaced if it breaks or more processing power is required.
- gnarbarian 6y ago
  Convenience. If you have a security camera up 50 ft on a tower 40 miles away connected to the internet via cell modem where do you put the PC? And is the PC able to fit in the power envelope and put up with environmental factors? An sbc is much easier to harden environmentally

paulsutter 6y ago

Local processing and video compression for a raw video camera (especially in autonomy applications). The compression in standard IP cameras introduce not only compression artifacts but also latency and jitter. And system scales better with one low cost GPU right at the camera vs trying to process for many cameras at one server.

ragebol 6y ago

Getting more CUDA buy-in? But more seriously: robotics.

fulafel 6y ago

The RPi was originally a tiny arm bolted to a decent mobile GPU :)

ArtWomb 6y ago

HDMI out and boots Linux, so could function as desktop replacement. The demos I've seen it really shines in small robotics. Think AWS Deep Racer League style competitions for K-12 ;)

wolrah 6y ago
> HDMI out and boots Linux, so could function as desktop replacement. As someone who has tried to run a desktop environment on various Pi-like devices for years, with every generation being "the one that will replace your PC" I just laugh. The modern web is nearly unusable with less than 4 GB of RAM and it's really easy to push beyond that. I personally would not recommend that anyone try to use anything with less than 8 GB of RAM as a general purpose desktop computer anymore. You can do it, sure, but there will be significant compromises and limitations along the way. A secondhand x86 PC with 8+ GB of RAM is almost always a better choice for general purpose desktop use. Leave the "hacker boards" for roles that benefit more from their compact size, low power consumption, and/or specialty I/O capabilities. I haven't tried the Pi 8GB, that should at least solve the RAM problem, but the lack of native high-speed storage still likely impacts usability. Here's hoping the major Pi-likes follow suit to offer variants with more memory soon. --- And of course as noted by other comments, another major reason to think of these things more as appliances is the generally spotty support for updating kernels and the like that is unfortunately standard in the ARM world. That entire side of the industry is infected with NDA culture and has no interest in curing it.

Zenst 6y ago

Good retro console emulation like the N64 alone may make this a go to for many.

colejohnson66 6y ago
For the price, you might as well do a Mister FPGA build using the DE10-Nano. That’s ~$140 versus the $200 for the Jetson Nano here. And the Mister FPGA, being FPGA based, can avoid most slowdowns software based emulators would have.
- monocasa 6y ago
  Is there an N64 implementation written in an HDL? cen64 is at least written in an RTL style, but it's not quite there.
- jamesgeck0 6y ago
  The Jetson Nano is less than half the price of the Mister FPGA.
- postalrat 6y ago
  Add on sram, expansion board, usb hub, and it gets to be a bit more than $140. I'm hoping someday we can get something like mister on the pano logic boards. You can find 2nd gen ones on ebay for $15 or so and have usb, dvi, audio, and I believe a more powerful fpga. All in a solid metal case.

catsdanxe 6y ago

I want to strap it to a drone and fly around with it.

riotnrrd 6y ago
Autonomous drone navigation (forest trail following) using a TX2: https://github.com/NVIDIA-AI-IOT/redtail https://github.com/NVIDIA-AI-IOT/redtail

CydeWeys 6y ago

Same as it's always been: Gaming. I bet even this $59 computer can play games surprisingly well. Obviously not modern AAA titles or anything like that, but there's many decades' worth of games it can play (including ones originally released for non-PC platforms). There's lot of interesting Pi-powered handhelds and mini-arcade setups running emulated games; this seems like it'd be significantly better at that.

Matsta 6y ago
ETA Prime already did a emulation, seems to have enough grunt to run some GameCube/Wii games: https://www.youtube.com/watch?v=5xrRxz5633I https://www.youtube.com/watch?v=5xrRxz5633I Compared to him testing on a Raspberry Pi 4: https://www.youtube.com/watch?v=l4TyYU9Xhcs https://www.youtube.com/watch?v=l4TyYU9Xhcs (note both those videos are a old so results could be different now) The video is a year old, but I imagine there would be less bugs now. With this cheaper board coming out, there might be a bit more of development activity.
shiftpgdn 6y ago
There is a fairly decent subreddit around single board computer gaming: https://www.reddit.com/r/SBCGaming/ https://www.reddit.com/r/SBCGaming/ .
thealienthing 6y ago
Gaming is not the purpose of GPU accelerated single board computers. Though you could reasonably play some games on the Jetson Nano, Nvidia markets the jetson series as embeddable machine learning systems. While there may be complaints of how hot the jetson gets or its performance with Ubuntu, the jetson is not intended to be used as a personal computer. It’s cheaper than dirt machine learning access. I highly recommend it as a way to get started in AI or CUDA with the smallest price barrier possible.
- CydeWeys 6y ago
  I said it was a use case, not the purpose. But realistically these single board computers are more suitable for gaming than for machine learning. If you don't have a computer at all then I guess you could use a Jetson for this purpose, but a real computer would be much better and is easily within the budgets of most companies and most software engineers. Is this a developing markets play? I've never heard of someone whose only computer is a Pi/Jetson. By the time you add all the necessary peripherals to use it as your only computer, it costs more than a cheap laptop. So the point of a Jetson is .. you only have a cheap laptop with terrible integrated graphics, and for $60 you get a much better GPU with this Jetson, and you remote into it? I think it'll do better in gaming.

gnarbarian 6y ago

Smart security cameras that can perform complex operations at the edge. Like real-time video stitching, object recognition etc.

FrojoS 6y ago

Anything that is not mass produced, needs to have a small size or weight and needs 3D graphics or computer vision. For instance, imagine a bike rental startup that wants to do traffic sign recognition integrated in their vehicles.

fulafel 6y ago

Does this run a stock distribution and a distro provided stock kernel such as Ubuntu? If not, how good are the security updates and for how long does NVidia commit to providing them?