5 ms·

My biggest complaint with the Jetson line is it's all ARM. Look, I get it. But the developer experience is horrible. Building Docker containers for ARM devices

My biggest complaint with the Jetson line is it's all ARM. Look, I get it. But the developer experience is horrible. Building Docker containers for ARM devices is a pain. Hell, building anything for a Jetson can be a pain unless it's a pre-packaged NVIDIA thing - really not a fan of building things from source. Add on top of that NVIDIA's very low level documentation for pretty much any tooling they ship, coupled with the difficulty in getting near-time engineering support (unless you want to post to one of their message boards, and hope you get an answer back in less than a week)... basically, it's really rough to do anything seriously useful with Jetson hardware. Second biggest complaint is deploying Jetsons in production environments. Dev kits aren't production stable, so you either need to build your own carrier board or find one pre-built, and frankly that's just a giant pain to do. Third biggest complaint is having to flash Jetsons manually. Misery. A production-ready x64 Jetson that you could order directly from NVIDIA would be my dream. Add up all of the shortcomings and overhead of ARM Jetsons and IMO you do not have a viable device for shipping AI solutions at scale.

deleted 6y ago
[deleted]
0x8BADF00D 6y ago
> Building Docker containers for ARM devices is a pain in the ass. It's not so bad, you just need a beefy ARM machine to build the containers in CI. It would be silly to build a Docker container on the Jetson itself. You would never use an embedded device for compiles and builds, why would you build Docker containers on one?
- linarism 6y ago
  Genuinely curious, what is an example of a beefy ARM machine?
  - pram 6y ago
    Anything with a ThunderX processor is maximum beefy. You can get on-demand servers like that from places like Packet. AWS also has their own A1 instances with lower core counts. These would all be good for cross compiling/builds. Comedy answer: iPad Pro
    - p1necone 6y ago
      > Comedy answer: iPad Pro I mean, the iPad Pro does have a relatively beefy processor. If only you could run arbitrary code on it.
      - BillinghamJ 6y ago
        A lot of instruction sets aren't there yet, but https://ish.app https://ish.app is doing a truly incredible job in this regard. Gives you a working Alpine Linux installation which you can download and install packages for normally, all within the bounds of the normal Apple sandbox, with decent enough performance. It doesn't have SSE or MMX yet, so eg Go and Node aren't usable at this point. But a shocking amount actually does work perfectly, so it's only a matter of time as more instruction sets are implemented.
        p1necone 6y ago
        It's pretty atrocious that you have to run an x86 emulator to use your iPad "pro" for this stuff. This project is super cool though.
        p1necone 6y ago
        Actually wouldn't a userspace ARM "emulator" be faster than an x86 one on an ARM device? Or are you so far removed from running actual CPU instructions that it doesn't matter?
        qubex 6y ago
        I suggest you also check out UTM (Universal Turing Machine), an emulator for the iPad that you can build and that allows you to emulate many different architectures and tun OSes written for them (including ARM). https://getutm.app https://getutm.app
  - panny 6y ago
    Amazon Graviton2 instances are faster (and cheaper) than x86_64 machines in the same class. https://blog.treasuredata.com/blog/2020/03/27/high-performance-sql-aws-graviton2-benchmarks-with-presto-and-arm-treasure-data-cdp/ https://blog.treasuredata.com/blog/2020/03/27/high-performan...
- king_magic 6y ago
  Spinning up an ARM machine to build a container that needs to be shipped over the network to a Jetson is a pain when developing.
  - 0x8BADF00D 6y ago
    Welcome to embedded computing :-)
    - king_magic 6y ago
      I get it, but I also simply don’t want to deal with it at an embedded level. Which is why I vote with my wallet and choose x64 hardware with Quadro GPUs to ship for production. More expensive, more power consuming? Sure. More sanity? Massively better dev experience? Massively better production/ops experience? Absolutely.
  - threeseed 6y ago
    You could spin up an AWS ARM instance in about 30s. Or just have Jenkins etc trigger the launch of one when your build job needs it.
- structural 6y ago
  You don't even need to do this: qemu support for executing arm64 binaries is fairly good, so you can just run the builds on your beefy x86 server anyways.
  - fulafel 6y ago
    What's the single thread performance difference of a fast x86 cpu running emulated ARM code vs a fast ARM server or an AWS ARM instance?
    - fluffything 6y ago
      It's quite fast if you use qemu-user.
      - fulafel 6y ago
        qemu-user just gets you native speed sustem calls right? That's normally <10% of execution time so it can't be a very big help.
- ericvh 6y ago
  You don't even need the Arm machine for CI, with docker multi-arch builds you can build on x86 -- https://community.arm.com/developer/research/b/articles/posts/continuous-cross-architecture-integration-with-gitlab https://community.arm.com/developer/research/b/articles/post...
arthurcolle 6y ago
Why is it a pain to build Docker container for the ARM architecture? Just curious, haven't had to do it myself.
- king_magic 6y ago
  If your dependencies don’t have ARM builds, you need to build them from source. Which is fine, unless you get unlucky and they don’t build. Which happens way more often than I’d like.
choppaface 6y ago
The dev support is also bottom-of-the-barrel even if you’re a high-margin cloud customer. For a generous upper bound of what Nvidia considers “software support,” look at TensorRT, where a majority of the useful stuff has either been written by third parties or scoped out the hard way by people trying to use it. Nvidia isn’t really a software company, and their core product has a very narrow user interface. These factors hamper whatever you can get out of a Jepsen.
- hthtegr 6y ago
  This is so frustrating to hear. I have a bin full of ARM single board computers, and while the hardware on all of them is pretty much up to the task, the software support from all the vendors has been terrible. I'm in the process of switching to Nvidia hoping it would be the exception. If anyone from Nvidia is reading this, please do everything you can to convince the bosses to allocate the resources required to support a linux machine properly. It takes much more than it seems. These devices are designed to be in production for a long time, so heavy investments now on the software support are going to give value for a long time. Rather than dragging the feet and slowly getting it right over time and devaluing the product in the process. Fingers crossed!!
  - PaulHoule 6y ago
    The dev box system they are selling looks like it might be the best non-x86 workstation in a while, where you really get $700 of computer for $700 as opposed to the bad old days with PA-RISC, Sun, etc. Also I can see something close to that being a competitor to the XSX and PS5. NVIDIA made the Shield and that led to a design win with the Nintendo switch -- wouldn't it be nice to play Nintendo games in VR?
  - Fredej 6y ago
    For what it's worth I've been pretty happy with Freescale. Not sure how they are now after NXP bought them, but the documentation of that hardware was the best I had seen. This translated into an easier time for me as a software guy, because it was easy to look things up. Additionally it seemed like there's a decent ecosystem around it of board suppliers, which then translates into pretty good software support. But again, not sure if that's still the case. I chose them for a project in 2015 and have not regretted it since - but have been able to use the exact same IC since then for other projects so I'm not sure if the experience would be different if starting from scratch today.
axegon_ 6y ago
Agreed. They need to figure out how to put an x86 chip in there to make them truly useful. Also 700 bucks is a bit steep for a raspberry pi with a low powered gpu, just saying...
sdan 6y ago
I use a CI system like Buildkite to build on my Raspberry Pis and then consequently deploy them via Docker Swarm. Working well so far. Although in a similar HN thread a few days ago, people pointed out that docker buildkit does do cross-architecture compilation, which has been slow, but works on anything!
fluffything 6y ago
> Building Docker containers for ARM devices is a pain. I do this every day from a Linux x64 host using qemu-user. What problems are you hitting ? > Second biggest complaint is deploying Jetsons in production environments. > > Third biggest complaint is having to flash Jetsons manually. I don't think these two use cases are the goal of the Jetsons. Feel like more the goal of the EGX devices, which can be programmed, updated, etc. fully remotely.
throwaway9d0291 6y ago
These seem like very weird complaints to me. What are you trying to do with the Jetson? Build some kind of AI-based web service or something? They're made as dev kits for people building "autonomous machines like delivery and logistics robots, factory systems, and large industrial UAVs". Deploying with Docker and running devkits in production isn't what I'd call normal in such applications. Usually you need to deal with that "giant pain" of properly integrating with your hardware. Flashing would usually happen in the factory as part of the process, either by flashing the flash before soldering/inserting it or through some exposed contacts on the board.

deleted 6y ago

[deleted]

0x8BADF00D 6y ago

> Building Docker containers for ARM devices is a pain in the ass. It's not so bad, you just need a beefy ARM machine to build the containers in CI. It would be silly to build a Docker container on the Jetson itself. You would never use an embedded device for compiles and builds, why would you build Docker containers on one?

linarism 6y ago
Genuinely curious, what is an example of a beefy ARM machine?
- pram 6y ago
  Anything with a ThunderX processor is maximum beefy. You can get on-demand servers like that from places like Packet. AWS also has their own A1 instances with lower core counts. These would all be good for cross compiling/builds. Comedy answer: iPad Pro
  - p1necone 6y ago
    > Comedy answer: iPad Pro I mean, the iPad Pro does have a relatively beefy processor. If only you could run arbitrary code on it.
    - BillinghamJ 6y ago
      A lot of instruction sets aren't there yet, but https://ish.app https://ish.app is doing a truly incredible job in this regard. Gives you a working Alpine Linux installation which you can download and install packages for normally, all within the bounds of the normal Apple sandbox, with decent enough performance. It doesn't have SSE or MMX yet, so eg Go and Node aren't usable at this point. But a shocking amount actually does work perfectly, so it's only a matter of time as more instruction sets are implemented.
      - p1necone 6y ago
        It's pretty atrocious that you have to run an x86 emulator to use your iPad "pro" for this stuff. This project is super cool though.
        p1necone 6y ago
        Actually wouldn't a userspace ARM "emulator" be faster than an x86 one on an ARM device? Or are you so far removed from running actual CPU instructions that it doesn't matter?
      - qubex 6y ago
        I suggest you also check out UTM (Universal Turing Machine), an emulator for the iPad that you can build and that allows you to emulate many different architectures and tun OSes written for them (including ARM). https://getutm.app https://getutm.app
- panny 6y ago
  Amazon Graviton2 instances are faster (and cheaper) than x86_64 machines in the same class. https://blog.treasuredata.com/blog/2020/03/27/high-performance-sql-aws-graviton2-benchmarks-with-presto-and-arm-treasure-data-cdp/ https://blog.treasuredata.com/blog/2020/03/27/high-performan...
king_magic 6y ago
Spinning up an ARM machine to build a container that needs to be shipped over the network to a Jetson is a pain when developing.
- 0x8BADF00D 6y ago
  Welcome to embedded computing :-)
  - king_magic 6y ago
    I get it, but I also simply don’t want to deal with it at an embedded level. Which is why I vote with my wallet and choose x64 hardware with Quadro GPUs to ship for production. More expensive, more power consuming? Sure. More sanity? Massively better dev experience? Massively better production/ops experience? Absolutely.
- threeseed 6y ago
  You could spin up an AWS ARM instance in about 30s. Or just have Jenkins etc trigger the launch of one when your build job needs it.
structural 6y ago
You don't even need to do this: qemu support for executing arm64 binaries is fairly good, so you can just run the builds on your beefy x86 server anyways.
- fulafel 6y ago
  What's the single thread performance difference of a fast x86 cpu running emulated ARM code vs a fast ARM server or an AWS ARM instance?
  - fluffything 6y ago
    It's quite fast if you use qemu-user.
    - fulafel 6y ago
      qemu-user just gets you native speed sustem calls right? That's normally <10% of execution time so it can't be a very big help.
ericvh 6y ago
You don't even need the Arm machine for CI, with docker multi-arch builds you can build on x86 -- https://community.arm.com/developer/research/b/articles/posts/continuous-cross-architecture-integration-with-gitlab https://community.arm.com/developer/research/b/articles/post...

arthurcolle 6y ago

Why is it a pain to build Docker container for the ARM architecture? Just curious, haven't had to do it myself.

king_magic 6y ago
If your dependencies don’t have ARM builds, you need to build them from source. Which is fine, unless you get unlucky and they don’t build. Which happens way more often than I’d like.

choppaface 6y ago

The dev support is also bottom-of-the-barrel even if you’re a high-margin cloud customer. For a generous upper bound of what Nvidia considers “software support,” look at TensorRT, where a majority of the useful stuff has either been written by third parties or scoped out the hard way by people trying to use it. Nvidia isn’t really a software company, and their core product has a very narrow user interface. These factors hamper whatever you can get out of a Jepsen.

hthtegr 6y ago
This is so frustrating to hear. I have a bin full of ARM single board computers, and while the hardware on all of them is pretty much up to the task, the software support from all the vendors has been terrible. I'm in the process of switching to Nvidia hoping it would be the exception. If anyone from Nvidia is reading this, please do everything you can to convince the bosses to allocate the resources required to support a linux machine properly. It takes much more than it seems. These devices are designed to be in production for a long time, so heavy investments now on the software support are going to give value for a long time. Rather than dragging the feet and slowly getting it right over time and devaluing the product in the process. Fingers crossed!!
- PaulHoule 6y ago
  The dev box system they are selling looks like it might be the best non-x86 workstation in a while, where you really get $700 of computer for $700 as opposed to the bad old days with PA-RISC, Sun, etc. Also I can see something close to that being a competitor to the XSX and PS5. NVIDIA made the Shield and that led to a design win with the Nintendo switch -- wouldn't it be nice to play Nintendo games in VR?
- Fredej 6y ago
  For what it's worth I've been pretty happy with Freescale. Not sure how they are now after NXP bought them, but the documentation of that hardware was the best I had seen. This translated into an easier time for me as a software guy, because it was easy to look things up. Additionally it seemed like there's a decent ecosystem around it of board suppliers, which then translates into pretty good software support. But again, not sure if that's still the case. I chose them for a project in 2015 and have not regretted it since - but have been able to use the exact same IC since then for other projects so I'm not sure if the experience would be different if starting from scratch today.

axegon_ 6y ago

Agreed. They need to figure out how to put an x86 chip in there to make them truly useful. Also 700 bucks is a bit steep for a raspberry pi with a low powered gpu, just saying...

sdan 6y ago

I use a CI system like Buildkite to build on my Raspberry Pis and then consequently deploy them via Docker Swarm. Working well so far. Although in a similar HN thread a few days ago, people pointed out that docker buildkit does do cross-architecture compilation, which has been slow, but works on anything!

fluffything 6y ago

> Building Docker containers for ARM devices is a pain. I do this every day from a Linux x64 host using qemu-user. What problems are you hitting ? > Second biggest complaint is deploying Jetsons in production environments. > > Third biggest complaint is having to flash Jetsons manually. I don't think these two use cases are the goal of the Jetsons. Feel like more the goal of the EGX devices, which can be programmed, updated, etc. fully remotely.

throwaway9d0291 6y ago

These seem like very weird complaints to me. What are you trying to do with the Jetson? Build some kind of AI-based web service or something? They're made as dev kits for people building "autonomous machines like delivery and logistics robots, factory systems, and large industrial UAVs". Deploying with Docker and running devkits in production isn't what I'd call normal in such applications. Usually you need to deal with that "giant pain" of properly integrating with your hardware. Flashing would usually happen in the factory as part of the process, either by flashing the flash before soldering/inserting it or through some exposed contacts on the board.