10 ms·
Jetson AGX Xavier
- weinzierl 7y agoThey seem to offer a cheaper 8 GB model too but unfortunately I see no price for it. I'm curious how much it'll be because, as much as I'd like to toy around with this, the $699 is a little to much for just experimentation. EDIT: The 8GB Module seems to be $679 here[1]. This makes the $699 or the 32 GB Developer Kit seem like a steal. Still, too expensive for play, I guess I'll stick with my Jetson Nanos for a while... [1] https://www.arrow.com/en/products/900-82888-0060-000/nvidia https://www.arrow.com/en/products/900-82888-0060-000/nvidia
- fluffything 7y agoThere is also the Jetson Nanokit which costs ~120 EUR.
- potiuper 7y agoMaxwell does not have unified memory -> custom code compared to latest generation along with performance disadvantages.
- fluffything 7y agoUnified memory is useful during development when porting the application, but I don't know of any well-tuned applications that use it.
- potiuper 7y agoAny "well-tuned" multi-threaded application requires unified memory for simultaneous access to managed memory from the CPU and GPUs as it is not possible with compute capability lower than 6.0. This is because pre-Pascal GPUs lack hardware page faulting, so coherence can’t be guaranteed. On these GPUs, an access from the CPU while a kernel is running will cause a segmentation fault.
- my123 7y agoI see that AGX Xavier devkit as much more of an Arm desktop platform, which it is very suitable for too.
- neuromancer2701 7y agoI thought I saw the 8GB module for 459$ but it is only the SOM so you have to purchase a carrier board as well so the 32GB Developer Kit seems the way to go. https://www.arrow.com/en/products/900-83668-0000-000/nvidia https://www.arrow.com/en/products/900-83668-0000-000/nvidia
- fvv 7y agoI'm definitely not expert and probably this is a dumb question , but why smart edge things like smart robot and not dumb edge with smart central brain ? Anyway data are useful aggregated central;ly why not incorporate the brain centrally too?
- fvv 7y agoI mean it could be way more powerful like stadia on phone vs using phone gpu , latency is not too high for the described usage .. imo just automotive may require a dedicated brain on the edge, I'm totally wrong ?
- mebr 7y agoI work at a startup that uses edge AI. There are many factors that edge is preferred over cloud. Security is one. Latency is important in many cases. If the internet connection is another dependency for a critical system, it can be a big headache. Once you start working on a real-world project you run into these issues. In return you give up monitoring the data and model that can be done with cloud deployment.
- halotrope 7y agoCloud GPU/TPU resources are still somewhat expensive. Also bandwidth can be an issue when you would first need to feed video through potentially metered connections. Last but not least latency can be an issue for e.g robotics and automotive.
- fluffything 7y agoNot a dumb question at all: data traffic is expensive. If you have thousands of remote sensors collecting Gbs and GBs of real time data, for ~1000$ you can add a "streaming" supercomputer to your sensor to analyze the data in place and save on network and storage costs. Notice however that the announcement is for an Nvidia AGX product, which is for autonomous machines. The Nvidia "edge" products for processing data on the sensors are called Nvidia EGX. For autonomous machines, you often need to analyze the data in the machine anyways, e.g., you don't want a drone falling if it looses network connectivity.
- farseer 7y agoHonest question, can something like this kill the market for embedded DSP processors made by Texas Instruments or Analog Devices?
- Cerium 7y agoI'm seeing lots of Analog Devices DSPs replaced by Tegras and Zync MPSoCs.
- augustt 7y agoI mean, this is absurdly more powerful than those dedicated DSPs.
- WJW 7y agoIt's also easily 10x the price. It really matters what the application is and how much processing power you need.
- qppo 7y agoNo, those face more threat from ARM M4/7 cores or RISC-V units with vector/SIMD coprocessors. DSP cores are being lifted out of discrete chips and placed as IP blocks into more integrated solutions, with the really complex algo stuff placed in general purpose hardware like FPGAs. The economics don't really make sense for TI/ADI DSPs imo. If you had an application where you needed a chip just to do DSP you'd probably use an ARM core instead - but the applications engineers at TI/ADI will gladly help you find a product in their catalog that has more features integrated into it (like ADC/DAC, even analog front ends for audio/RF, USB/Bluetooth stacks) for your product. Basically there's no market to kill, from what I've seen.
- smg 7y agoThe cheapest Volta GPUs I have seen so far cost over 2K for 12GB. Can the GPU provided in this kit be used for training?
- fizixer 7y agoNope, the use of the words 'edge' and 'inference' in the tag-line pretty much mean there is no learning, no training.
- wmf 7y agoDoes it have "fake" tensor cores? Aren't those for training?
- fizixer 7y agoYou still need tensor cores for inference. But they don't do weight updates. Learning/training is all about updating the weights (through backpropagation or whatever). So another way to put it: its tensor cores do feed-forward calculations, but no backpropagation, and no weight updates.
- rrss 7y agoThe hardware and platform is capable of training just fine. It's just rarely done because it is slower than training on pretty much any discrete GPU.
- dchichkov 7y agoYes, if your model is small enough or, if you are fine-tuning small number of layers. TendorFlow 1.15 and 2.0 are available on Xavier. I understand that PyTorch could be built as well. Nite that the number of CUDA kernels and amount of memory available is smaller, if compared to descrete Volta GPUs.
- fizixer 7y agoYou say it can do training for small models because of the presence of the small (512-core) GPU? (plus maybe some left-over, control calculations by the CPU)
- fizixer 7y agoInference only. (So this is competing with Google TPUv1; a few years late and way more expensive, but with more memory)
- my123 7y agoIt can do training with its GPU, not the fastest thing in the world though.
- fizixer 7y agoSo looks like it can do "checkbox training" (add an iota of training capability just so you could check the box labelled "it does training"). Got it.
- wmf 7y agoYou can't put a TPUv1 in a car because Google doesn't sell them.
- rrss 7y ago1. This isn't inference only, it has the full capabilities of a normal GPU, just small and low power (and therefore much slower than normal GPUs). 2. TPUv1 is a matrix multiply ASIC that requires a host CPU to do anything. This thing is a SoC that includes both a CPU and a GPU. The CPU is pretty fast for what it is - much faster than e.g. raspberry pi, see https://www.phoronix.com/scan.php?page=article&item=nvidia-jetson-xavier&num=3 https://www.phoronix.com/scan.php?page=article&item=nvidia-j.... 3. not sure how you know whether this is more expensive than a TPUv1, since the TPUv1 was never sold or available outside of google. A much better comparison would be between this and the Edge TPU development board.
- TaylorAlexander 7y agoI have one of these powering my open source four wheel drive robot. [1] I've started doing machine learning experiments with it finally. (See [1] for details) There's a few tricks to getting the best performance. You want to convert your neural network to run with NVIDIA's TensorRT library instead of just tensorflow or torch. TensorRT does all the optimized goodness that gets you the most out of the hardware. Not all possible network operations can run in TensorRT (though nvidia updates the framework regularly). This means some networks can't be easily converted to something fully optimized for this platform. Facebook's detectron2 for example uses some operations that don't readily convert. [2] But then if you're new like me you've got to both find some code that will ultimately produce something you can convert to TensorRT, and you also need something that you can easily train. I've learned that training using your own dataset is often non-obvious. A lot of example code shows how to use an existing dataset but they totally gloss over the specific label format those datasets use. That means you've got to do some digging to figure out how to make your own dataset load properly in to the training code. After trying a few different things, I've gotten some good results training using Bonnet(al) [3]. I was able to make enough sense of its training code to use my own dataset, and it looks like it will readily convert to TensorRT. Then you load the converted network using NVIDIA's Deepstream library for maximum pipeline efficiency [4]. The performance numbers for the AGX Xavier are very good, and I am hopeful I will get my application fully operation soon enough. [1] https://reboot.love/t/new-cameras-on-rover/ https://reboot.love/t/new-cameras-on-rover/ [2] https://github.com/facebookresearch/detectron2/issues/192 https://github.com/facebookresearch/detectron2/issues/192 [3] https://github.com/PRBonn/bonnetal https://github.com/PRBonn/bonnetal [4] https://developer.nvidia.com/deepstream-sdk https://developer.nvidia.com/deepstream-sdk
- m3at 7y agoCool robot! For dealing with layers not supported by TensorRT, you might want to try to export to onnx instead and then use tvm[1] to compile your model for the hardware. I have not used it on nvidia boards yet, but I had good experience on other less powerful ARM boards, and as tvm docs show some examples running on jetson tx I imagine the AGX Xavier is likely fine too. [1] https://tvm.apache.org/ https://tvm.apache.org/
- my123 7y agoFor a moddable Arm platform with all batteries included, there isn't really an alternative to this, especially at $699. It's much stronger than an RPi and could fill the gap between RPi and Arm-based server platforms.
- mappu 7y agoIn terms of using it as an ARMv8 desktop workstation (with decent CPU performance, real SATA / Ethernet / PCI-e connectors) - some other contenders include the MACCHIATObin (quad A72) and Honeycomb LX2K (16-core A72, 750USD) from Solid-Run.
- king_magic 7y agoMy biggest complaint with the Jetson line is it's all ARM. Look, I get it. But the developer experience is horrible. Building Docker containers for ARM devices is a pain. Hell, building anything for a Jetson can be a pain unless it's a pre-packaged NVIDIA thing - really not a fan of building things from source. Add on top of that NVIDIA's very low level documentation for pretty much any tooling they ship, coupled with the difficulty in getting near-time engineering support (unless you want to post to one of their message boards, and hope you get an answer back in less than a week)... basically, it's really rough to do anything seriously useful with Jetson hardware. Second biggest complaint is deploying Jetsons in production environments. Dev kits aren't production stable, so you either need to build your own carrier board or find one pre-built, and frankly that's just a giant pain to do. Third biggest complaint is having to flash Jetsons manually. Misery. A production-ready x64 Jetson that you could order directly from NVIDIA would be my dream. Add up all of the shortcomings and overhead of ARM Jetsons and IMO you do not have a viable device for shipping AI solutions at scale.
- deleted 7y ago[deleted]
- 0x8BADF00D 7y ago> Building Docker containers for ARM devices is a pain in the ass. It's not so bad, you just need a beefy ARM machine to build the containers in CI. It would be silly to build a Docker container on the Jetson itself. You would never use an embedded device for compiles and builds, why would you build Docker containers on one?
- linarism 7y agoGenuinely curious, what is an example of a beefy ARM machine?
- pram 7y agoAnything with a ThunderX processor is maximum beefy. You can get on-demand servers like that from places like Packet. AWS also has their own A1 instances with lower core counts. These would all be good for cross compiling/builds. Comedy answer: iPad Pro
- snek 7y agoThe lineup continues to balloon in price. A lot of students would the TK and TX models for robotics and whatnot, since they were only 200-300 bucks.
- shmolyneaux 7y agoThe Jetson Nano[0] seems like it would be better for students, it's only $99. [0]: https://developer.nvidia.com/embedded/jetson-nano-developer-kit https://developer.nvidia.com/embedded/jetson-nano-developer-...
- deleted 7y ago[deleted]
- jerryyu 7y agoJust built a gesture controlled robot[1] with the Xavier board. We were able to run OpenPose[2] at 27FPS, which we found was even faster than running it on K80 on AWS p2.xlarge. It was a pain to install caffe and all the dependencies on an ARM processor, but it worked out eventually. We were able to train and run Tensorflow 2 models quickly also. Felt like using an actual GPU at a fraction of the cost. [1] https://www.youtube.com/watch?v=AF8zmTaa17s https://www.youtube.com/watch?v=AF8zmTaa17s [2] https://github.com/CMU-Perceptual-Computing-Lab/openpose https://github.com/CMU-Perceptual-Computing-Lab/openpose
- mchusma 7y agoThe original posted title was more helpful. The Jetson AGX Xavier has been out for a couple of years, but it dropped in price from $999 to $699 and has double the ram now at 32GB.
- fxtentacle 7y agoI wonder how to train for those. The biggest one has 32GB of RAM and needs a frozen inference graph converted to TensorRT. So one would need a GPU with 32GB of RAM in addition to this to be able to train the network. AFAIK, Nvidia doesn't sell anything with that much RAM.
- Symmetry 7y agoNo details on the CPU I wonder if it's going to be a Carmel (via the Transmeta lineage).
- crelex 7y agoI develop on one of these every single day I just use Visual Studio Code with custom launch.json and task.json that allows me to ssh into the jetson, copy over the compiled for linux code, and attach a remote debugger.. I like never even touch the jetson and all my builds and shit are pushed over... its actually an easy as shit development experience at this point.. You have to kinda know what you're doing... but its all totally usable. The dev kit i have has been running 415 days without any issues running 5 custom programs I've written.