19 ms·

Oh wow, this is a mixed bag. First off, NVENC is gone. Instead of 4x 1080p @ 30 (HEVC) that the Jetson Nano could do with GPU acceleration, this one can barely

Oh wow, this is a mixed bag. First off, NVENC is gone. Instead of 4x 1080p @ 30 (HEVC) that the Jetson Nano could do with GPU acceleration, this one can barely manage 2x 1080p @ 30. Apparently, too many people (myself included) used the Jetson Nano as a cheap camera recorder. Next, price went from $99 for a Jetson Nano to $299 for Jetson Orin Nano. And lastly, they now advertise 40 TOPS for Jetson Orin Nano but that's measured at 50% sparsity and for INT8... So it is in no way comparable to the 0.5 TFLOPS that a Jetson Nano has, which was measured at dense FP16. And the fact that I can't find any FP16 number for any Jetson Orin device is not a good look. In summary, this looks to me more like a TPU than a GPU, because they moved from FP16 to INT8 precision. That makes this much more similar to the mini PCIe Coral priced at $25 delivering 4 TOPS dense. Compared to that, Jetson Orin Nano is 5x faster and 12x more expensive. EDIT: In case you're wondering why I care about INT8 vs. FP16: The Jetson Nano can autonomously collect data and then calculate gradients and fine-tune the AI model. If the Jetson Orin Nano lacks FP16, it won't be able to do gradients and learning. So going from FP16 to INT8 is a downgrade from having AI training capabilities to pure execution of pre-trained AI models.

sjnair96 4y ago
Damn, why shoot themselves in the foot for image processing tasks. Appreciate the insights!
c_o_n_v_e_x 4y ago
Out of curiosity, what use cases require training at the edge?
- hasperdi 4y ago
  I'm no expert in ML, one use case is that if you want to refine the already trained model that was trained elsewhere.
  - bravura 4y ago
    Additionally, privacy-preserving AI.
- fxtentacle 4y ago
  In my example, it is very helpful if you can adjust a speech model a bit to better match the user's dialect. So you don't train the full model, rather you just adjust the most coarse 5% of parameters in a pre-trained model by a tiny bit. But that can lead to great quality improvements at almost no additional cost.
aseipp 4y ago
Orin has support for FP16 of course, it would be insane not to have it when INT8 isn't viable for a lot of models anyway. There are some preliminary benchmarks that can confirm this that weren't hard to find.[1] The number juicing by way of using "TOPS" to mean "Sparse INT8" or whatever has been going on since the beginning of the Ampere line, though, so yes you should take that with a grain of salt (Ampere being the gen they added sparse tensor cores.) But not including FP16 support would just be a ridiculous move, frankly. According to this table on this page[2], the lack of the NVENC support is due to the "Nano" vs "NX" product lines which they are introducing. NX started with Xavier, there was never a Xavier Nano, to my knowledge (since you could just buy the original Jetson Nano.) Xavier NX still has NVENC support, and now you can buy separate Orin NX modules with even more RAM. So they're just segregating their products further, it looks like. (Hit "Specifications" then expand the table and look at Video Encode/Decode support.) So for people like us, we just want the Orin NX series. But yes, no more ultra cheap cameras. I'm guessing they're eventually going to gear for Orin as an eventual successor to the original Nano, but who knows. It'll take a few price cuts to get there. [1] https://www.edge-ai-vision.com/2022/04/is-the-new-nvidia-jetson-agx-orin-really-a-game-changer-we-benchmarked-it/ https://www.edge-ai-vision.com/2022/04/is-the-new-nvidia-jet... [2] https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/ https://www.nvidia.com/en-us/autonomous-machines/embedded-sy...
- csdvrx 4y ago
  > the lack of the NVENC support is due to the "Nano" vs "NX" product lines Translated into plain English: "we're crimping the regular product because we want to extract more cash from consumers: we didn't like that they enjoyed so much this hardware encoding support. Let's make them cough up a few extra hundreds by cutting it out!"
- fxtentacle 4y ago
  Your link [1] doesn't seem to mention the Orin Nano anywhere? It says that the bigger SOMs with DLA have FP16. But the Orin Nano doesn't have DLA.
bpye 4y ago
On the flip side, 6xA78 cores with decent range of IO (7 PCIe 3 lanes for example) is quite nice. There aren't many SoCs that individuals can really use that are comparable.

sjnair96 4y ago

Damn, why shoot themselves in the foot for image processing tasks. Appreciate the insights!

c_o_n_v_e_x 4y ago

Out of curiosity, what use cases require training at the edge?

hasperdi 4y ago
I'm no expert in ML, one use case is that if you want to refine the already trained model that was trained elsewhere.
- bravura 4y ago
  Additionally, privacy-preserving AI.
fxtentacle 4y ago
In my example, it is very helpful if you can adjust a speech model a bit to better match the user's dialect. So you don't train the full model, rather you just adjust the most coarse 5% of parameters in a pre-trained model by a tiny bit. But that can lead to great quality improvements at almost no additional cost.

aseipp 4y ago

Orin has support for FP16 of course, it would be insane not to have it when INT8 isn't viable for a lot of models anyway. There are some preliminary benchmarks that can confirm this that weren't hard to find.[1] The number juicing by way of using "TOPS" to mean "Sparse INT8" or whatever has been going on since the beginning of the Ampere line, though, so yes you should take that with a grain of salt (Ampere being the gen they added sparse tensor cores.) But not including FP16 support would just be a ridiculous move, frankly. According to this table on this page[2], the lack of the NVENC support is due to the "Nano" vs "NX" product lines which they are introducing. NX started with Xavier, there was never a Xavier Nano, to my knowledge (since you could just buy the original Jetson Nano.) Xavier NX still has NVENC support, and now you can buy separate Orin NX modules with even more RAM. So they're just segregating their products further, it looks like. (Hit "Specifications" then expand the table and look at Video Encode/Decode support.) So for people like us, we just want the Orin NX series. But yes, no more ultra cheap cameras. I'm guessing they're eventually going to gear for Orin as an eventual successor to the original Nano, but who knows. It'll take a few price cuts to get there. [1] https://www.edge-ai-vision.com/2022/04/is-the-new-nvidia-jetson-agx-orin-really-a-game-changer-we-benchmarked-it/ https://www.edge-ai-vision.com/2022/04/is-the-new-nvidia-jet... [2] https://www.nvidia.com/en-us/autonomous-machines/embedded-systems/jetson-orin/ https://www.nvidia.com/en-us/autonomous-machines/embedded-sy...

csdvrx 4y ago
> the lack of the NVENC support is due to the "Nano" vs "NX" product lines Translated into plain English: "we're crimping the regular product because we want to extract more cash from consumers: we didn't like that they enjoyed so much this hardware encoding support. Let's make them cough up a few extra hundreds by cutting it out!"
fxtentacle 4y ago
Your link [1] doesn't seem to mention the Orin Nano anywhere? It says that the bigger SOMs with DLA have FP16. But the Orin Nano doesn't have DLA.

bpye 4y ago

On the flip side, 6xA78 cores with decent range of IO (7 PCIe 3 lanes for example) is quite nice. There aren't many SoCs that individuals can really use that are comparable.