3 ms·
It seems they’re being very careful not to undercut their enterprise offerings or even the 4090. Assuming they’re not completely tone deaf, I can only assume th
by deepsquirrelnet 3y ago
It seems they’re being very careful not to undercut their enterprise offerings or even the 4090. Assuming they’re not completely tone deaf, I can only assume this is the explanation.
- schaefer 3y agoThe Jetson AGX Orin Developer Kit [1] has 64 GB of unified 256-bit LPDDR5 memory. It costs $2,000 and might get some people someplace interesting. [1]: https://developer.nvidia.com/embedded/learn/getting-started-jetson#documentation https://developer.nvidia.com/embedded/learn/getting-started-...
- eurekin 3y agoThanks! That is every interesting. Here's a direct amazon link: https://www.amazon.com/dp/B0BYGB3WV4 https://www.amazon.com/dp/B0BYGB3WV4 And a running demo: https://forums.developer.nvidia.com/t/llama-2-llms-w-nvidia-jetson-and-textgeneration-web-ui/264003/22?page=2 https://forums.developer.nvidia.com/t/llama-2-llms-w-nvidia-...
- gautamcgoel 3y agoLPDDR5 doesn't have nearly as much memory bandwidth of GDDR6.
- skavi 3y agoPer chip? Not the full story when discussing a system which can integrate multiple. The Orin has more memory bandwidth than an RTX 4050 even though the latter uses GDDR6. The M3 Max has double the bandwidth of the Orin, but also uses LPDDR5.
- wmf 3y agoOrin is kind of expensive for what it does. I think you'd be better off with a Mac Studio for $2,400 at this point.
- cma 3y ago5FP32 TFLOPs, if not doing sparse low precision inference it seems to be about in line with mid-high end 2014 Nvidia consumer card performance (gtx 980), one decade old. For running sparsified/quantized llama2 it might be good, not sure about for fine tuning. I didn't see any FP16 numbers.