3 ms·
You can get an Orin with 8GB. Or the 32/64GB AGX module for 1.3/2.3k (3k for the dev kit). Not cheap but plenty of ram and 60W power target maximum. You could
by joshvm 3y ago
You can get an Orin with 8GB. Or the 32/64GB AGX module for 1.3/2.3k (3k for the dev kit). Not cheap but plenty of ram and 60W power target maximum.
You could maybe run something on the 4GB Jetson Nano?
But very slow: https://www.reddit.com/r/LocalLLaMA/comments/12c7w15/the_pointless_experiment_jetson_nano_2gb_running/ https://www.reddit.com/r/LocalLLaMA/comments/12c7w15/the_poi...
- brucethemoose2 3y agoIIRC 32GB of shared RAM is not enough for llama 70B, and 8GB is just barely enough for llama 7B... So yeah, that value proposition is not good at all. A 32GB+ ddr5 laptop with a dGPU and some RAM will (IIRC, just barely) do llama 70B for far less money and a similar TDP.