14 ms·
Nvidia's Project Digits is a 'personal AI supercomputer'
- teleforce 2y agoThis is it guys, the most valuable company in the world is selling Linux desktop, after decades of speculating finally the year of Linux desktop is 2025! Joking aside, personally will buy this workstation in a heartbeat if I have the budget to spare, one in the home and another in the office. Currently I have an desktop/workstation for AI workloads with similar 128GB RAM that I bought few years back that cost around USD5K without the NVIDIA GPU that I bought earlier for about USD1.5K, with a total of about USD6.5K without a display monitor. This the same price of NeXT workstation (with a monitor) when it's sold back in 1988 without adjusting for inflations (now around USD18K) but it is more than 200 times faster in CPU speed and more than 1000 times RAM capacity than the original 25 MHz CPU and 4 MB RAM, respectively. The later updated version of NeXT has graphic accelerator with 8 MB VRAM, since the workstation has RTX 2080 it is about 1000 times more. I believe the updated NeXT with graphic accelerator is the one that used to develop original Doom software [1]. If NVIDIA can sell the Project Digits Linux desktop at USD3K with similar or more powerful setup configurations, it's going to be a winner and probably can sell by truckloads. It seems to has NeXT workstation vibe to it that used to develop the original WWW and Doom software. Hopefully it will be used to develop many innovative software but now using open source Linux software eco-system not proprietary one. The latest Linux kernel now has real-time capability for more responsive desktop experience and as saying goes, good things come to those who wait. [1] NeXT Computer: https://en.m.wikipedia.org/wiki/NeXT_Computer https://en.m.wikipedia.org/wiki/NeXT_Computer
- magicalhippo 2y agoNot much was unveiled but it showed a Blackwell GPU with 1PFLOP of FP4 compute, 128GB unified DDR5X memory, 20 ARM cores, and ConnectX powering two QSFP slots so one can stack multiple of them. edit: While the title says "personal", Jensen did say this was aimed at startups and similar, so not your living room necessarily.
- computably 2y agoFrom the size and pricing ($3000) alone, it's safe to conclude it has less raw FLOPs than a 5090. Since it uses LPDDR5X, almost certainly less memory bandwidth too (5090 @ 1.8 TB/s, M4 Max w/ 128GB LPDDR5X @ 546 GB/s). Basically the only advantage is how much VRAM it packs in a small form factor, and presumably greater power efficiency at its smaller scale. The only thing it really competes with is the Mac Studio for LocalLlama-type enthusiasts and devs. It isn't cheap enough to dent the used market, nor powerful enough to stand in for bigger cards.
- sliken 2y agoI believe $3,000 is for the unmentioned minimum config, no idea on the mentioned 4TB storage and 128GB ram version. Running a 96GB ram model isn't cheap (often with unified memory 25% is reserved for CPUs), so maybe it will win there.
- ac29 2y agoThe NVIDIA press release [0] says "Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage", which makes it sound like the RAM is fixed size. [0] https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
- sliken 2y agoAwesome. Maybe there will be storage options of 1,2,and 4TB and optional 25/100/200/400 GBit interfaces. Or maybe everything except the CPU/GPU is constant, but having a 50%, 75%, or 100% of the CPU/GPU cores so they can bin their chips.
- kcb 2y agoMaking comparisons to the 5090 is silly. That thing draws 500W+ and will require a boat anchor of metal to keep it cool. The device they showed is something more along the lines of a mobile dev kit.
- The28thDuck 2y ago[flagged]
- deactivatedexp 2y ago[flagged]
- blurbleblurble 2y agoIs this going to make up for the lack of VRAM in the new consumer GPUs?
- modeless 2y agoFinally a real ARM workstation from Nvidia! This will be much faster than Apple's offerings for AI work. And at $3000 it is much cheaper than any Mac with 128 GB RAM.
- sliken 2y agoOn the CPU size the Neoverse N2 doesn't compete particularly well with apple's M4, or the Zen5 for the matter. Bit bit hard to tell what's on offer on the GPU side, I wouldn't be surprised if it was RTX 4070 to 5070 in that range. If the price/perf is high enough $3k wouldn't be a bad deal, I suspect a Strix Halo (better CPU cores, 256GB/sec memory interface, likely slower GPU cores) will be better price/perf, same max ram for unified memory, and cheaper.
- modeless 2y agoAI work happens predominantly on the GPU, not the CPU. This GPU with CUDA will run rings around M4 with MLX. And with much more RAM than you can get in a Mac for $3k. A lot of people have been justifying their Mac Studio or Mac Pro purchases by the potential for running large AI models locally. Project Digits will be much better at that for cheaper. Maybe it won't run compile Chromium as fast, but that's not what it's for.
- sliken 2y agoThe quotes I've seen mention the maximum config (128GB ram and 4TB of storage) and the minimum price. Nothing saying $3k for 128GB ram and 4TB of storage. I hope I'm wrong, but I'm betting the max price is at least twice the minimum price.
- gnabgib 2y agoNVidia says 128GB ram at $3k[0], it looks like the 4TB storage might be variable (and possibly CPU or GPU cores?). This article says 128GB too.. but used up to twice in a row with different meanings which doesn't help. [0]: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
- Karupan 2y agoI feel this is bigger than the 5x series GPUs. Given the craze around AI/LLMs, this can also potentially eat into Apple’s slice of the enthusiast AI dev segment once the M4 Max/Ultra Mac minis are released. I sure wished I held some Nvidia stocks, they seem to be doing everything right in the last few years!
- behringer 2y agoNot only that, but it should help free up the gpus for the gamers.
- dagmx 2y agoI think the enthusiast side of things is a negligible part of the market. That said, enthusiasts do help drive a lot of the improvements to the tech stack so if they start using this, it’ll entrench NVIDIA even more.
- option 2y agotoday’s enthusiast, grad student, hacker is tomorrow’s startup founder, CEO, CTO or 10x contributor in large tech company
- Mistletoe 2y ago> tomorrow’s startup founder, CEO, CTO or 10x contributor in large tech company Do we need more of those? We need plumbers and people that know how to build houses. We are completely full on founders and executives.
- davrosthedalek 2y agoWe might not, but Nvidia would certainly like it.
- hatboat 2y agoIf they're already an "enthusiast, grad student, hacker", are they likely to choose the "plumbers and people that know how to build houses" career track? True passion for one's career is rare, despite the clichéd platitudes ecouraging otherwise. That's something we should encourage and invest in regardless of the field.
- rafaelmn 2y agoFeels like data center AI HW demand peak is over now that these things are trickling down to consumers and they are diversifying customers. Also going lower than expected on gaming HW, seems like they have enough fab capacity.
- bushbaba 2y agoTheir DC sales likely aren’t growing at the rates prior seen. Law of big numbers. They gotta diversify to satisfy growth & profit expectations
- narrator 2y agoNvidia releases a Linux desktop supercomputer that's better price/performance wise than anything Wintel is doing and their whole new software stack will only run on WSL2. They aren't porting to Win32. Wow, it may actually be the year of Linux on the Desktop.
- CamperBob2 2y agoWhere does it say they won't be supporting Win32?
- narrator 2y agoHere he says that in order for the cloud and the PC to be compatible, he's going to only support WSL2, the Windows subsystem for Linux which is a Linux API on top of Windows. Here's a link to the part of the keynote where he says this: https://youtu.be/MC7L_EWylb0?t=7259 https://youtu.be/MC7L_EWylb0?t=7259
- stonogo 2y ago"Linux API on top of Windows" is an interesting way to describe a virtual machine.
- sedatk 2y agoThat’s more like WSL1, yes.
- SteveNuts 2y agoWSL2 is no longer a VM, afaik.
- gnabgib 2y agoOther way around (1 wasn't, 2 runs in a managed HyperV VM) https://learn.microsoft.com/en-us/windows/wsl/compare-versions#comparing-features https://learn.microsoft.com/en-us/windows/wsl/compare-versio...
- gigel82 2y agoWhat we need is more diversity in the space. Something between the Jetson and this thing, at under $1000 that can run in a LAN to do LLM, STT, TTS, etc. would be an awesome (if niche) device to enable truly local / private AI scenarios for privacy-sensitive folks.
- lithiumii 2y ago[dead]
- tkanarsky 2y agoThis seems surprisingly cheap for what you get! Excited to see what people cook with this
- giacomoforte 2y agoIt costs the equivalent of 2 years of cloud GPU H100s at current prices. Edit: Sorry fucked up my math. I wanted to do 40x52x4, $4/hr being the cloud compute price but that us actually $8300, so it is actually equivalent to about 4.5 months of cloud compute. 40 hours because I presume that this will only be used for prototyping and debugging, i.e during office hours.
- sabareesh 2y agowhat are current h100 price ? lowest i have seen is only 0.99 per hour
- billconan 2y agoand 2 years have 17520 hours.
- YetAnotherNick 2y agoWhere can you find 99c/hour? Cheapest I can find is double that.
- saagarjha 2y agoLambda has on-demand GH200 right now for $1.49. There might be a cheaper deal elsewhere for a contract.
- YetAnotherNick 2y agoWeird that they have cheapest H100 for $2.49. It should either be shared GH200 or it could just a promotional price.
- saagarjha 2y agoIt’s a temporary thing I think because nobody wants to use ARM
- 2y ago
- anigbrowl 2y agoHonestly surprised at how affordable this is, I was expecting $5-6k as I scanned the opening paragraphs.
- sliken 2y agoMight well be, from what I can tell the price "starts at $3k", which might well be config'd like the minimum mac studio for RAM and storage. Mac studios easily hit $5k-$6k or more.
- rubatuga 2y agoWould consider at a lower price of $500 USD, way too expensive for what it brings.
- lz400 2y agoI think it's cheap at $3000. 128gb RAM, top of the line GPU capabilities, 4tb storage... it's much better than what a top shelf mbp can do and much cheaper
- sergiotapia 2y agocorrect, I just spent $4k on an "AI" machine to do stuff. 96GB ram, ryzen 9 9950x 16 core, 4TB nvme, 24tb hdd, rtx 4090. If this thing was available six months ago I would have bought it instead!
- Mistletoe 2y agoWhat do you do on it?
- lispm 2y agoupto 4tb storage
- saagarjha 2y agoThis definitely loses on CPU performance.
- sliken 2y agoI originally thought so, since the previous Grace CPUs used the neoverse N2, which loses to Apple's M4 cores. However apparently 10 of the cores are the Cortex-X925 CPUs, which are a serious upgrade. Basically 10 performance cores and 10 efficiency cores that should be pretty competitive with any current apple CPU.
- saagarjha 2y ago
- ttul 2y agoI bought a Lamba Labs workstation with a 4090 last year. I guess I’m buying one of these things now because the Lambda workstation just became a relic…
- trhway 2y ago$3000? The GB10 inside it seems to be a half of GB200 which is like $60K. One can wonder about availability at those $3K.
- kcb 2y agoNo way is the GPU half a GB200. I'd expect something much lower end and power conscious. They mention 1 PFLOP for FP4, GB200 is 40 PFLOP.
- rs38 2y agoat least the naming implies factor ~20 less :)
- sliken 2y agoTheRegister mentions: Specs we’ve seen suggest the GB10 features a 20-core Grace CPU and a GPU that packs manages a 40th the performance of the twin Blackwell GPUs used in Nvidia’s GB200 AI server.
- mrtksn 2y agoOkay, so this is not a peripheral that you connect to your computer to run specialized tasks, this is a full computer running Linux. It's a garden hermit. Imagine a future where everyone has one of those(not exactly this version but some future version), it lives with you it learns with you and unlike the cloud based SaaS AI you can teach it things immediately and diverge from the average to your advantage.
- noduerme 2y ago"garden hermit" is a very interesting and evocative phrase. Where is that from?
- mrtksn 2y agoIt's a real thing: https://en.wikipedia.org/wiki/Garden_hermit https://en.wikipedia.org/wiki/Garden_hermit In the past, in Europe, some wealthy people used to look after of a scholar living on their premises so they can ask them questions etc.
- Mistletoe 2y agoThis is so strange, my girlfriend was just telling me about those yesterday. The word “ornamental hermit” fills me with about as much disgust as I can experience. > Later, suggestions of hermits were replaced with actual hermits – men hired for the sole purpose of inhabiting a small structure and functioning as any other garden ornament.
- noduerme 2y agoaha, this is really something. I just got around to watching "Furiosa" last night. So something like having a personal "history man" (although, my take on the whole Mad Max series is that it's just bottled up fear-porn about white settlers going uncivilized and becoming "tribal" - a colonial horror tale, "The Heart of Darkness" with motorcycles - common anywhere a relatively small group spread themselves out on a lot of ill-gotten land, did some nasty deeds and lost touch with the mothership). In the Furiosa context, it's a bit like a medicine man or shaman, then. A private, unreliable source of verbal hand me downs, whose main utility is to make elites feel like they have access to knowledge without needing to acquire it for themselves or question its veracity. We really are entering a new dark age.
- YetAnotherNick 2y agoI highly doubt it's half or ever quarter of GB200, unless they have hidden water cooling or something outside. GB200 is 1200 Watts. Digits doesn't look like it would be above 200W, and cooling 200W would be impressive.
- palmfacehn 2y agoWould love to see something like this with an ATX form factor, socketed GPU, socketed GPU and non-soldered RAM.
- blackoil 2y agoIsn't that just a regular PC with one or more 5090 or equivalent workstation GPU?
- palmfacehn 2y agoThere would be unified GPU/CPU memory and an ARM processor that isn't soldered to the board.
- pbalcer 2y agoOne of the reasons why they can do unified memory efficiently is because the CPU/GPU is a single SoC. If you separate them, you end up with a normal PC architecture, with memory having to go through a PCIe bus. This is possible to do with reasonable latency and bandwidth (thanks to CXL), but we haven't seen that in consumer hardware. Even in server space I think only MI300 supports CXL, and even then I don't think it's something AMD particularity promotes. Personally I think Strix Halo workstations may come with expendable memory, storage and free PCIe slots. But then you have to deal with ROCm...
- Tepix 2y agoWith more and more personal AI, i think having a truly private device that can run large LLMs (remember: larger is better) is fantastic! Ideally we can configure things like Apple Intelligence to use this instead of OpenAI and Apple's cloud.
- friend_Fernando 2y agoLittle by little, we're getting an answer to the question: "What kind of investment does an outrageous influx of capitalization spur?" One might think it would be an AI silicon-moat, and it might yet be some of that. But it's clear that everyone's favorite goal is keretsuification. If you're looking for abnormal profits, you can't do better than to add a letter to FAANG. Nvidia already got into the cloud business, and now it's making workstations. The era of specialists doing specialist things is not really behind us. They're just not making automatic money, nor most of it. Nvidia excelled in that pool, but it too can't wait to leave it. It knows it can always fail as a specialist, but not as a kereitsu.
- ryao 2y agoThis looks like a successor to the Nvidia Jetson AGX Orin 64GB Developer Kit: https://www.okdo.com/wp-content/uploads/2023/03/jetson-agx-orin-64GB-developer-kit-datasheet-web-us.pdf https://www.okdo.com/wp-content/uploads/2023/03/jetson-agx-o... I wonder what the specifications are in terms of memory bandwidth and computational capability.
- kcb 2y agoHopefully, the OS support isn't as awful as the Jetson platforms usually are. Unless they change, you'll get 1 or 2 major kernel updates ever and have to do bizarre stuff like install a 6 year old Ubuntu on your x86 PC to run the utility to flash the OS.
- ryao 2y agoThe community likely will make instructions for installing mainstream Linux distributions on it.
- kcb 2y agoDoesn't really help though if it requires an nvidia kernel.
- ryao 2y agoThe Linux kernel license requires Nvidia to disclose their Linux kernel sources and Nvidia open sourced their kernel driver. That said, you can probably boot a Debian or Gentoo system using the Nvidia provided kernel if need be.
- bionade24 2y agoIt always has been the userspace of the Jetsons which was closed source and tied to Nvidia's custom kernel. I have not heard from people running Jetpack on a different userland than the one provided by Nvidia. Companies/Labs that update the OS don't care about CUDA, Nvidia contributes to Mesa support of the Jetsons and some only need a bit more GPU power than a RasPi.
- bobheadmaker 2y agoPricing seems off!
- poisonborz 2y agoWelcome to tomorrow's "personal" computer, a single unmodifiable SoC with closed source software stack.
- blackoil 2y agoThat is the PC of today and yesterday and yesteryears.
- papichulo2023 2y agoThis would be closer to a personal cloud. It's meant for other devices to connect with.
- otabdeveloper4 2y agoSo exactly like the PC of yesterday then?
- vegabook 2y agoLike a mac?
- blackoil 2y agoIs there any effort in local cloud computing? I can't justify $3000 for a fun device. But if all devices (6 phone, 2 iPads, a desktop and 2 laptops) in my home can leverage that for fast LLM, gaming, and photo/video editing, now it makes so much more sense.
- KeplerBoy 2y agoYou can just setup your local openAI like API endpoints for LLMs. Most devices and apps won't be able to use them, because consumers don't run self-hosted apps, but for a simple chatGPT style app this is totally viable. Today.
- papichulo2023 2y agoMost tools expose openai-like apis that you can easily integrate with.
- TiredOfLife 2y agoThat is literally how it was announced as. AI cloud in a box. That can also be used as Linux desktop.
- reissbaker 2y agoOpen WebUI, SillyTavern, and other frontends can access any OpenAI-compatible server, and on Nvidia cards you have a wealth of options that will run one of those servers for you: llama.cpp (or the Ollama wrapper), of course, but also the faster vLLM and SGLang inference engines. Buy one of these, slap SGLang or vLLM on it, and point your devices at your machine's local IP address. I'm mildly skeptical about performance here: they aren't saying what the memory bandwidth is, and that'll have a major impact on tokens-per-second. If it's anywhere close to the 4090, or even the M2 Ultra, 128GB of Nvidia is a steal at $3k. Getting that amount of VRAM on anything non-Apple used to be tens of thousands of dollars. (They're also mentioning running the large models at Q4, which will definitely hurt the model's intelligence vs FP8 or BF16. But most people running models on Macs runs them at Q4, so I guess it's a valid comparison. You can at least run a 70B at FP8 on one of these even with fairly large context size, which I think will be the sweet spot.)
- 2y ago
- sam_goody 2y agoSo, a company that doesn't feel like sharing all their secret sauce with Anthropic can run DeepSeek Coder on three of these for $9K, and it should be be more or less the same experience. Do I understand that right? It seems way to cheap.
- Havoc 2y agoSuch a setup would likely work - assuming DS3 support on software stack but wouldn’t be able to serve as many requests in parallel as a classic gpu setup. Main issue is the ram they’re using here isn’t the same as is in GPUs
- derbaum 2y agoI'm a bit surprised by the amount of comments comparing the cost to (often cheap) cloud solutions. Nvidia's value proposition is completely different in my opinion. Say I have a startup in the EU that handles personal data or some company secrets and wants to use an LLM to analyse it (like using RAG). Having that data never leave your basement sure can be worth more than $3000 if performance is not a bottleneck.
- sensesp 2y ago100% I see many SMEs not willing to send their data to some cloud black box.
- jckahn 2y agoExactly this. I would happily give $3k to NVIDIA to avoid giving 1 cent to OpenAI/Anthropic.
- originalvichy 2y agoEven for established companies this is great. A tech company can have a few of these locally hosted and users can poll the company LLM with sensitive data.
- lolinder 2y agoHeck, I'm willing to pay $3000 for one of these to get a good model that runs my requests locally. It's probably just my stupid ape brain trying to do finance, but I'm infinitely more likely to run dumb experiments with LLMs on hardware I own than I am while paying per token (to the point where I currently spend way more time with small local llamas than with Claude), and even though I don't do anything sensitive I'm still leery of shipping all my data to one of these companies. This isn't competing with cloud, it's competing with Mac Minis and beefy GPUs. And $3000 is a very attractive price point in that market.
- ynniv 2y agoI'm pretty frugal, but my first thought is to get two to run 405B models. Building out 128GB of VRAM isn't easy, and will likely cost twice this.
- bionade24 2y agoSmart move for Nvidia to subsidise their ARM CPU and platform business by selling a big GPU packet with a CPU that most users don't really care about. Even if the margin is less than selling the raw GPU power would be (which I doubt), it'll look good on the shareholders conference if other business segments go up steep, too.
- quick_brown_fox 2y agoHow about “We sell a computer called the tinybox. It comes in two colors + pro. tinybox red and green are for people looking for a quiet home/office machine. tinybox pro is for people looking for a loud compact rack machine.” [0] [0] https://tinygrad.org/#tinybox https://tinygrad.org/#tinybox
- kkzz99 2y agoThese look terrible. For 5 times the price you get worse performance.
- nilstycho 2y agoAre you comparing tinybox red with 738 FP16 TFLOPS at $15K to Project Digits with 1 FP4 PFLOP at $3K? Or did they announce the Project Digits FP16 performance somewhere?
- moffkalast 2y ago>tinybox >the size of several ATX desktops
- mft_ 2y agoThis was my second thought; while we don’t have full performance data, it’s probably a bad day for tinybox.
- loudmax 2y agoGoing by the specs, this pretty much blows Tinybox out of the water. For $40,000, a Tinybox pro is advertised as offering 1.36 petaflops processing and 192 GB VRAM. For about $6,000 a pair of Nvidia Project Digits offer about a combined 2 petaflops processing and 256 GB VRAM. The market segment for Tinybox always seemed to be people that were somewhat price-insensitive, but unless Nvidia completely fumbles on execution, I struggle to think of any benefits of a Tinygrad Tinybox over an Nvidia Digits. Maybe if you absolutely, positively, need to run your OS on x86. I'd love to see if AMD or Intel has a response to these. I'm not holding my breath.
- 2y ago
- treprinum 2y agoNvidia just did what Intel/AMD should have done to threaten CUDA ecosystem - release a "cheap" 128GB local inference appliance/GPU. Well done Nvidia, and it looks bleak for any AI Intel/AMD efforts in the future.
- deleted 2y ago[deleted]
- mft_ 2y agoI think you nailed it. Any basic SWOT analysis of NVidia’s position would surely have to consider something like this from a competitor - either Apple, who is already nibbling around the edges of this space, or AMD/Intel who could/should? be. It’s obviously not guaranteed to go this route, but an LLM (or similar) on every desk and in every home is a plausible vision of the future.
- iszomer 2y agoNvidia also brought Mediatek into the spotlight..
- stuaxo 2y agoOf course no chance of this with x86 because of market segmentation.
- TiredOfLife 2y agoNo chance because they don't have x86 license.
- sliken 2y agoThis Nvidia is quite a bit like the AMD Strix Halo, which has the same 128GB max memory, x86, unified memory, and a 256 bit wide bus. I've seen much hopeful speculation that the nvidia is 512 bits wide, but I'm dubious.
- openrisk 2y agoWill there be a healthy "personal AI supercomputer" economy to generate demand for this? (NB: spam generators are only a parasite on any digital economy, viable to the extend they don't kill the host). One can only wish for this, but Nvidia would be going against the decades-long trend to emaciate local computing in favor of concentrating all compute on somebody else's linux (aka: cloud).
- pimeys 2y agoNVIDIA has a robotics department where I could see this tech fitting pretty nicely. Also I consider this a dev board. Soon this tech will be everywhere, in our phones, computers... You could already plug that to your home assistant and have your own Star Trek computer you can ask questions from. And NVIDIA seems to know this is the future and they were the first in the market.
- nabla9 2y agoAmortized cost is 10 cents per petaflop hour if you run it 5-6 years 24/7. I'm including the cost of electricity. This is really game changer. They should make a deal with Valve to turn this into 'superconsole' that can run Half Life 3 (to be announced) :)
- jeleh 2y ago...but will it run DOOM?
- henearkr 2y agoIt's based on Mediatek CPU cores, so I am really pessimistic about their open source support... I'm bracing for a whole new era of unsufferable binary blobs for Linux users, and my condolences if you have a non-ultramainstream distro.
- magicalhippo 2y agoThe press release[1] says it's using NVIDIA's Grace CPU[1], as well as this: MediaTek, a market leader in Arm-based SoC designs, collaborated on the design of GB10, contributing to its best-in-class power efficiency, performance and connectivity. I assume that means USB and such peripherals is MediaTek IP, while the Blackwell GPU and Grace CPU is entirely NVIDIA IP. That said, NVIDIA hasn't been super-great with the Jetson series, so yeah, will be interesting to see what kind of upstream support this gets. [1]: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
- a_bonobo 2y agoThere's a market not described here: bioinformatics. The owner of the market, Illumina, already ships their own bespoke hardware chips in servers called DRAGEN for faster analysis of thousands of genomes. Their main market for this product is in personalised medicine, as genome sequencing in humans is becoming common. Other companies like Oxford Nanopore use on-board GPUs to call bases (i.e., from raw electric signal coming off the sequencer to A, T, G, C) but it's not working as well as it could due to size and power constraints. I feel like this could be a huge game changer for someone like ONT, especially with cooler stuff like adaptive sequencing. Other avenues of bioinformatics, such as most day-to-day analysis software, is still very CPU and RAM heavy.
- newsclues 2y agoIs this for research labs, health clinics, or peoples homes?
- a_bonobo 2y agoONT sells its smallest MinION to regular people, too. But Illumina's and ONT's main market is universities, followed by large hospitals
- evandijk70 2y agoThis is, at least for now, a relatively small market. Illumina acquired the company manufacturing these chips for $100M. Analysis of a genome in the cloud generally costs below $10 on general purpose hardware. It is of course possible that these chips enable analyses that are currently not possible/prohibited by cost, but at least for now, this will not be the limiting factor for genomics, but cost of sequencing (which is currently $400-500 per genome)
- mycall 2y agoThe bigger picture is that OpenAI o3/o4.. plus specialized models will blow open the doors to genome tagging and discovery, but that is still 1 to 3 years away for ASI to kick in.
- 2y ago
- fweimer 2y agoIf they end up actually shipping this, lots of people will buy these machines to get an AArch64 Linux workstation—even if they are not interested in AI or Nvidia GPUs. At $3,000, it will be considerably cheaper than alternatives available today (except for SoC boards with extremely poor performance, obviously). I also expect that Nvidia will use its existing distribution channels for this, giving consumers a shot at buying the hardware (without first creating a company and losing consumer protections along the way).
- kllrnohj 2y ago> At $3,000, it will be considerably cheaper than alternatives available today $3000 gets me a 64-core Altra Q64-22 from a major-enough SI today: https://system76.com/desktops/thelio-astra-a1-n1/configure https://system76.com/desktops/thelio-astra-a1-n1/configure And of course if you don't care about the SI part, then you can just buy that motherboard & CPU directly for $1400 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-ampere-altra-max-ampere-altra-processors/p/N82E16813140134 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-amp... with the 128-core variant being $2400 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-ampere-altra-max-ampere-altra-processors/p/N82E16813140135 https://www.newegg.com/asrock-rack-altrad8ud-1l2t-q64-22-amp...
- adrian_b 2y agoThat Altra may be a good choice for certain server applications, like a Web server, but when used as a workstation it will be sluggish, because it uses weak cores, with much lower single-threaded performance than the Arm cores used in NVIDIA Digits. For certain applications, e.g. for those with many array operations, the 20 cores of Digits might match 40 cores of Altra at equal clock frequency, but the cores of Digits are likely to also have a higher clock frequency, so for some applications the 20 Arm cores of Digits may provide a higher throughput than 64 Altra cores, while also having a much higher single-thread performance, perhaps about double. So at equal price, NVIDIA Digits is certainly preferable as a workstation instead of a 64-core Altra. As a server, the latter should be better.
- 2y ago
- Havoc 2y agoCan one game on it? If one can skip buying gaming rig with a 5090 with its likely absurd price then this 3k becomes a lot easier for dual use hobbyists to swallow Edit 5090 is 2k
- ThatMedicIsASpy 2y agoIt is not made for gaming and the form factor says a lot. With this being an option for AI there is less of a need to buy tons of nvidia RTX GPUs. The 5090 surprised me with the two slot height design while having a 575W power budget.
- Havoc 2y agoIt’s certain not its primary use, but it may still work well - powerful modern GPU anyway. And that may be enough that want to use it for gaming as secondary use
- sliken 2y agoI'm not sure it has a video out. There is a AI produced image of the digit and a monitor though.
- Havoc 2y agohmm...interesting point. Putting a screen next to your product that doesn't have video out would be quite disingenuous though. So I'd be surprised if it has zero output at all. ...I guess there is a risk that it has an output but it's more like a CPU iGPU style basic output rather than being powered by the main GPU.
- tim333 2y agoI've followed progress since Moravec's "When will computer hardware match the human brain?" since that came out in 1997. It starts: >This paper describes how the performance of AI machines tends to improve at the same pace that AI researchers get access to faster hardware. The processing power and memory capacity necessary to match general intellectual performance of the human brain are estimated. Based on extrapolation of past trends and on examination of technologies under development, it is predicted that the required hardware will be available in cheap machines in the 2020s. and this is about the first personal unit that seems well ahead of his proposed specs. (He estimated 0.1 petaflops. The nvidia thing is "1 petaflop of AI performance at FP4 precision"). (paper https://jetpress.org/volume1/moravec.pdf https://jetpress.org/volume1/moravec.pdf)
- deleted 2y ago[deleted]
- thntk 2y agoAnyone know if it can run training/fine-tuning and not just 4-bit inference? Does it support mixed precision training with either BF16 or FP16?
- tobyhinloopen 2y ago$3000 seems incredibly good value
- delegate 2y agoI think this is version 1 of what's going to become the new 'PC'. Future versions will get more capable and smaller, portable. Can be used to train new types models (not just LLMs). I assume the GPU can do 3D graphics. Several of these in a cluster could run multiple powerful models in real time (vision, llm, OCR, 3D navigation, etc). If successful, millions of such units will be distributed around the world within 1-2 years. A p2p network of millions of such devices would be a very powerful thing indeed.
- mycall 2y ago> A p2p network of millions of such devices would be a very powerful thing indeed. If you think RAM speeds are slow for the transformer or inference, imagine what 100Mbs would be like.
- ben_w 2y agoDepends on the details, as always. If this hypothetical future is one where mixtures of experts is predominant, where each expert fits on a node, then the nodes only need the bandwidth to accept inputs and give responses — they won't need the much higher bandwidth required to spread a single model over the planet.
- gavi 2y agothis image seems to be AI Generated - :-) https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/project-digits.png https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/... Source: https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
- diggan 2y agoAgree, unless I've missed some recent invention where keyboards now have two of either Enter/Backspace/Shift keys on the right side. Not sure if that isn't expected though? Likely most people wouldn't even notice, and the company can say they're dogfooding some product I guess.
- patrulek 2y ago> unless I've missed some recent invention where keyboards now have two of either Enter/Backspace/Shift keys on the right side It doesnt have to be two enter/backspace/shift. Keyboard layout seems to be almost identical to Azio L70 Keyboard (at least the keys).
- tsimionescu 2y agoThe keyboard layout seems perfectly reasonable, and rather common: from top to bottom, the rightmost column of keys after the letters would be backspace, |\, enter, shift, ctrl. On the left, mirrored, you have ~`, tab, caps lock, shift, ctrl. The sizes and shapes match many common keyboard layouts I've seen.
- paxys 2y agoThe text on the screen is an obvious giveaway.
- diggan 2y agoDamn, you're right. I didn't even consider looking at the monitor itself as "They can't be so lazy they don't even use a real screenshot" while faking the rest kind of makes sense, otherwise you need a studio setup. Never underestimate how lazy companies with a ~$3 trillion market cap can be.
- timmg 2y agoOne thing I didn't see mentioned: this would be a good motivation for Nvidia to release "open weights" models. Just like Mac OS is free when you buy a Mac, having the latest high-quality LLM for free that just happens to run well on this box is a very interesting value-prop. And Nvidia definitely has the compute to make it happen.
- deleted 2y ago[deleted]
- logicchains 2y agoThey did exactly this, announcing at the same event new Nemo models upcoming (a fine-tune of llama).
- swalsh 2y agoThey already do release open weight models, in this very keynote he released some of the bigget open weight models yet: https://huggingface.co/collections/nvidia/cosmos-6751e884dc10e013a0a0d8e6 https://huggingface.co/collections/nvidia/cosmos-6751e884dc1...
- deleted 2y ago[deleted]
- cess11 2y agoWith a bit of luck it'll mean some of the Jetson series will get cheaper. While I'm quite the "AI" sceptic I think it might be interesting to have a node in my home network capable of a bit of this and that in this area, some text-to-speech, speech-to-text, object identification, which to be decent needs a bit more than the usual IoT- and ESP-chips can manage.
- neom 2y agoIn case you're curious, I googled. It runs this thing called "DGX OS": "DGX OS 6 Features The following are the key features of DGX OS Release 6: Based on Ubuntu 22.04 with the latest long-term Linux kernel version 5.15 for the recent hardware and security updates and updates to software packages, such as Python and GCC. Includes the NVIDIA-optimized Linux kernel, which supports GPU Direct Storage (GDS) without additional patches. Provides access to all NVIDIA GPU driver branches and CUDA toolkit versions. Uses the Ubuntu OFED by default with the option to install NVIDIA OFED for additional features. Supports Secure Boot (requires Ubuntu OFED). Supports DGX H100/H200."
- yoyohello13 2y agoI wonder what kind of spyware is loaded onto DGX OS. Oh, sorry I mean telemetry.
- ZeroTalent 2y agoCybersecurity analysts check and monitor these things daily, and they are pretty easy to catch. Likely nothing malicious, as history shows.
- thunkshift1 2y agoCorrect, highly concerning.. this is totally not the case with existing os’s and products
- AtlasBarfed 2y agoNvidia optimize meaning non-public patches, a non-upgradable operating system like what happens if you upgrade with a binary blob Nvidia driver?
- wmf 2y agoYou can upgrade to a newer release of DGX OS.
- Abishek_Muthian 2y agoI'm looking at my Jetson Nano in the corner which is fulfilling its post-retirement role as a paper weight because Nvidia abandoned it in 4 years. Nvidia Jetson Nano, A SBC for "AI" debuted with already aging custom Ubuntu 18.04 and when 18.04 went EOL, Nvidia abandoned it completely without any further updates to its proprietary jet-pack or drivers and without them all of Machine Learning stack like CUDA, Pytorch etc. became useless. I'll never buy a SBC from Nvidia unless all the SW support is up-streamed to Linux kernel.
- lolinder 2y agoThis is a very important point. In general, Nvidia's relationship with Linux has been... complicated. On the one hand, at least they offer drivers for it. On the other, I have found few more reliable ways to irreparably break a Linux installation than trying to install or upgrade those drivers. They don't seem to prioritize it as a first class citizen, more just tolerate it the bare minimum required to claim it works.
- stabbles 2y agoNow that the majority of their revenue is from data centers instead of Windows gaming PCs, you'd think their relationship with Linux should improve or already has.
- lolinder 2y agoIt's possible. I haven't had a system completely destroyed by Nvidia in the last few years, but I've been assuming that's because I've gotten in the habit of just not touching it once I get it working...
- pplonski86 2y agoI got similar experience. I really prefer switch CUDA version with whole PC machine. What is more, the speed and memory of hardware improves quickly in time as well.
- 2y ago
- pizza 2y agoDoes this also answer the question "What am I supposed to do with my old 4090 and my old 3090 once I get a 5090?" ie can we attach them as PCIe hardware to Digits?
- gigatexal 2y agoWhat are the CPU specs? Idk about the GPU but a really fast ARM cpu and a ton of ram and it already runs Linux?!! If it’s competitive with the M chips from Apple this might be my next box.
- sliken 2y ago10 cortex-x925 and 10 cortex-a725 I believe.
- gigatexal 2y agoLikely not M class fast but not slouches? https://www.anandtech.com/show/21399/arm-unveils-2024-cpu-core-designs-cortex-x925-a725-and-a520-arm-v9-2-redefined-for-3nm- https://www.anandtech.com/show/21399/arm-unveils-2024-cpu-co...
- prollyjethi 2y agoNvidia could potentially bring us all the year of Linux Desktop.
- theptip 2y ago$3k for a 128GB standalone is quite favorable pricing considering the next best option at home is going to be a 32GB 5090 at $2k for the card alone, so probably $3k when you’re done building a rig around it.
- egorfine 2y agoThe press-release says "up to 128GB" while the price is a single figure of $3,000. So it won't be out of the real of possibility that the 128GB version would cost quite a bit more.
- mysteria 2y agoFrom what I've seen the general consensus is that the 128GB of memory is standard across all models, and that the price would vary for different storage and networking configurations. Their marketing materials say that "Each Project DIGITS features 128GB of unified, coherent memory and up to 4TB of NVMe storage." https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwell-on-every-desk-and-at-every-ai-developers-fingertips https://nvidianews.nvidia.com/news/nvidia-puts-grace-blackwe...
- egorfine 2y agoIndeed!
- lhl 2y agoThe memory bandwidth has not been announced for this device. It's probably going to be more appropriate to compare vs a 128GB M4 Max (410-546GB/s MBW) or an AMD Ryzen AI Max+ 395 (yes, that's its real name) at 256GB/s of MBW. The 5090 has 1.8TB/s of MBW and is in a whole different class performance-wise. The real question is how big of a model will you actually want to run based on how slowly tokens generate.
- elorant 2y agoWell obviously it has to be low otherwise they would cannibalize their high end GPUs.
- varelse 2y ago[dead]
- rapatel0 2y agoI'm buying one. It's cheaper then my 4090RTX+192GB of ram for more performance and model traning headroom. It's also probably a beast for data science workloads.
- smcl 2y agoDo I need a "personal AI supercomputer"?
- tmoneymoney 2y agoNo but you want one regardless
- gnatman 2y ago>> The IBM Roadrunner was the first supercomputer to reach one petaflop (1 quadrillion floating point operations per second, or FLOPS) on May 25, 2008. $100M, 2.35MW, 6000 ft^2 >>Designed for AI researchers, data scientists, and students, Project Digits packs Nvidia’s new GB10 Grace Blackwell Superchip, which delivers up to a petaflop of computing performance for prototyping, fine-tuning, and running AI models. $3000, 1kW, 0.5 ft^2
- DannyBee 2y agoDigits is petaflops of FP4, roadrunner is petaflops of FP32. So at least a factor of 8 difference, but in practice much more. (IE I strongly doubt digits can do 1/8th petaflop of FP32) Beyond that, the factors seem reasonable for 2 decades?
- dotancohen 2y agoWhy even use a floating point if you have only 4 bits? Models with INT8 features are not unheard of.
- cjbgkagh 2y ago1 sign and 3 exponent bits. AFAIK at the small number of bits it’s basically a teeny tiny look up table so you can precompute the table to be whatever math you want. Having an exponent instead of mantissa just means that the values that can be expressed are not linearly separated.
- dotancohen 2y ago> Having an exponent instead of mantissa just means that the values that can be expressed are not linearly separated. Being expressed as 1 10 100 1000 is not really different than being expressed as 1 2 3 4. There's still only four bits, i.e. 16 different possible values, no matter how we decide to express that in human-friendly terms.
- stassats 2y ago
- sabareesh 2y agoI am pretty sure memory bandwidth will be low it doesn't eat up their enterprise lineup. If we are luck we might get 512GB/S this is still half of 4090
- haunter 2y agoThe monitor is AI generated in the product photo.... Nvidia please https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/project-digits.png https://s3.amazonaws.com/cms.ipressroom.com/219/files/20250/...
- adam_arthur 2y agoFinally! First product that directly competes on price with Macs for local inferencing of large LLMs (higher RAM). And likely outperforms them substantially. Definitely will upgrade my home LLM server if specs bear out.
- sfrules 2y agoNewbie question: can you connect a RTX 5090 to the GB10 Digits computer?
- HumanifyAI 2y agoRunning large language models locally could be the next major shift in personal computing, similar to how GPUs transformed gaming. The privacy implications of not sending data to cloud services are significant.
- erikvanoosten 2y ago> It’s a cloud computing platform that sits on your desk … This goes against every definition of cloud that I know off. Again proving that 'cloud' means whatever you want it to mean.