19 ms·
Nvidia DGX GH200: 100 Terabyte GPU Memory System
- natas 3y agookay, but can it play crysis at 60fps
- mandeepj 3y agoI know my comment is not directly related to the post, but it's not completely unrelated. Nvidia's CEO gave a commencement speech recently. In that, he said: I contacted the CEO of Sega and suggested that they should find another partner. But I also needed Sega to pay us in full or Nivida would be out of business. Just like failures, there are lessons for learning behind every success. Does anyone here have any insight - how did Nvidia came out of the above embarrassing and incompetency phase, and became a path-breaking, and trend-setting power house?
- gen220 3y agoI don't have a direct answer, but the "Acquired" podcast did a two-part series on Nvidia in 2022Q1, it's a good/deep answer to why they are the way they are. part 1: https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993-2006 https://www.acquired.fm/episodes/nvidia-the-gpu-company-1993... part 2: https://www.acquired.fm/episodes/nvidia-the-machine-learning-company-2006-2022 https://www.acquired.fm/episodes/nvidia-the-machine-learning... In short, Jensen has an almost Elon-like appetite for "bet the company" tier risk. He's never been comfortable with a plateau, and is always looking for the next mountain to jump to, before the plate tectonics of the industry come around to form it. There aren't a lot of CEOs and companies that oversee a company or industry-transforming shift more than once, he's definitely in that camp. But that's a gross oversimplification, the story is more interesting. Check it out!
- mandeepj 3y agoThank you for your response and those links. It seems Jensen is either reserved or media shy. He does not do as many public appearance as his contemporaries like Musk/Jobs/Gates/Bezos. Also, there has not been any book on him. But, there are couple on the way - https://www.amazon.com/s?k=Jensen+Huang&ref=nb_sb_noss https://www.amazon.com/s?k=Jensen+Huang&ref=nb_sb_noss . I hope he'll choose to publish a 2k/3k pages biography. Love reading life stories of successful leaders.
- winstonprivacy 3y agoI can finally play Crysis on highest settings.
- lhl 3y agoI watched Huang's Computex presentation and was pretty impressed by the hardware they're launching, but it's worth noting some caveats. While it was announced to be in "full production," according to the blog post, DGX GH200 won't be available until the end of the year, which puts it about on the same timeline as AMD's delivery of MI300. Also, while the 1 exaFLOPS topline number is impressive, this has some asterisks. Each GH H100 GPU only does 34 TFLOPS of FP64 according to the data sheet. [1] At 256 nodes, this is a mere 8.7 petaFLOPS, or 0.0087 exaFLOPS. You only get to the 1 exaFLOPS number (from looking at the data sheet) if you are doing sparse FP8 Tensor Core FLOPS (3968 TFLOPS/GH, non-sparse is halved). It's worth keeping this in mind when comparing to something like Frontier (1.1 exaFLOPS) [2] or the upcoming El Capitan (expected 2 exaFLOPS) [3] - Top500 uses LINPACK, which benchmarks FP64 FLOPS. Of course, for AI training, FP8 or BF16 is probably the most relevant numbers for perf/W and perf/$... Frontier and El Capitan are each estimated to cost ~$600M, and while exact numbers weren't given, I'd expect a full 256-node DGX H200 to come in between $50-100M. AMD will be having a "Data Center and AI" event on June 13th, so we'll get to see soon how competitive they are (the announced MI300 specs is 24 x Zen 4 cores with a CDNA3 architecture that is 8X faster than MI250X (383 FP16/BF16 TFLOPS), so about 3000 TFLOPS, with 128GB of unified HBM3 on a 8196 bit bus (6.5TB/s theoretical), which is in the same ballpark as Grace Hopper - I think it'll mostly come down to software, but with drop-in PyTorch support, OpenAI's Triton, etc, I'm somewhat optimistic that it will be worth it for big players do to some software lift (that others can benefit from), if the cost competitiveness of the hardware is there. For details already made public, see: https://www.tomshardware.com/news/new-amd-instinct-mi300-details-emerge-debuts-in-2-exaflop-el-capitan-supercomputer https://www.tomshardware.com/news/new-amd-instinct-mi300-det... [1] https://resources.nvidia.com/en-us-dgx-gh200/nvidia-grace-hopper-superchip-datasheet https://resources.nvidia.com/en-us-dgx-gh200/nvidia-grace-ho... [2] https://en.wikipedia.org/wiki/Frontier_(supercomputer) https://en.wikipedia.org/wiki/Frontier_(supercomputer) [3] https://en.wikipedia.org/wiki/El_Capitan_(supercomputer) https://en.wikipedia.org/wiki/El_Capitan_(supercomputer)
- DiabloD3 3y agoShame they don't seem to be keeping pace with AMD, even after the Mellanox acquisition.
- ftxbro 3y agoimagine a beowulf cluster of them
- sheeshkebab 3y agoBut can it play Doom?
- zakki 3y agoYou mean millions of Doom?
- Xeoncross 3y agoAt this point, I'm convinced that someone is here which could make that cluster play trillions of Doom. I mean, just how many pregnancy tests worth of compute is this?
- robotnikman 3y agoWith each doom instance being played by an AI?
- WithinReason 3y agoMegadoom
- speed_spread 3y agoHell, with a developer emulation force of 13 MegaCarmacks it can rewrite 50,000 Doom per second! (But that number drops to only 10/s if the rewrites are in Rust)
- bogwog 3y ago> (But that number drops to only 10/s if the rewrites are in Rust) Because of the slow toolchain, or because of the trademark lawsuits?
- RandomBK 3y agoYes, you will be able to train an AI model that is capable of beating Doom using this machine.
- brrrrrm 3y ago
- deleted 3y ago[deleted]
- belltaco 3y agoChrome will still find a way to eat all that up and lag.
- boredemployee 3y agoBest comment of the day!
- parker_mountain 3y agoEpic meme sir, here's your updoot
- iaw 3y agoHow far are we from fully modeling the human brain? I mean besides an easy way to identify all the neuronal connections... This makes me feel like we're close to that one terrifying short-story.
- neurostimulant 3y agoIs human consciousness just a product of neural networks, or is there other additional mechanisms we're not aware of?
- ChatGTP 3y agoWouldn't modelling the human brain mean we'd be using less power? We're using brute force to try get similar results to what the brain does.
- TheAlchemist 3y agoYou would need to define what you mean by 'modeling the human brain'. If it means AGI or anything similar, then we're very far. To paraphrase an analogy I've heard somewhere (in similar context) - We're building better and better ladders, maybe even lifts with this last push in ML field. But the brain is on the moon - even the best lifts won't get us there.
- tbenst 3y agoWe are massively far away from modeling the human brain. First of all, no one can agree what level is necessary to model the brain, and that varies tremendously by scientific question. Personally, my lower limit would be something like the computational package Neuron which models voltages across axon compartments and distribution of ion channels, My upper limit confidence bound is we don’t care about anything subatomic. At the upper bound: In molecular dynamics, which is used extensively in modern day neuroscience to understand the function of ion channels and GPCRs, a single H100 can model 70ns/day of compute for 1M atoms. There are 8.64e+13 nanoseconds per day. There are ~10^26 atoms in a human brain. Therefore, an upper limit back of envelope is you need fewer than 10e+26 atoms / 10e+9 atoms * 8.64e+13 ns / 70 ns = 1.23e+29 H100 GPUs. Calculating the lower bound is more difficult, but let’s start by saying you can get away with a fp16 for each synapse. Storing the weights of that model for 100 trillion synapses is 200 Terabytes, and if you figure weight size * 4 or so to do anything useful then this is in spitting distance. Note that this example lower bound is massively less complex than the Neuron model I suggested, as the entire field of neuromodulators, homeostatic mechanisms, glia, and more are thrown out, which are all important for modeling how the brain works under certain computational regimes.
- ftxbro 3y agoI can't find how much it will cost or how much power it will use. I mean it will be a lot and maybe only Google and Microsoft and Facebook can afford it but I still want to know.
- virtuallynathan 3y agoI would guess about $100m and ~1.5-2MW
- Bedon292 3y agoThe DGX A100 was $200k at launch. I found a DGX H100 in the mid $300k area. And those are 8 GPU systems. So you need 32 of those, and each one will definitely cost more plus networking. Super low estimate would be $500k each for $16M total. But considering its moving from 98GB to 480GB RAM per GPU. Might be more like $1.5M per 8, round it to say $50M. And at 1/8th the power per GB, you have 700 Watts / 96GB / 8 * 480GB come to around 450 Watts per. And 115kw for the 256.
- ftxbro 3y agoWhat does this mean for the AI race? For example what if a newish company (newer than Google/Facebook/Microsoft/etc.) like Anthropic, Scale, Perplexity, or Stability is able to scrape together $5B USD funding and spend their hardware budget on these things. Say that can buy $1B of them and spend the rest on hackers and operating expenses (idk if that's realistic). So maybe they could purchase and operate like 20 of them. Say that they spend six months doing experimental things and then the next six months training their Tsar Model. If they follow the Chinchilla scaling laws and normal architectures, how good will these models be?
- bushbaba 3y agoFor AI race means it opens the door to a competitor. Could be AMD, Google, or Amazon. All which have offerings in this space. However while the hardware isn’t cheap, it’s still likely not a blocker. Costs do inhibitor more experimental research.
- 3y ago
- deleted 3y ago[deleted]
- coffeebeqn 3y agoThat’s very interesting - why such an increase in memory capacity? I hope they can translate this to cheaper cards too
- ttul 3y agoIt is surely driven by the gargantuan increase in demand for training and running massive models.
- jrk 3y agoIt's not an increase in capacity of per-GPU "GPU memory" (the HBM directly connected to the H100 here is up to 96GB, where the previous generation was 80GB), but rather reflects the product of two things: 1. Each node here is a more tightly-coupled CPU+GPU two-chip pairing, and the CPU side has a significantly larger pool of 480GB of LPDDR ("regular" RAM). So each GPU is part of a node that includes up to 480+96GB of total memory. 2. There are way more nodes: 256, up from 8.
- tacticus 3y ago`Nvidia discovers DMA`
- llm_nerd 3y agoThis device has a fully switched fabric allowing comms between any of the 256 "superchip" clusters at 900GB/s. That is dramatically faster than a direct host to GPU 32-lane PCI-E connection (which is crazy), and obviously dwarfs any existing machine to machine connectivity. The actual usability of shared memory across the array is improved significantly. I mean...nvidia has obviously been using DMA for decades. This isn't just DMA.
- deleted 3y ago[deleted]
- YetAnotherNick 3y agoWow, 480 GB per GPU! What happened to end of Moore's law? I hope this improvement translates to consumer GPUs as 24GB is a big limitation.
- bushbaba 3y agoMoores law stated transistor density per square inch roughly doubled every 2 years. I see no end in sight for that specific law. As we can always go vertically if needed. It’s also held through so far in 2020[1]. The folks who conflated moores law to also mean doubling of compute processing capabilities of a CPU double every 2 years were wrong. [1]https://en.wikipedia.org/wiki/Moore%27s_law https://en.wikipedia.org/wiki/Moore%27s_law
- karmasimida 3y agoSmaller transistor means lower energy consumption, going vertical won’t solve this. This ceiling will be hit much earlier than what process/technique allowed.
- bigmattystyles 3y agosmaller energy per transistor, but if you're packing more in the package, the package's consumption will grow up. Also, I think leakage current (and the heat that comes with it) goes up the smaller the feature size.
- valianteffort 3y agoNot to be pedantic but wouldn't stacking transistors have no effect on density per square inch? Since it would only increase density per cubic inch.
- bushbaba 3y agoI view it as if you cut 1 square inch of a motherboard. That the every 2 years you’d expect to see roughly double the number of transistors in that cut out piece. Scaling vertically would “technically” still meet the above.
- rektide 3y agoI really have to wonder if anyone can compete with this kind of systems integration capability. A core having 900GBps connectivity to the cluster memory at such relative low power is epic beyond words. 800Gbps ethernet across PCIe is uncompetitive in extreme. How the rest of the industry can respond is such a mystery. And will it be lone competitors, or will a new PC era be able to start, with an ecosystem of capabilities?
- briffle 3y agoI'm curious how you can keep that fed with data fast enough. What kind of interfaces to your network do you need to keep it busy and not just waiting on data.
- makestuff 3y agoI have to wonder why these engineers are not paid millions of dollars per year. As a lowly backend dev this seems so much more impressive than my new API that retrieves something from a database...
- akomtu 3y agoEngineers are really poor negotiators, probably because they neglect "people skills".
- 3y ago
- rvz 3y agoCelebrating new ways to further burn up the planet rather than discovering more efficient and better ways for training, inference and fine-tuning AI systems without needing to scale up more GPUs, TPUs, data centers and water for the same purpose. The end result of this announcement is another expensive system only available to the same incumbent of tech giants with tens of billions at their disposal.
- renewiltord 3y agoWe found a way: nuclear fission reactors. So that problem is solved.
- anaganisk 3y agoYet, the climate is still changing. Inflation is rising. The world definitely has advanced but never became a better place.
- renewiltord 3y agoThe world has never been better. This is an incredible time to be alive.
- gwoolhurme 3y agoI don't feel that way personally... I have Nth level anxiety, maybe even N+1 level anxiety about losing my job to AI. Maybe in the future whoever comes next can enjoy things, but this literally keeps me up at night. With talks of extinction, job loss, etc. I feel like I wish I wasn't alive at this time.
- rvz 3y ago> We found a way: nuclear fission reactors. Nope. I'm talking about efficient methods in training, inferencing and fine-tuning these AI models that doesn't require lots of data centers, TPUs, GPUs, etc. You're talking about something else. Petrol and diesel cars are already burning the planet, but the main difference is, that there are efficient alternatives available today like electric cars to use instead. AI (Deep learning) however, does not have any viable and efficient methods in training, fine-turning these AI models, at all [0] [1] and wastes a tremendous amount of resources, all to keep up with scalability. So that problem is still NOT solved after a decade of using GPUs, the wastage is getting worse. [0] https://gizmodo.com/chatgpt-ai-water-185000-gallons-training-nuclear-1850324249 https://gizmodo.com/chatgpt-ai-water-185000-gallons-training... [1] https://www.independent.co.uk/tech/chatgpt-data-centre-water-consumption-b2318972.html https://www.independent.co.uk/tech/chatgpt-data-centre-water...
- kqr2 3y agoHow does this compare to Cerebras? https://www.cerebras.net/ https://www.cerebras.net/
- modeless 3y agoI think this has a lot more memory than Cerebras. Their site doesn't say how much memory they can attach to each Cerebras chip and I've gotta imagine that's because it doesn't look good vs their competitors.
- mupuff1234 3y agoDid supercomputers ever produce something meaningful or did advancement usually come out of more "scrappier" setups? I remember hearing a lot about rankings of supercomputers, but less so about what they actually achieved.
- neurostimulant 3y agoWeather simulations and forecasting are very useful to society and practically almost all available public weather forecasting datasets were computed in some supercomputer cluster.
- faeriechangling 3y agoYes, especially when you account for shared university supercomputers, and especially when it took a supercomputer to do much of anything. Generally given there are a much larger number of less powerful computers, more accessible to much scrappier interests, one would expect more innovation to be done on them.
- why_only_15 3y agoSupercomputers exist in meaningful part to compensate for our lack of ability to do nuclear tests. This is why the national labs run them.
- anaganisk 3y agoGoogle might've been built on a laptop, but it can't scale on a laptop. Same applies to coding an algorithm on a scrappy setup, and then scaling it to sequence DNA or simulate a phenomenon.
- kortilla 3y agoNot sure what this means, because google effectively scaled on laptops (generic x86).
- why_only_15 3y agoIn my head the way I differentiate "supercomputers" (national labs) and "warehouse-scale computers" (google/amazon/azure) is: 1. workload for national labs this is mostly sparse fp64 in my understanding, for warehouse-scale computing is lots of integer work, highly branchy, lots of pointer chasing, stuff like that. 2. latency/reliability vs throughput warehouse-scale computing jobs often run at awful utilization, in the 5-20% range depending on how you measure, in order to respond to shocks of various kinds and provide nice abstractions for developers. fundamentally these systems are used live by humans and human time is very valuable so making sure it stays up always and returns quickly is paramount. In my understanding supercomputing workloads are much more throughput-oriented, where you need to do an enormous amount of computation to get some answer but it doesn't much matter whether the answer comes in one week or two weeks. 3. interconnect warehouse-scale computing workloads are mostly fairly separable and the place where different requests become intertangled is in the database. In the supercomputing world, in my understanding, there are often significant interconnect needs all the time, so extremely high performance networking is emphasized.
- valine 3y agoThere were rumors floating around that GPT-4 was going to be a 100 trillion parameter model. Those rumors seemed ridiculous in hindsight, but this announcement makes me rethink how ridiculous it really was. 100 Terabytes of GPU memory is exactly what you need to train that class of model. However, I’m not even sure enough text data exists in the world to saturate 100T parameters. Maybe if you generated massive quantities of text with GPT-4 and used that dataset as your pre-training data. Training on the entirety of the internet then becomes just another fine tuning step. The bulk of the training could be on some 400TB dataset of generated text.
- jandrese 3y agoI have to wonder how much improvement you would get with a 100 trillion parameter model. There seems to be diminishing returns in model size. That effort could almost certainly be better spent.
- andai 3y agoFor the numerically challenged like me: 100TB is 100 trillion bytes, giving you 1 byte per parameter at 100T params. LLaMA can apparently run quantized to 4 bits per param (not sure if worth it though), which would allow you to run a 200TB model on one of these cards if I'm understanding right.
- valine 3y agoThat would work for inference, but for efficient training you’d also want you training set to fit in memory.
- airgapstopgap 3y agoI think people talking about a 100T GPT didn't mean a dense transformer but some sort of extreme Mixture-of-Experts which is much more amenable to low-resource setups and complicates this discussion. In any case, it's almost certainly not bigger than 1T, even if it's not a dense transformer (PaLM-2 is and makes do with 340B, but it isn't exactly on par).
- renonce 3y ago
- thwoiu4o2i34234 3y agoThe trouble with all this is that people are forced to buy these insanely expensive systems merely for the benefit of fitting all that stuff in the vram even if they don't end up using the compute cores on these machines (which, let's be honest, aren't really all that better than the gaming gpus). The needs are more in line with consumer server hardware with user-choice on cpu/ram etc. Sounds to me like there's a market for disruption. Pity that the deep-learning community is under the choke-hold of nvidia's software.
- tzhenghao 3y agoEh, time will tell. Nvidia is killing it because they have "semi decent" software toolchains like CUDA when just about hardware player botched software. That said, a lot of interesting development on the XLA and PyTorch 2.0 sides that lower straight down to LLVM, bypassing Nvidia's CUDA moat today.
- tucnak 3y agoThe same is also true for https://github.com/ROCmSoftwarePlatform/rocBLAS https://github.com/ROCmSoftwarePlatform/rocBLAS and https://github.com/ROCmSoftwarePlatform/hipBLASLt https://github.com/ROCmSoftwarePlatform/hipBLASLt although the build stack, distribution— leaves a lot to be desired, and otherwise quite unstable.
- unwind 3y agoThis is awe-inspiring and almost scary, it's pretty much beyond my understanding how much data these systems are meant to process. What is also beyond me is how someone at Nvidia thinks that the label sequence "1.00E+2; 1.00E+3; 1.00E+4; 1.00E+5; 1.00E+6" for the vertical axis in "Figure 1" is more readable than "100; 1,000; 10,000; 100,000; 1,000,000" would have been. The latter is 5 chars less (total), even. Or, if exponential notation is important for the Big Serious Computing People, then perhaps they could have dropped the ".00" part from each value? Or, if I'm allowed to dream, gone with actual exponential notation?
- csomar 3y agoI'm not sure what's the problem with the exponential notation? It shows scale in order of magnitudes.
- csdvrx 3y ago> I'm not sure what's the problem with the exponential notation? Same. It's just more efficient and readable that counting the 0 while considering the culture bound digit group separator norms like thousands/millions 3,3 vs laks/crore 2,2,3 cf https://en.wikipedia.org/wiki/Indian_numbering_system?useskin=vector#Numbering_systems_in_Hindu_epics https://en.wikipedia.org/wiki/Indian_numbering_system?useski... Personally, I think it'd have been better to say the the .00 adding nothing: 1E2 1E3 etc would be far better.
- vasco 3y agoThe exponent number is the number of zeros, its way more readable and faster to interpret than counting zeros.
- PartiallyTyped 3y agoIt's easier to think in (possibly relative) orders of magnitude than with absolute numbers, instinctively it's what we do when we read large numbers.
- tuetuopay 3y ago
- fock 3y agoSo how exactly (in the technical sense) is this more energy efficient than both PCIe and Infiniband (which seems to be a claim somewhere too, together with the added bandwidth)? EDIT: so the whitepaper is surprisingly good for that (somehow all the articles are very weird...): https://resources.nvidia.com/en-us-grace-cpu/nvidia-grace-hopper https://resources.nvidia.com/en-us-grace-cpu/nvidia-grace-ho... - essentially they connect the GPUs with NVlink instead of PCIe (so, vertical integrators heaven) and then NVLink forms a separate interconnect for GPUs. So this is cool and essentially what Fujitsu, Google, ... have done for some time. A fun thing is, that they like to add up their nvlink-duplex bandwith and don't do for PCIe... (which then suddenly would have the same bandwith as the GPU-side). Still very cool to see the mainframe come back alive ... (it's a bit sad they bought Mellanox - monopolies are sad...)
- Aromasin 3y agoArticle written by marketing. White Paper written by apps engineering. You can always tell so much more about semiconductor products if you can get your hands on material written by engineers. The information hasn't been distilled through layers of internal training sessions.
- xipix 3y agoA large system, so much higher chance of something breaking. What happens if it loses a node or a link? Or some memory becomes unreliable? This thing needs some sophisticated fault tolerance.
- ioedward 3y agoNvidia's enterprise GPUs are surprisingly unreliable. Working on a 128 GPU A100 cluster on AWS, 1 would fail every few days. I didn't have any insight on whether it was a hardware or software failure.
- csdvrx 3y ago> Working on a 128 GPU A100 cluster on AWS, 1 would fail every few days Define "fail". > I didn't have any insight on whether it was a hardware or software failure. Have scripts check nvidia-smi for ECC errors and dmesg for devices dropping of the PCI bus. For the former, replace the card. For the later, just perform a device reset (a power toggle of the device and a rescan of the bus is often enough to be back online within 5 seconds)
- robotresearcher 3y agoWhat does an AWS user do with this advice?
- csdvrx 3y agoThey either figure out how to write scripts or ask AWS support how to get that done.
- prmoustache 3y agoWhy do we still call those processors GPU when they are not meant to process graphics?
- ru552 3y agoA GPU specializes in Vector math, which is what is used in gaming graphics. Hence the name Graphics Processing Unit and it's original use case. It just so happens that LLMs are powered by vector math much like graphics applications are.
- prmoustache 3y agoSo it should be called VMPU then, not GPU.
- tzhenghao 3y agoGood point. "AI accelerators" is a thing now that competes directly with "GPUs used for AI as its sole purpose".
- pixelpoet 3y agoCPUs are also arguably not the centre of processing in such systems.
- ioedward 3y agoThe technical term is GPGPU, general-purpose computing on GPUs, but I like to call them GPUs for short.
- moffkalast 3y agoSo, a GPPU.
- sour-taste 3y agoHave people experimented with distributed training of parts of the model to avoid needing these absolutely massive GPUs? Anyone have pointers to large scale distributed training done recently?
- fock 3y agothis is distributed training with RDMA-aware and directly interconnected GPUs?
- Aromasin 3y agoYes, certainly. One industry use-case that comes to mind if Baidu; white-paper link below [1]. Pretty much all the large model developers distribute their model training across hardware in some way, using a blend of GPU/TPU/FPGA accelerators across multiple CPU nodes. Moving all the data around is expensive though, in both power consumption and time, which is why NVIDIA's new system would be of interest. [1] http://research.baidu.com/Public/uploads/5e76df66c467b.pdf http://research.baidu.com/Public/uploads/5e76df66c467b.pdf
- sour-taste 3y agoThis is fantastic, thanks.
- wongarsu 3y agoThe issue is that distributed training needs high bandwidth and very low latency to be efficient. In a single computer you can fit about 8-10 GPUs, or if you go to extremes like in this system you might fit 16. To scale beyond that, you connect multiple computers in the same rack via Infiniband (a optical fibre network solution, the system in the article comes with a 400G Infiniband network adapter). But systems that can host many GPUs tend to be expensive, and electricity is expensive, so at scale the expensive GPUs make sense. For a homebrew solution you can stick four consumer GPUs in a case and might save a buck.
- nojito 3y agoYes. OpenAI is a heavy user of Ray.io https://thenewstack.io/how-ray-a-distributed-ai-framework-helps-power-chatgpt/ https://thenewstack.io/how-ray-a-distributed-ai-framework-he...
- deleted 3y ago[deleted]
- tombert 3y agoYet another thing to put on my wishlist for whenever I become an eccentric billionaire. I know that I will never be able to afford such a thing (or possibly even afford to power it for more than a few minutes), but a man can dream.
- bullen 3y agoThat RAM is distributed among the GPUs right? 256 x 450W = 115KW = 82MWh = $82.000 / month at peak EU costs this winter (which will be normal next winter) For what, something that gets everything wrong?
- smoldesu 3y agoIt's an agnostic system. They could use this thing for curing cancer or predicting earthquakes, but... just you watch and see what the Free Market uses it for.
- bullen 3y agoSeen how bad both those are doing and will do in the future I guess it's "progress". But correction to my comment above: These become one global memory... how slow it is and how many corruptions they will have is unknown but holy hell...
- maxdo 3y agoHow doe it compared with tesla dojo?