18 ms·
Any EEs that can comment on at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory lives on a PCIe slot? It se
by SlightlyLeftPad 1y ago
Any EEs that can comment on at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory lives on a PCIe slot? It seems like that would also have some power delivery advantages.
- vincheezel 1y agoGood to see I’m not the only person that’s been thinking about this. Wedging gargantuan GPUs onto boards and into cases, sometimes needing support struts even, and pumping hundreds of watts through a power cable makes little sense to me. The CPU, RAM, these should be modules or cards on the GPU. Imagine that! CPU cards might be back..
- ksec 1y agoIt is not like CPU aren't getting higher wattage as well. Both AMD and Intel have roadmap for 800W CPU. At 50-100W for IO, this only leaves 11W per Core on a 64 Core CPU.
- linotype 1y ago800 watt CPU with a 600 watt GPU, I mean at a certain point people are going to need different wiring for outlets right?
- jchw 1y agoAt least with U.S. wiring we have 15 amps at 120 volts. For continuous power draw I know you'd want an 80% margin of safety, so let's say you have 1440 Watts of AC power you can safely draw continuously. Power supplies built on MOSFETs seem to peak at around 90% efficiency, but you could consider something like the Corsair AX1600i using gallium nitride transistors, which supposedly can handle up to 1600 watts at 94% efficiency. Apparently we still have room, as long as you don't run anything else on the same circuit. :)
- cosmic_cheese 1y agoWhere things get hairy are old houses with wiring that’s somewhere between shaky and a housefire waiting to happen, which are numerous.
- kube-system 1y agoYeah, but it ain't nothing that microwaves, space heaters, and hair dryers haven't already given a run for their money.
- jchw 1y agoHair dryers and microwaves only run for a few minutes, so even if you do have too much resistance this probably won't immediately reveal a problem. A space heater might, but most space heaters I've come across actually seem to draw not much over 1,000 watts. And even then, even if you do run something 24/7 at max wattage, it's definitely not guaranteed to start a fire even if the wiring is bad. Like, as long as it's not egregiously bad, I'd expect that there's enough margin to cover up less severe issues in most cases. I'm guessing the most danger would come when it's particularly hot outside (especially since then you'll probably have a lot of heat exchangers running.)
- jchw 1y agoAs an old house owner, I can attest to that for sure. In fairness though, I suspect most of the atrocities occur in wall and work boxes, as long as your house is new enough to at least have NM sheathed wiring instead of ancient weird stuff like knob and tube. That's still bad but it's a solvable problem. I've definitely seen my share of scary things. I have a lighting circuit that is incomprehensibly wired and seems to kill LED bulbs randomly during a power outage; I have zero clue what is going on with that one. Also, often times opening up wall boxes I will see backstabs that were not properly inserted or wire nuts that are just covering hand-twisted wires and not actually threaded at all (and not even the right size in some cases...) Needless to say, I should really get an electrician in here, but at least with a thermal camera you can look for signs of serious problems.
- 1y ago
- chronogram 1y agoThat's still not much for wiring in most countries. A small IKEA consumer oven is only 230V16A=3860W. Those GPUs and CPUs only consume that much at max usage anyway. And those CPUs are uninteresting for consumers, you only need a few Watts for a single good core, like a Mac Mini has.
- dv_dt 1y agoSo Europe ends up with an incidental/accidental advantage in the AI race?
- buckle8017 1y agoIn residential power delivery? yes In power cost? no I'm literally any other way? also no
- kube-system 1y agoConsumers with desktop computers are not winning any AI race anywhere.
- atonse 1y agoAll American households get mains power at 240v (I'm missing some nuance here about poles and phases, so the electrical people can correct my terminology). It's often used for things like ACs, Clothes Dryers, Stoves, EV Chargers. So it's pretty simple for a certified electrician to just make a 240v outlet if needed. It's just not the default that comes out of a wall.
- kube-system 1y agoTo get technical -- US homes get two phases of 120v that are 180 degrees out of phase with the neutral. Using either phase and the neutral gives you 120v. Using the two out of phase 120v phases together gives you a difference of 240v. https://appliantology.org/uploads/monthly_2016_06/large.57581ad8d9a5d_240VACinResidentialPowerSupplies.png.f176ab25d2092125b3d05d648597b1f8.png https://appliantology.org/uploads/monthly_2016_06/large.5758...
- orra 1y agoLaughs in 230V (sorry).
- AnthonBerg 1y agoʰᵉₕₑheʰᵉₕₑhe in 400V
- tracker1 1y agoThere already are different outlets for these higher power draw beasts in data centers. The amount of energy used in a 4u "AI" box is what an entire rack used to draw. Data centers themselves are having to rework/rewire areas in order to support these higher power systems.
- t0mas88 1y agoA simple kitchen top water cooker is 2000W, so a 1500W PC sounds like no big deal.
- kube-system 1y agoKettles in the US are usually 1500W, as the smallest branch circuits in US homes support 15A at 120V and the general rule for continuous loads is to be 80% of the maximum.
- linotype 1y agoTrue but kettles rarely run for very long.
- kube-system 1y agoBut computers do, which was why I included that context. You don't really want to build consumer PC >1500W in the US or you'd need to start changing the plug to patterns that require larger branch circuits.
- CyberDildonics 1y agoKettles and microwaves are usually 1100 watts and lower, but space heaters and car chargers can be 1500 watts and run for long periods of time.
- wtallis 1y agoMicrowave ovens top out around 1100-1250W output from a ~1500W input from the wall. Apparently there's a fair bit of energy lost in the power supply and magnetron that doesn't make it into the box where the food is.
- Tor3 1y agoMicrowave ovens have a different issue, which I found when I upgraded my breaker board to a modern one in my house. The startup pulse gives a type of load which trips a standard A-type 10A breaker (230V). Had to get those changed to a "slow" type, but even that will trip every blue moon, and if there's something else significant on the same circuit the microwave oven will trip even so, every two weeks or so (for the record, I have several different types of microwave ovens around the house, and this happens everywhere there's a 10A circuit). The newer circuits in the house are all 16A, but the old ones (very old) are 10A. A real pain, with new TN nets and modern breakers.
- triknomeister 1y agoAnd cooling. Look here: https://www.fz-juelich.de/en/news/archive/press-release/2025/jupiter-supercomputer-propels-european-computing-power https://www.fz-juelich.de/en/news/archive/press-release/2025... Especially a special PDU: https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-2025/04_jupiter_030-3000.jpg/@@images/image/larger https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-202... And cooling: https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-2025/10_jupiter_035-3000.jpg/@@images/image/larger https://www.fz-juelich.de/en/newsroom-jupiter/images-isc-202...
- 0manrho 1y agoThis is a legitimate problem in datacenters. They're getting to the point where a single 40(ish)OU/RU rack can pull a megawatt in some hyperdense cases. The talk of GPU/AI datacenters consuming inordinate amounts of energy isn't just because the DC's are yuge, (although some are), but because the power draw per rack unit space is going through the roof as well. On the consumer side of things where the CPU's are branded Ryzen or Core instead of Epyc or Xeon, a significant chunk of that power consumption is from the boosting behavior they implement to pseudo-artificially[0] inflate their performance numbers. You can save huge (easily 10%, often closer to 30%, but really depends on exact build/generation) on energy by doing a very mild undervolt and limiting boosting behavior on these cpus and keeping the same base clocks. Intel 11th through 14th gen CPU's are especially guilty of this, as are most Threadripper CPU's. you can often trade single digit or even negligible performance losses (depends on what you're using it for and how much you undervolt/underclock/restrict boosting) for double digit reductions in power usage. This phenomenon is also true for GPU's when compared across the enterprise/consumer divide, but not quite to the significant extent in most cases. Point being, yeah, it's a problem in data centers, but honestly there's a lot of headroom still even if you only have your common American 15A@120VAC outlets available before you need to call your electrician and upgrade your panel and/or install 240VAC outlets or what have you. 0: I say pseudo-artificial because the performance advantages are real, but unless you're doing some intensive/extreme cooling, they aren't sustainable or indicative of nominal performance, just a brief bit of extra headroom before your cooling solution heat-soaks and the CPU/GPU's throttle themselves back down. But it lets them put the "Bigger number means better" on the box for marketing.
- esseph 1y agoYes and this is something I've been thinking about for awhile. A computer is becoming a Home Appliance in the it will need 20A wiring and plugs soon, but should move to 220/240v soon anyway (and change the jumper on your standard power supply).
- nehalem501 1y agoIt is mostly an issue in countries with 120V mains (I know that in the US 240V outlets exist though). In France for example it is required that standard plugs must be able to deliver at least 16A on each outlet, at the 230V used here, we get 3600W of power, that’s more than enough.
- jacquesm 1y agoYou can up the voltage to 240 and re-use the wiring (with some minor mods to the ends), for double the power. Insulation class should be sufficient. That makes good sense anyway. You may still have an issue if the powersupply can't handle 240/60 but for most of the ones that I've used that would have worked. Better check with the manufacturer to be sure though. It's a lot easier and faster than rewiring.
- avgeek23 1y agoAnd the memory should be a onboard module on the cpu card intel/amd should replicate what apple did with a unified same ringbus sort of memory modules. Lower latency,higher throughput. Would push performance further. Although companies like intel would bleed the consumer dry with, a certain i5-whatever cpu with onboard memory of 16 gigs could be insanely priced compared to what you'd pay for addon memory.
- 0x457 1y agoThat would pretty much make both intel and amd to start market segmentation by CPU Core + Memory combination. I absolutely do not want that.
- sitkack 1y agohttps://en.wikipedia.org/wiki/Compute_Express_Link https://en.wikipedia.org/wiki/Compute_Express_Link
- derefr 1y agoBut all of the most-ridiculous hyperscale deployments, where bandwidth + latency most matter, have multiple GPUs per CPU, with the CPU responsible for splitting/packing/scheduling models and inference workloads across its own direct-attached GPUs, providing the network the abstraction of a single GPU with more (NUMA) VRAM than is possible for any single physical GPU to have. How do you do that, if each GPU expects to be its own backplane? One CPU daughterboard per GPU, and then the CPU daughterboards get SLIed together into one big CPU using NVLink? :P
- wmf 1y agoGPU as motherboard really only makes sense for gaming PCs. Even there SXM might be easier.
- mensetmanusman 1y agoIt’s always going to be a back and forth on how you attach stuff. Maybe the GPU becomes the motherboard and the CPU plugs into it.
- db48x 1y agoNo, for a gaming computer what we need is the motherboard and gpu to be side by side. That way the heat sinks for the CPU and GPU have similar amounts of space available. For other use cases like GPU servers it is better to have many GPUs for every CPU, so plugging a CPU card into the GPU doesn’t make much sense there either.
- verall 1y agoIf you look at a any of the nvidia DGX boards it's already pretty close. PCIe is a standard/commodity so that multiple vendors can compete and customers can save money. But at 8.0 speeds I'm not sure how many vendors will really be supplying, there's already only a few doing serdes this fast...
- Melatonic 1y agoIsnt it about latency as well with DGX boards? Vs PCI-E. You can only fit so much RAM on a board that will realistically be plugging into a slot
- snerbles 1y agoMost current DGX server assemblies are stacked and compression-fit, much higher density and more amenable to liquid cooling. https://www.servethehome.com/micron-socamm-memory-powers-next-gen-nvidia-grace-gb300-servers/ https://www.servethehome.com/micron-socamm-memory-powers-nex...
- eggsome 1y agoHas the DGX actually shipped anywhere yet?
- verall 1y agoDo you mean the new one? The older ones have been around for so long you can buy off-leases of them: https://www.etb-tech.com/nvidia-dgx-1-ai-gpu-server-2-x-e5-2698v4-2-2ghz-twenty-core-512gb-1-x-480gb-sata-ssd-4-x-1-92tb-sata-ssd-8-x-tesla-v100-32gb-gpus-svr-nvidiadgx-1-s.html https://www.etb-tech.com/nvidia-dgx-1-ai-gpu-server-2-x-e5-2...
- y1n0 1y agoThere are companies that specialize in memory controller ip that every one else uses, including large semi companies like Intel. The ip companies are the first to support new standards, make their money selling to intel etc. Allowing intel or whomever to take their time to build higher performance ip.
- MurkyLabs 1y agoYes I agree, let's bring back the SECC style CPU's from the Pentium Era, I've still got my Pentium II (with MMX technology)
- Razengan 1y agoIsn't that what has kinda sorta basically happened with Apple Silicon?
- MBCook 1y agoGPU + CPU on the same die, RAM on the same package. A total computer all-in-one. Just no interface to the world without the motherboard.
- trenchpilgrim 1y agoAnd AMD Strix Halo.
- Dylan16807 1y agoAnd limit yourself to only one GPU? Also CPUs are able to make use of more space for memory, both horizontally and vertically. I don't really see the power delivery advantages, either way you're running a bunch of EPS12V or similar cables around.
- burnt-resistor 1y agoFigure out how much RAM, L1-3|4 cache, integer, vector, graphics, and AI horsepower is needed for a use-case ahead-of-time and cram them all into one huge socket with intensive power rails and cooling. The internal RAM bus doesn't have to be DDRn/X either. An integrated northbridge would deliver PCIe, etc.
- kvemkon 1y ago> at what point do we just flip the architecture over so the GPU pcb is the motherboard and the cpu/memory Actually the RapsberryPi (appeared 2012) was based on a SoC with a big and powerful GPU and small weak supporting CPU. The board booted the GPU first.
- LeoPanthera 1y agoBring back the S100 bus and put literally everything on a card. Your motherboard is just a dumb bus backplane.
- MBCook 1y agoWe were moving that way, sorta, with Slot 1 and Slot A. Then that became unnecessary when L2 cache went on-die.
- leoapagano 1y agoOne possible advantage of this approach that no one here has mentioned yet is that it would allow us to put RAM on the CPU die (allowing for us to take advantage of the greater memory bandwidth) while also allowing for upgradable RAM.
- MBCook 1y agoCouldn’t we do that today if we wanted to? What’s keeping Intel/AMD from putting memory on package like Apple does other than cost and possibly consumer demand?
- iszomer 1y agoSupply + demand, the manufacturing-capacity rabbit hole.
- themafia 1y agoI think you'd want to go the other way. GPU RAM is high speed and power hungry. So there tends to not be very much of it on the GPU card. This is part of the reason we keep increasing the bandwidth is so the CPU can touch that GPU RAM at the highest speeds. It makes me wonder though if a NUMA model for the GPU is a better idea. Add more lower power and lower speed RAM onto the GPU card. Then let the CPU preload as much data as is possible onto the card. Then instead of transferring textures through the CPU onto the PCI bus and into the GPU why not just send a DMA request to the GPU and ask it to move it from it's low speed memory to it's high speed memory? It's a whole new architecture but it seems to get at the actual problems we have in the space.
- pshirshov 1y agoCan I just have a backplane? Pretty please?
- colejohnson66 1y agoSockets (and especially backplanes) are absolutely atrocious for signal integrity.
- vFunct 1y agoVMEBus for the win! (now VPX...)
- pezezin 1y agoThe hot stuff nowadays is µTCA: https://www.picmg.org/openstandards/microtca/ https://www.picmg.org/openstandards/microtca/
- crimony 1y agoIf I remember correctly the military / aerospace shy away from this spec because the connector with the pins is on the backplane, with the sockets on the cards. So if you incorrectly insert a card and bend a pin you're in trouble. VPX has the sockets on the backplane so avoids this issue, if you bend pins you just grab another card from spares. This may have changed since I last looked at it. Telecoms industry definitely seem to favour TCA though.
- dylan604 1y agoWouldn't that mean an complete mobo replacement to upgrade the GPU? GPU upgrades seem much more rapid and substantial compared to CPU/RAM. Each upgrade would now mean taking out the CPU/RAM and other cards vs just replacing the GPU
- p1esk 1y agoGPUs completely dominate the cost of a server, so a GPU upgrade typically means new servers.
- BobbyTables2 1y agoAgree - newer GPU likely will need faster PCIe speeds too. Kinda like RAM - almost useless in terms of “upgrade” if one waits a few years. (Seems like DDR4 didn’t last long!)
- chrismorgan 1y ago> GPU upgrades seem much more rapid and substantial compared to CPU/RAM. I feel like I’ve been hearing about people selling five-to-ten-year-old GPUs for sometimes as many dollars as they bought them for, for the last five years; and people choosing to stay on 10-series NVIDIA cards (2016) because the similar-cost RTX 30-, 40- or 50-series was actually worse, because they’d been putting the effort and expense into parts of the chips no one actually used. Dunno, I don’t dGPU.
- deleted 1y ago[deleted]
- 0manrho 1y agoWe're already there. That's what a lot of people are using DPU's are for. An example, This is storage instead of GPU's, but as the SSD's were PCIe NVMe, it's pretty nearly the same concept: https://www.servethehome.com/zfs-without-a-server-using-the-nvidia-bluefield-2-dpu-nvme-arm-aic-iscsi/ https://www.servethehome.com/zfs-without-a-server-using-the-...
- undersuit 1y agoTo continue the ServeAtHome links, https://www.servethehome.com/microchip-adaptec-smartraid-4300-a-new-era-of-nvme-raid-controller-without-drive-connectivity/ https://www.servethehome.com/microchip-adaptec-smartraid-430... PCI-e Networks and CXL are the future of many platforms... like ISA backplanes.
- 0manrho 1y agoYep, I have a lot of experience with CXL devices and networked PCIe/NVMe (over Eth/IB) Fabrics and deploying "Headless"/"Micro-Head" compute units which are essentially just a pair of DPU's on a PCIe multiplexer (basically just a bunch of PCIe slots tied to a PCIe Switch or two). That said my experience in this field is more with storage than GPU compute, but I have done some limited hacking about in the GPGPU space with that tech as well. Really fascinating stuff (and often hard to keep up with and making sure every part in the chain supports the features you want to leverage, not to mention going down the PCIe root topology rabbit hole and dealing with latency/trace-length/SnR issues with retimers vs muxers vs etc etc etc). It's still a nascent field that's very expensive to play in, but I agree it's the future of at least part of the data infrastructure field. Really looking forward to finally getting my hands on CXL3.x stuff (outside of a demo environment.)
- coherentpony 1y agoThe concept exists now. You can "reverse offload" work to the CPU.
- bgnn 1y agoEE here. There's no reason to not deliver power directly to the GPU by using cables. I'm not sure if it's sooving anything. But you are right, there's no hierarchy in the systems anymore. Why do we even call something a motherboard? There's a bunch of chips interconnected.
- j16sdiz 1y agoIt is not a EE problem. It is an ecosystem problem. You need a whole catalog of compatible hardware for this.
- iszomer 1y agoI wonder how many additional layers are required in the PCB to achieve this + how this will dramatically affect the TDP; the GPU's aren't the only components with heat tolerance and capacitance.
- ratmeadow 1y agoPersonally I hope this point comes after we realise we don't need 1kW GPUs doing a whole lot of not much useful