6 ms·
This is really trippy. GPUs generally need a fast interface like PCI-E and won't work over a network like a disk. So to elastically provide GPUs over a rack ho
by arielweisberg 10y ago
This is really trippy. GPUs generally need a fast interface like PCI-E and won't work over a network like a disk.
So to elastically provide GPUs over a rack how does that work? How do you not have a ton of GPUs just sitting around due to the physical constraints of PCI-E given that you can attach GPUs to some common instance types at any time? How do you not run out of capacity and just have to say no?
- blackaspen 10y agoI was wondering this too. The best I could think is that they're trunking PCI-E over TCP or something? Or maybe all of their new instance types have something like Infiniband?
- sargun 10y agoYou can buy 10M PCIe cables commercially. In fact, some equipment use them for their backplane.
- wmf 10y agoOne thing you can do is put 4-8 GPUs in each server and just try to bin pack different customers properly. Or you could use PCIe switches and cables to dynamically reassign GPUs to different physical servers.
- MBCook 10y agoSome other sort of ultra-high bandwidth interconnect between instances such as infiband (or whatever, I don't know that space well)? (This is speculation/questioning, not implying knowledge. I noticed the placement of the ? makes that a bit ambiguous)
- wmf 10y agoPCIe is really the only option; the question is just whether it's in one box or cabled.
- MBCook 10y agoI was thinking that you could package the commands across some other wire to load on the GPU. As long as your dataset fits on GPU (so you don't have access main memory, which would be horrifically slow) seems like it might fit the GPGPU use case. I agree that external PCIe is the obvious good/best solution. I didn't realize that external PCIe switch boxes existed.
- unkoman 10y agoScale. AWS is huge.
- djb_hackernews 10y ago> How do you not run out of capacity and just have to say no? I don't know the answer to your other questions but EC2 can and often does refuse our run instance requests because they don't have the resources available in the AZ we've requested.
- homero 10y agoThat's only for graphics, for gpgpu people routinely downgrade the pcie to x1, etc
- tw04 10y agoYou can deploy PCIe with switches over distance and keep the GPUs in an external enclosure: http://www.dolphinics.com/products/pent-dxseries-dxs410.html http://www.dolphinics.com/products/pent-dxseries-dxs410.html http://www.dell.com/us/business/p/poweredge-c410x/pd http://www.dell.com/us/business/p/poweredge-c410x/pd *there are newer/better/faster versions of the above available, just two examples I had handy as my guess of how to make this happen. I'm confident that even if AWS is doing something similar it's being done on customized hardware.
- tachyonbeam 10y agoThe problem is that transferring data to/from GPUs is already problematic performance wise. I guess it depends on your application whether the data transfer rate hit is acceptable or not.
- zitterbewegung 10y agoLooks like the use case that they have in mind is focused on CAD / CAM. Don't see much mention about deep learning and I would guess that isn't being targeted at all.
- erichocean 10y agoI'm in the process of moving our local graphics workstations to GRID-powered workstations on AWS, with Raspberry Pi 3 "heads" and 1080p 60fps streams. The goal is to cut our capital costs, allow access to GPU-based graphics apps "anywhere", and host all of our virtual film production assets on AWS, where they are accessible for rendering, simulation, etc.
- arielweisberg 10y agoAs a gamer I was initially surprised that works. It makes sense if server side use cased are more batch oriented/compute intensive. Latency to start and stop "jobs" is critical in gaming as you are trying to hit a 60-144hz job time.