4 ms·
I feel like this is the card for the desktop deep learning scientist. The A100 is much more expensive, and are better for the datacenter and don't really suit
by binarymax 4y ago
I feel like this is the card for the desktop deep learning scientist. The A100 is much more expensive, and are better for the datacenter and don't really suit a home rig. The recent 4090's looked nice but don't support NVLink. For me, it's all about the RAM. 48GB is a dream.
- moyix 4y agoYep, it's targeted at the same kind of people who bought the A6000s (me).
- bitL 4y agoMe as well, but with 48GB of RAM what is the point? They should sell at least 96GB one. There is no pressure/enticement to upgrade from A6000 to be honest.
- moyix 4y agoAgreed (especially since I just got a third A6000 this week), but by the time they come out the faster training/inference time might be enticing to some. I would love to see an 80GB workstation card though...
- bick_nyers 4y agoThat all depends. For the price of the RTX 6000, you could purchase FOUR 4090.
- runnerup 4y agoYeah obviously better to go with 4090's if your model fits in the VRAM. But if the models you want to work with are larger than 24GB you'll need to spend the big money.
- monkmartinez 4y agoOr purchase ampere cards with NVLink and pool the memory.
- binarymax 4y agoBut the RTX 40 series doesn't support NVLink - so you can't tie the cards together to gain parallelism.
- deleted 4y ago[deleted]
- monkmartinez 4y agoWhere are you seeing that, I am unable to find this information.
- binarymax 4y agoFrustratingly I can't link directly to the info, but on this page, click 'Specs' and then the green button 'Full Specs' to get a modal: https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/rtx-4090/ https://www.nvidia.com/en-us/geforce/graphics-cards/40-serie... Here's the full table pasted: GPU Engine Specs: NVIDIA CUDA® Cores 16384 Boost Clock (GHz) 2.52 Base Clock (GHz) 2.23 Memory Specs: Standard Memory Config 24 GB GDDR6X Memory Interface Width 384-bit Technology Support: Ray Tracing Cores 3rd Generation Tensor Cores 4th Generation NVIDIA Architecture Ada Lovelace NVIDIA DLSS 3 NVIDIA Reflex Yes NVIDIA Broadcast Yes PCI Express Gen 4 Yes Resizable BAR Yes NVIDIA® GeForce Experience™ Yes NVIDIA Ansel Yes NVIDIA FreeStyle Yes NVIDIA ShadowPlay Yes NVIDIA Highlights Yes NVIDIA G-SYNC® Yes Game Ready Drivers Yes NVIDIA Studio Drivers Yes NVIDIA Omniverse Yes Microsoft DirectX® 12 Ultimate Yes NVIDIA GPU Boost™ Yes NVIDIA NVLink™ (SLI-Ready) No Vulkan RT API, OpenGL 4.6 Yes NVIDIA Encoder (NVENC) 2x 8th Generation NVIDIA Decoder (NVDEC) 5th Generation AV1 Encode Yes AV1 Decode Yes CUDA Capability 8.9 VR Ready Yes Display Support: Maximum Digital Resolution (1) 7680x4320 Standard Display Connectors HDMI(2), 3x DisplayPort(3) Multi Monitor 4 HDCP 2.3 Card Dimensions: Length 304 mm Width 137 mm Slots 3-Slot (61mm) Thermal and Power Specs: Maximum GPU Temperature (in C) 90 Graphics Card Power (W) 450 W Minimum System Power (W) (4) 850 W Supplementary Power Connectors 3x PCIe 8-pin cables (adapter in box) OR 450 W or greater PCIe Gen 5 cable
- bick_nyers 4y ago
- mrtranscendence 4y agoStupid question, maybe, but do you need NVLink for deep learning workflows?
- binarymax 4y agoNot a stupid question! NVLink is important if (1) if your model doesn't fit in the RAM on a single card or (2) you want to increase batch size past the point of each batch fitting in the RAM of a single card or (3) to increase training speed by spreading the workload.
- m0RRSIYB0Zq8MgL 4y agoI don't think this supports NVLink anymore. The product page for the RTX A6000 mentions NVLink but it is gone from the product page of the new RTX 6000. https://www.nvidia.com/en-us/design-visualization/rtx-a6000/ https://www.nvidia.com/en-us/design-visualization/rtx-a6000/ https://www.nvidia.com/en-us/design-visualization/rtx-6000/ https://www.nvidia.com/en-us/design-visualization/rtx-6000/
- monkmartinez 4y ago$4800 is what the previous RTX A6000 costs in the US after taxes. That is a BIG cost for anyone outside of a professional setting. You could buy 2 x 3090's with NVLINK for half that. I recently bought a datacenter M40 with 24GB of RAM for $150 on Ebay then added water cooling. The Dell Precision 5810 I bought second hand can now run some really interesting stuff on those 24GB of RAM albeit slower than a 3090. For less than $500 I can have two watercooled M40's that will run almost anything I can throw at them.
- bitL 4y ago2x3090 with NVLink isn't appearing as a single GPU even on Linux.
- monkmartinez 4y agoMy understanding may be flawed, but I am under the impression that 2 x GPUs with NVLink will pool memory. It does not change the underlying 2 x GPU architecture in a way that would represent 2x as a single when looking at the system config. Furthermore, I believe the application code needs to have NVLink enabled to take advantage of the pooled memory. Pytorch and Tensorflow have knobs to turn in order to take advantage of NVLink... where programs like DaVinci Resolve may need patching, etc.
- p1esk 4y ago2 x GPUs with NVLink will pool memory None of major ML frameworks such as Pytorch of TF support that. I’m not sure why.
- lostmsu 4y agoI think they both support data exchange over the NVLink via NCCL.
- p1esk 4y agoThat's different, NCCL manages traffic between GPUs, but each GPU (with its memory) is still treated as a separate entity by Pytorch or TF. Basically if your model does not fit in a single GPU memory, Pytorch will not automatically distribute it across two GPUs even if the GPUs are connected with NVLink. You can use model/tensor parallel methods to do this, but it's not going to be automatic (will require writing extra code).