10 ms·
2080 RTX performance on Tensorflow with CUDA 10
- lhlmgr 8y ago"I do like the RTX 2080Ti but I just love the Titan V! The Titan V is a great card and even though it seems expensive from a "consumer" point of view. I consider it an incredible bargain." .. quite hard to read as a phd student..
- _Wintermute 8y agoCompared to other equipment in research environments it's incredibly cheap.
- ekianjo 8y agoUsually you dont buy such equipment by yourself anyway.
- sigi45 8y agoAs a student, you get more free offers to use university ressources or something else than someone who just 'works'.
- aetimmes 8y agoConsidering the V100 runs around $10-11k and the Titan V provides similar performance for around $3k, the author isn't wrong.
- bitL 8y ago32GB vs 12GB. Enables a lot more. If you don't care about memory and FP64, 2080Ti would be a much better deal than Titan V. V100 vs Quadro RTX 8000 would be more interesting. Still, I think 2x1080Ti is a better deal than 1x2080Ti and costs the same.
- ZeroCool2u 8y agoFrankly it probably is, if only because you can do batch processing while still doing more experimental work on the other card.
- deleted 8y ago[deleted]
- infocollector 8y agoThe author recommends Titan V - without justifying its $3k price. The 1080 series is less than half that price with comparable benchmarks. Am I missing something?
- swerner 8y agoThe benchmarks where the 1080 doesn't even compete - FP16/Tensor Cores.
- bitL 8y agoMany state-of-art models won't train well on FP16. But for inferencing it's extraordinarily good. 2x1080Ti is the sweet spot for FP32 training on "budget" at the moment.
- jongomez 8y agoGot any sources? Was thinking about buying one just for the tensor cores, but if this is the case I probably won't.
- bitL 8y agoYou can even see it in author's comments in the original article: "When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the best use case. Non-linear optimization is just too susceptible to precision loss." There was also some NVidia video presentation recommending mixed FP32/FP16 training instead of pure FP16.
- option 8y agoMixed precision training can give you tensor core speedups. Paper: https://arxiv.org/abs/1710.03740 https://arxiv.org/abs/1710.03740 Toolkit which implements it on top of Tensorflow: https://github.com/NVIDIA/OpenSeq2Seq https://github.com/NVIDIA/OpenSeq2Seq
- shaklee3 8y agoAnother thing that's not clear from the benchmarks is the Titan has both more tensor cores, but also much higher memory bandwidth with hbm2. I'd be curious to see how much that affected the results compared to the number of cores. Also the 2080ti can do lower precision math (int8/4) in the tensor cores, while the Titan v cannot.
- mychael 8y agoDoes anyone know why they are using Xeon processors instead of the AMD Threadripper? Is it the support for ECC memory? If so, why is it that so important? Example: https://www.pugetsystems.com/nav/peak/tower_single/customize.php https://www.pugetsystems.com/nav/peak/tower_single/customize...
- zamadatix 8y agoThe Xeon they are using is 14 cores in a single NUMA node so maybe that's it since a 16 core Threadripper is 2 separate nodes in one die. I'm pretty sure Threadripper supports ECC: "With the most memory channels you can get on desktop, the Ryzen™ Threadripper™ processor can support Workstation Standard DDR4 ECC (Error Correcting Mode) Memory to keep you tight, tuned and perfectly in sync." from https://www.amd.com/en/products/ryzen-threadripper https://www.amd.com/en/products/ryzen-threadripper
- simcop2387 8y agoI can definitely confirm the ECC support on TR cpus. I've got one running right now and have had it report errors and corrections when trying to overclock things (I don't know what I'm doing with that so it's nice to have it warn me that I'm at the limit).
- mychael 8y agoHow does that warning surface? From the underlying operating system?
- jamesblonde 8y agoBecause it's a single-root pci complex. See here: https://www.servethehome.com/how-intel-skylake-sp-changes-impacted-single-root-pcie-deep-learning-servers/ https://www.servethehome.com/how-intel-skylake-sp-changes-im...
- scientist 8y agoCould you please explain your comment? The link that you have provided explains that with the new Intel Xeon Scalable generation it is difficult to implement single-root PCI complex on typically available motherboards, while according to [1] "the new Intel® Xeon® W processors are based on the Intel® Xeon® Scalable processor microarchitecture". Therefore, Intel Xeon W would have the same problems for supporting single-root PCI complex as the Xeon Scalable mentioned in the link you have provided. [1] https://www.intel.com/content/www/us/en/processors/xeon/xeon-scalable-w-workstation-brief.html https://www.intel.com/content/www/us/en/processors/xeon/xeon...
- bigmit37 8y agoAre there benefits to using FP32 vs FP16? I’ve been dabbling with deep learning but not really sure how much affect higher precision is having. Though more precision is better I suppose.
- bitL 8y agoTraditionally Deep Learning frameworks were all using FP32. With FP16 one can theoretically get 2x speed and 2x larger models with the same VRAM capacity. For inferencing with INT8/INT4 it can be even way better (good for embedded stuff). The downside is that sometimes more complex/deep models don't converge (or converge less often than FP32). Sometimes there are framework issues with some advanced FP16 stuff.
- visionscaper 8y agoFrom experience I know that models using RNNs have trouble training with FP16 precision. The common solution is to do training in FP32 and inference in FP16. To make this happen you often have to implement custom code (e.g. using Tensorflow or Keras as a meta framework)
- sabalaba 8y agohttps://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks/ https://lambdalabs.com/blog/2080-ti-deep-learning-benchmarks... These numbers match up with the performance that we’ve measured in our own tests that were posted last week. The Titan V is simply too expensive for Deep Learning. The 2080 TI is, by far and away, the best GPU from a price/performance perspective. As mentioned in the article, only possible reason that you might want a Titan V is if you care about FP64 performance: i.e., nobody training neural networks.
- visionscaper 8y agoAlthough the RTX 2080 Ti performs significantly better than the 1080 Ti, I'm still drawn towards the 1080 Ti; I can buy two second-hand 1080 Ti's for the price of one new 2080 Ti, providing me the double amount of memory, plus, the computing performance of 2x 1080 Ti is much better in FP32 than one 2080 Ti. I'm using my GPUs to train large sequence to sequence models (with long sequences) that need FP32 for training and can use FP16 for inference (mixed-precision training), so I can't even use the FP16 performance of the Tensorcores for training. The only disadvantage is that the energy costs are higher using two 1080 Ti's compared to one 2080 Ti.
- zrav 8y agoGranted, the benchmark only covers training, but for a chip that spends significant die space on dedicated AI circuitry the performance gain over the previous generation is disappointing.