5 ms·
This article buys in in the hype. It reads like they just copied statements of the press release. The article on anandtech[0] is a lot better. Also, do not for
by onli 11y ago
This article buys in in the hype. It reads like they just copied statements of the press release. The article on anandtech[0] is a lot better.
Also, do not forget that benchmarks made by Nvidia mean nothing. How fast the cards really are will only be clear if independent people do real benchmarks. That's true for games, but it will also be true for all the speculation found here over the ML performance.
[0] http://anandtech.com/show/10304/nvidia-announces-the-geforce-gtx-1080-1070 http://anandtech.com/show/10304/nvidia-announces-the-geforce...
- ultramancool 11y agoAnandtech also has extremely good benchmarks, if you're ever looking to optimize bang for your buck when it comes to GPUs they have an excellent comparison tool.
- onli 11y agoI know, I use them in my hardware recommender ;) They also are very accessible under http://anandtech.com/bench/GPU15/1248 http://anandtech.com/bench/GPU15/1248, it is a nice tool.
- varelse 11y agoFair enough, but when neither Intel nor AMD is bothering to put out competitive software/hardware solutions for deep learning, which is apparently going to be a ~$40B market by 2020, I predict continued dominance here. That said, since AMD is getting beaten by a factor of 2-3 in performance while Intel's Accelerator/FPGA solutions are getting beaten by a factor of 5-10, AMD has a real opportunity to be the competitor to NVIDIA here. And despite questions about it, GTX 1080 will be a fine deep learning board, especially if the framework engineers get off their lazy butts and implement Alex Krizhevsky's one weird trick algorithm, an approach which allows an 8-GPU Big Sur with 12.5GB/s P2P bandwidth to go toe to toe with an 8-GPU DGX-1 on training AlexNet. http://arxiv.org/abs/1404.5997 http://arxiv.org/abs/1404.5997
- nl 11y agoPretty sure most frameworks support "one weird trick" style learning. Eg Caffe: https://github.com/BVLC/caffe/blob/master/docs/multigpu.md https://github.com/BVLC/caffe/blob/master/docs/multigpu.md
- varelse 11y ago"The current implementation uses a tree reduction strategy. e.g. if there are 4 GPUs in the system, 0:1, 2:3 will exchange gradients, then 0:2 (top of the tree) will exchange gradients, 0 will calculate updated model, 0->2, and then 0->1, 2->3." Nope... This is the stupid way... They should be using ring reductions, handily provided by the NCCL library: https://github.com/NVIDIA/nccl https://github.com/NVIDIA/nccl
- db1024 11y agoI wouldn't call it stupid. It was fine for when they wrote it. I would say their solution is a bit outdated though. I am a bit surprised that NVIDIA isn't allocating more engineering support to Caffe to make sure they support all of NVIDIA's latest features in a timely fashion. Torch at least tries to use nccl for nn.DataParallelTable by default.
- varelse 11y agoIMO no it wasn't. Ring reductions are the sort of things the MPI guys had a decade ago. This is (once again IMO) the sort of thing that happens when one doesn't do lit searches and then Juergen Schmidhuber has a temper tantrum when we all find out one of his students probably has a good claim to have previously invented something supposedly new. My heuristic whenever I think I've invented something is to aggressively exploit Google to prove me wrong. It usually does so, but sometimes in very unusual ways. Glad to hear about Torch though. Do they have automagic model parallelization (not data parallelization) as well?
- db1024 11y ago
- dang 11y agoOK, we've changed the URL to that from http://wccftech.com/nvidia-geforce-gtx-1080-launch/ http://wccftech.com/nvidia-geforce-gtx-1080-launch/. Thanks.