3 ms·

The benchmarks where the 1080 doesn't even compete - FP16/Tensor Cores.

bitL 8y ago
Many state-of-art models won't train well on FP16. But for inferencing it's extraordinarily good. 2x1080Ti is the sweet spot for FP32 training on "budget" at the moment.
- jongomez 8y ago
  Got any sources? Was thinking about buying one just for the tensor cores, but if this is the case I probably won't.
  - bitL 8y ago
    You can even see it in author's comments in the original article: "When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the best use case. Non-linear optimization is just too susceptible to precision loss." There was also some NVidia video presentation recommending mixed FP32/FP16 training instead of pure FP16.
    - option 8y ago
      Mixed precision training can give you tensor core speedups. Paper: https://arxiv.org/abs/1710.03740 https://arxiv.org/abs/1710.03740 Toolkit which implements it on top of Tensorflow: https://github.com/NVIDIA/OpenSeq2Seq https://github.com/NVIDIA/OpenSeq2Seq
- IshKebab 8y ago
  "Inference" or "inferring". "Inferencing" isn't a word any more than "defencing" is.
  - Symmetry 8y ago
    "Inference" is a term of art in machine learning jargon so those replacements you propose have much broader meanings than the original and are not suitable replacements.
    - lettucehead 8y ago
      This is outrageous. Consider math terminology as a so-called "term of art." An attractor, in dynamic systems, is something which attracts. A repellor is something which repels. Should it be called a "repellence?" Wouldn't that mean something undesirable? Should the opposite of "attractor" be a "repugnor," because repugnance is the opposite of attraction? What is the corresponding correct form of something that repels? Repellor (or repeller, for those who prefer American suffixes). Forms matter. Colloquial meanings also matter, but not as much, particularly when they're an egregious violation of English and decency.

bitL 8y ago

Many state-of-art models won't train well on FP16. But for inferencing it's extraordinarily good. 2x1080Ti is the sweet spot for FP32 training on "budget" at the moment.

jongomez 8y ago
Got any sources? Was thinking about buying one just for the tensor cores, but if this is the case I probably won't.
- bitL 8y ago
  You can even see it in author's comments in the original article: "When I first looked at fp16 Inception3 was the largest model I could train. Inception4 blew up until I went back to fp32. Mixed precision needs extra care, scaling of gradients and such. Still I think it is a good thing. What I really want to test is model size reduction for inference with TensorRT targeted to tensorcores. I think that is probably the best use case. Non-linear optimization is just too susceptible to precision loss." There was also some NVidia video presentation recommending mixed FP32/FP16 training instead of pure FP16.
  - option 8y ago
    Mixed precision training can give you tensor core speedups. Paper: https://arxiv.org/abs/1710.03740 https://arxiv.org/abs/1710.03740 Toolkit which implements it on top of Tensorflow: https://github.com/NVIDIA/OpenSeq2Seq https://github.com/NVIDIA/OpenSeq2Seq
IshKebab 8y ago
"Inference" or "inferring". "Inferencing" isn't a word any more than "defencing" is.
- Symmetry 8y ago
  "Inference" is a term of art in machine learning jargon so those replacements you propose have much broader meanings than the original and are not suitable replacements.
  - lettucehead 8y ago
    This is outrageous. Consider math terminology as a so-called "term of art." An attractor, in dynamic systems, is something which attracts. A repellor is something which repels. Should it be called a "repellence?" Wouldn't that mean something undesirable? Should the opposite of "attractor" be a "repugnor," because repugnance is the opposite of attraction? What is the corresponding correct form of something that repels? Repellor (or repeller, for those who prefer American suffixes). Forms matter. Colloquial meanings also matter, but not as much, particularly when they're an egregious violation of English and decency.