6 ms·
Inside Pascal: Nvidia 's Newest Computing Platform
- wyldfire 11y agoHalf precision ftw! ML is the use case they're designing for, but we all get to reap the benefits.
- pavlov 11y agoHasn't half precision (16-bit float) been in NVidia GPUs forever? I could swear it was available back in the very first shader-capable Geforce FX days already.
- wyldfire 11y agoIIRC what's new here is native support in the ALUs, etc. I think the older support was probably in software.
- bsprings 11y ago(Post author here.) Yes, FP16 has been supported in NVIDIA GPUs as a texture format "forever" -- since before it was incorporated into the IEEE 754 standard. Indeed what is new in GP100 is hardware ALU support (and note denormals are full speed, which is even more important for lower precision formats).
- gnuvince 11y agoPlease say it's programmable in Pascal.
- overcast 11y agoHush you!
- venomsnake 11y agoThat will be only for the Turbo models. It was awesome language.
- m_mueller 11y agoI'm not sure whether it was called Turbo Pascal, but we had this Pascal editor with built in 2D draw windows to learn programming in high school. I'm still looking for something equivalent (but maybe a bit more modern/portable) to give to my kid when he's showing some interest. Scratch is nice, but the visual programming becomes limiting very quickly. Is there anything like this today?
- nitrogen 11y agoThere's Processing and Processing.js, or you might be able to find a modern LOGO. Some terms to search for: "live coding", "turtle graphics".
- draven 11y agoOpenframeworks and Cinder are C++, so that's probably not fit for a first experience. Check out LÖVE: https://love2d.org/ https://love2d.org/ It's in Lua (so learning the language won't be a big part of the whole experience) and runs on many different platforms.
- m_mueller 11y agoLöve looks indeed awesome, thanks!
- 11y ago
- marmaduke 11y agoIs there a description somewhere without all the bla bla hype? Comparison with past architectures would also be welcome, also without hype.
- dr_zoidberg 11y agoWhat I got from the article: * ~5.5 TFLOPs on FP64 * "About 2x" performance on FP32 (so 11 TFLOPs) * "Up to 2x" performance on FP16 (compared to FP32, so about 22 TFLOPs). * FP16 is also aimed to neural nets training, because when the weights of the net are FP16, it's a more compact representation. * 3840 general purpose processors. * More/better texture units, memory units, etc. So it's not about raw power, but also about a better design. Guess that's about the important stuff. I just skimmed the article over the top, reading a bit here and there, but that seemed to be the most remarkable stuff.
- ghshephard 11y agoHow does performance like this compare to the TFLOPs on a supercomputer? I.E. how does one draw parallels to the performance you can get versus what we've been historically able to do with http://www.top500.org/statistics/perfdevel/ http://www.top500.org/statistics/perfdevel/?
- stuntprogrammer 11y agoPascal won't be cheap so comparing to a top-end Xeon E5 v4, it's about 7x the theoretical FP64 performance (assuming the Xeon is 2.2GHz * 22 cores * 8 avx pipes per * 2 for FMA per socket, given the price range). Similar story at FP32. However, the GPU wins on FP16. For historical performance, just pick one of the machines that did x teraflops. E.g. the first teraflop computer used ~6000 200MHz pentium pro chips around 1996.
- ghshephard 11y agoWhat I'm trying to figure out, is whether a teraflop directly comparable. That is, on the top 500 list, the first 4.9 Teraflop computer was in 2000, but does that mean that Pascal could provide performance similar to the Supercomputer on the LINPACK benchmark? In describing the benchmark, they say, In an attempt to obtain uniformity across all computers in performance reporting, the algorithm used in solving the system of equations in the benchmark procedure must conform to LU factorization with partial pivoting. In particular, the operation count for the algorithm must be 2/3 n^3 + O(n^2) double precision floating point operations. This excludes the use of a fast matrix multiply algorithm like "Strassen's Method" or algorithms which compute a solution in a precision lower than full precision (64 bit floating point arithmetic) and refine the solution using an iterative approach. So, to summarize, if in 2000 the fastest supercomputer on the planet ran at about 4.9 TFLOPs, does that mean, apples-apples on the LINPACK (and only the LINPACK), that Pascal today would outperform that Supercomputer?
- JustSomeNobody 11y ago300 Watts. Toasty.
- tomkinstinch 11y agoWow. It has 15.3 billion transistors. It's amazing we can buy something with that many engineered parts. Even if the transistors are the result of duplication and lithography, it's an astonishing number. Creating the mask must have taken a while. Does anyone know what the failure rate is for the transistors (or transistors of a similar production process)? Do they all have to function to spec for a GPU, or are malfunctioning transistors disabled or corrected? What does the QC process look like?
- djcapelis 11y agoExact failure and bin rates for most semiconductor companies are considered deep dark internal trade secret. Other than pure scale, yield rates are one of the biggest factors in semiconductor cost and profit margin. And the answer is it depends. If you lose some transistors, you expect to lose the entire chip. But the vast majority of the transistors on each chip are part of cache or many many duplicate GPU cores, which if they fail to pass tests, can be disabled or downclocked and then the chip is binned into the appropriate product line. With GPUs this is much easier than other types of chips, because the level of functional duplication that exists allows a lot of flexibility. If a core is bad, you use a different one, and GPU cores are small enough they'd be stupid not to put some spares on each chip. Same with memories. Generally one can safely assume: * Most chips that come off the line are binned into a lower category and do not function at max spec for everything, which is why the price jump is so high at the extreme upper end of a hardware series. * With ASIC lithography most transistor malfunction isn't correctable, you mostly have to either downclock (some types of faults) or disable (the rest) that piece. * Rates of transistor malfunction is still incredibly fucking amazingly phenomenally low. Like with 15B transistors on a chip, you have trouble affording a failure rate of even one in a billion. So your line has to be, as the kids say: on fleek.
- Salgat 11y agoIs there an approximation you can give, such as a magnitude? Is it around 1 transistor per million, per billion, per thousand?
- mtgx 11y agoIBM's Power9 and its future Power 3.0 ISA CPUs, which should increasingly focus on deep-learning/big data optimization combined with Nvidia's GPUs which will increasingly optimize for the same, should make an interesting match over the next 5+ years. On the gaming side, I do hope they continue to optimize for VR. I think AMD is even slightly ahead of them on that.
- timeu 11y agoSorry but does it run Crysis ? ;-) But seriously quite impressive piece of hardware.