5 ms·
Great news all-around for deep learning practitioners. Nvidia says memory bandwidth is "460GB/s," which will probably have the most impact on deep learning app
by cs702 10y ago
Great news all-around for deep learning practitioners.
Nvidia says memory bandwidth is "460GB/s," which will probably have the most impact on deep learning applications (lots of giant matrices must be fetched from memory, repeatedly, for extended periods of time). For comparison, the GTX 1080's memory bandwidth is quoted as "320GB/s." The new Titan X also has 3,584 CUDA cores (1.3x more than GTX 1080) and 12GB of RAM (1.5x more than GTX 1080).
We'll have to wait for benchmarks, but based on specs, this new Titan X looks like the best single GPU card you can buy for deep learning today. For certain deep learning applications, if properly configured, two GTX 1080's might outperform the Titan X and cost about the same, but that's not an apples-to-apples comparison.
A beefy desktop computer with four of these Titan X's will have 44 Teraflops of raw computing power, about "one fifth" of the raw computing power of the world's current 500th most powerful supercomputer.[1] While those 44 Teraflops are usable only for certain kinds of applications (involving 32-bit floating point linear algebra operations), the figure is still kind of incredible.
[1] https://www.top500.org/list/2015/06/?page=5 https://www.top500.org/list/2015/06/?page=5
- pavlov 10y agoA beefy desktop computer with four of these Titan X's will have 44 Teraflops of raw computing power, about "one fifth" of the raw computing power of the world's current 500th most powerful supercomputer. There have been calls that Moore's Law is dead for at least 10 years, and yet desktops keep catching up to supercomputing. I know this increase is not in single-threaded general-purpose computing power in the fashion of the old gigahertz race... But on the other hand, the scope of what's considered "general purpose" keeps expanding too. Machine learning may be part of mainstream consumer applications in 5 years.
- joelschw 10y agoIt already is - SwiftKey has been doing it right on my phone for a while!
- fnord123 10y ago>Machine learning may be part of mainstream consumer applications in 5 years. It doesn't seem likely since the Titan X pulls 250 watts. Maybe if we get asics for deep learning which consume less power; and a lot of research is done on deforesting trained networks so users can compute classifications more cost effectively, then sure.
- rasz_pl 10y agoMoore's Law IS dead for single core CPU performance.
- pavlov 10y agoYes. My point was that the scope of applications seems to be expanding (even on the consumer level) such that single-threaded performance on an x86-style instruction set and memory model isn't the end-all definition of computing power anymore.
- mrb 10y ago"Moore's Law IS dead for single core CPU performance." Moore's Law was never about single core CPU performance. It is about the number of transistors in a single chip; this has always been its definition.
- honkhonkpants 10y agoActually Moore's law is about the number of transistors you can etch for a single dollar.
- astrodust 10y agoMoore's Law: "Moore's law (/mɔərz.ˈlɔː/) is the observation that the number of transistors in a dense integrated circuit doubles approximately every two years." That's the definition I've always known. It has nothing to do with money or speed or performance, but usually these things are correlated.
- rasz_pl 10y agook, "Moore's Law benefits ENDED for single core CPU performance"
- dave_sullivan 10y agoMoore's law is more about what happens when you get a lot of people working on a clearly defined problem + network effects where a new breakthrough helps everyone else get nearer to another breakthrough. This is why you find Moores law in so many places. It studies rate of change in technological progress. In a particular area (eg, single core cpu performance) it eventually follows a sigmoid curve (where all hockey sticks must go) as lower hanging fruit exposes itself (eg, higher parrallelism and GPU computing). Kurzweil (say what you will) has written a lot on the topic.
- Cybiote 10y agoMoore's law talks about scaling transistor density at a minimal cost. Its colloquial understanding is actually a particular case of the Experience Curve Effect with an added bonus. What made transistors special and unique for a while was the huge benefit from Dennard Scaling. However, the days where we could expect a next version to be of much smaller components while much faster at same power levels, for the same or lower price, are gone. The experience and returns to scale are still plodding along but Dennard Scaling is gone and with it, its magic, vanished by physics.
- ralusek 10y agoMoore's law is a linear constant. Catching up to super computers could mean that progress on all fronts is slowing. Even if performance of super computers increases by 1.01x, and personal computers increase by 1.02x, that would still fit the bill of "catching up to supercomputers." Neither would be the required doubling required to fit the definition of Moore's law.
- astrodust 10y agoThe specs on this singular card would put it at the top of the Top 500 computer list from 2004: https://www.top500.org/lists/2004/06/ https://www.top500.org/lists/2004/06/ With four or six cards, as many systems built around this component are likely to be spec'd, you'd be able to build something that could fall in the top 5 in the last ten years.
- emn13 10y agoWell... this means you can train models that are very slightly larger in the same time. It wont scale linearly. And those slightly larger models will typically be a very, very, very tiny bit better - quality scales far below linearly. Calling this great news is exaggerating greatly.
- mrmondo 10y agoalso some times exaggerating == marketing
- cs702 10y agoGoing by the specs, I would expect that training current models on the new Titan X will be be 1.3x to 2.0x faster than with the older Titan X. In practice, this means deep neural nets that now take four weeks to train will take two or three weeks instead. That's great news.
- nl 10y agoBigger models don't (usually) mean dealing with higher quality images or something. They mean larger NN layers - and more of them - are able to fit in memory. This is important for image stuff, true, but it becomes really important for things like Memory Networks for NLP tasks and Neural Turing Machine-like architectures. Bigger networks mean they can "remember" further "backwards" in their dependencies, and so do things they can't do before. That's GREAT news. Also, the speed up is pretty significant. People train for weeks at the moment - a 10% speed up means often means savings days. This might not seem much, but it lets you iterate much, much quicker.
- dharma1 10y agomight be relevant if your model size is large - two gtx 1080's would give you twice the gpu RAM
- SEJeff 10y agoWith 4 of those in a desktop, you'd also have a serious cooling problem. Most likely, you'd burn them out. Note, I run many servers with > 4 of these types of GPUs in them for $day_job machine learning type of problems.
- etangent 10y agoLiquid cooling.
- lloyd-christmas 10y agoIt's nothing to write home about when it comes to long-running applications. I have 2x 980tis with liquid cooling, and it runs pretty damn hot when going for a week at 90%+ capacity. I can't use it in the summer months, even leaving my AC on while I'm at work.