16 ms·

The Age of Nvidia

213 points by douche 9y ago

sensecall 9y ago
Cached: http://webcache.googleusercontent.com/search?q=cache:JDuUy_X4iUcJ:www.alexstjohn.com/WP/2017/06/10/the-age-of-nvidia/ http://webcache.googleusercontent.com/search?q=cache:JDuUy_X...
0x4f3759df 9y ago
The youtube of "NVidia GTC 2017 conference" is worth a look.
- visarga 9y ago
  For a moment they made me feel like it was 2007 again and I was watching an Apple keynote. I didn't expect to have that feeling again from any company.
  - 0x4f3759df 9y ago
    Elon Musk was at the 2015 conf, so you might like that too.
ethbro 9y ago
In the DirectX comments, I'm struck by how interesting the strategic development of Intel and Microsoft is. I'd never thought about it before, but there aren't many industries that could instantly (2-4 years) be killed solely by technical development. If Intel had developed an OS-independent abstraction layer for devices, if Microsoft had pushed harder on ISA-independent programs, etc, how would today look?
- Symmetry 9y ago
  Oh, Intel was certainly a contributor to Linux and Microsoft flirted with having Windows run on other hardware. But backwards compatibility prevented either from commoditizing their complement on the desktop.
  - AceJohnny2 9y ago
    Talk about co-dependent relationships...
  - tachyonbeam 9y ago
    FYI, Microsoft now has an emulator to run x86 programs on ARM. This is for future laptops/tablet products.
- mtgx 9y ago
  There is a reason this partnership/cartel has been called "Wintel." Both knew it's in their best interest not to screw each other over by adopting other platforms more aggressively. I think they've been increasingly more dissatisfied with that arrangement since Intel was trying to get into iPads/iPhones, and then Microsoft built Windows RT and Windows Phone for ARM-only. Similarly, Google has been pushing Intel chips in Chromebooks (even though Chrome OS is architecture-agnostic, as you were hoping Microsoft to do with Windows), because they get those sweet discounts and early-access to Intel server chips. Plus, I think Intel is still on Google's board.
  - StillBored 9y ago
    Well if google is pushing "intel" in chrombooks, it still doesn't explain why the intel chrombooks tend to have far more ram/disk space/etc than the ARM ones. Side by side, the ARM chrombooks seem like phones with big screens while the intel devices appear to be inexpensive ultrabooks.
Apreche 9y ago
Knowing this, what should a developer learn if they want to get a job in the brave new non-wintel world? ARM? CUDA?
- zanny 9y ago
  Knowing the architecture of ARM is a perk, but the software is going to be all CUDA / OpenCL / Vulkan Compute - the CPU side is just going to be a shim layer to get kernels to execute on that graphics card and won't take much development and will hopefully just be the same open standard everyone uses.
  - visarga 9y ago
    Yep. Same programming. Details hidden by compiler.
    - sqeaky 9y ago
      I agree with you, except when they aren't hidden... For some large percentage of the problems we fix I agree with you, but every once in a while you need to peel back a layer or two of abstraction and ask "Why did it do that?" Most abstractions leak, despite whatever heavy lifting they do for us. Having familiarity with those underlying implementations can make that trouble-shooting much easier. So I guess it is up to each developer to understand how likely they are to peel back different abstractions. I suspect a JS dev will rarely want to look at the assembly and C/C++ might want to every week.
- golergka 9y ago
  Most of developers are not working with code that requires top-notch computer power; I don't think that Nvidia/Intel rivalry will affect the market for people who know JS, CSS and HTML to a significant degree. But generally speaking, learning Caffe, Tensorflow and math in general sounds like a much more important investment than low-level libraries for specific hardware.
  - pm90 9y ago
Pigo 9y ago
All of this did give me some insight into why I see ARM-based Windows servers in Azure.
flipgimble 9y ago
" 2017, the year GPU’s finally begin to permanently displace the venerated x86 based CPU" he never specifies what he means by displace exactly, and follows up with vacuous statements like "x86 party ends in 2017". The entire thing sounds like trying to manufacture a momentous event out of a gradual shift of market profits and priorities over time. Best you could hope for is to state that Intel has failed to take a significant market in high end throughput computing (ie. desktop and HPC GPUs), but its doing quite well with integrated GPUs. Don't forget this is the same visionary that wrote the game industry playbook on how to hire and exploit developers: https://www.kotaku.com.au/2016/04/alex-st-johns-ideas-about-game-development-are-terrifying/ https://www.kotaku.com.au/2016/04/alex-st-johns-ideas-about-...
- ballenf 9y ago
  Even gradual events have recognizable tipping points or points of no return. I think that is what he is calling 2017 for the gradual transition from CPU dominance to the future of GPU/FPGA dominance. Or just CPU irrelevance. Don't know how that article you link has any relevance to the facts and analysis he presents here.
- gambiting 9y ago
  It's the same as calling 20XX "year of linux" for the n-th time. Sure there is a gradual shift and things are getting better, but X86 is not dead, just like windows is still doing fine.
  - II2II 9y ago
    If you are trying to compare it to the year of Linux, it would be wise to look at how the market changed in general rather than how the market changed for Microsoft. Linux did not decimate Windows, but it has certainly hurt other platforms and it has likely hurt Microsoft's reach. Likewise, I doubt that GPGPU applications will decimate Intel. It will likely hurt Intel's reach. This is particularly true if AMD, ARM, and IBM are facilitating the use of to GPU as a high performance coprocessor.
- lnanek2 9y ago
Hydraulix989 9y ago
Uh... GPUs and CPUs fill very different niches. GPUs aren't going to _replace_ CPUs any time soon because GPUs aren't capable nor designed for general-purpose processing. It's a fundamentally REALLY HARD problem to use GPUs to speed up any arbitrary computations (solve it and you'll be a shoo-in for the Turing Award). Businesses aren't going to be "moving" to GPUs. The "age of NVIDIA" is primarily predicated on its role in accelerating training the machine learning algorithms hyped up as "deep learning." And AMD still very much has an uphill battle on both the CPU and the GPU fronts. But yes, the excitement around Intel that used to be there is now gone. You can probably blame the "death of the PC" -- we teach kids coding in the Bay Area, and every single kid has an iPad tablet, but not a laptop. Sure professional engineers like us still have x86 laptops, but the average person does not.
- visarga 9y ago
  > Uh... GPUs and CPUs fill very different niches. I think they actually mean CPU = Intel, GPU = NVIDIA because strictly speaking Intel is also a GPU manufacturer, albeit low end GPUs. NVIDIA will play a major role in AI and simulation, that is clear.
- ethikal 9y ago
  It's not just about accelerating ML, specifically deep learning. There are many other enterprise technologies that can benefit from GPUs. One example: OLAP-focused databases (such as MapD - https://www.mapd.com/ https://www.mapd.com/). For some benchmarks, check out this blog: http://tech.marksblogg.com/benchmarks.html http://tech.marksblogg.com/benchmarks.html. The DL "training" use-case is well-known at this point, but there are many others which are emerging.
  - deleted 9y ago
    [deleted]
  - jhj 9y ago
    A GPU database isn't that useful, because the arithmetic intensity (ops/byte) is relatively low. Cross-sectional memory bandwidth is what really matters; you can get similar effects with a cluster of CPU machines provisioned appropriately, with a shard or a replica of the database on each CPU machine. I say this as someone who has written a GPU in-memory database of sorts that is used at Facebook (Faiss), but what is interesting if you can tie that to something that has higher arithmetic intensity before or after the database lookup on the GPU. GPUs are only really being used for machine learning due to the sequential dependence of SGD and the relatively high arithmetic intensity (flops/byte) of convolutions or certain GEMMs. The faster you can take a gradient descent step means the faster wall clock time to converge, and you would lose by limiting memory reuse (for conv/GEMM) or on communication overhead or latency if you attempt to split a single computation between multiple nodes. The Volta "tensor cores" (fp16 units) make the GPU less arithmetic bound for operations such as convolution that require a GEMM-like operation, but the fact that the memory bandwidth did not increase by a similar factor means that Volta is fairly unbalanced. The point about Intel not increasing their headline performance by as much as GPUs is also misleading. Intel CPUs are very good at branchy codes and are latency optimized, not throughput optimized (as far as a general purpose computer can be). Not everything we want to do, even in deep learning, will necessarily run well on a throughput-optimized machine.
digitalzombie 9y ago
I always thought Microsoft keeping dominance via video game and directx was their main drive to keep Linux out. Didn't realize directx was their move to fight against Intel. The unintentional effect of this was GPU manufacturer florish, which is something I didn't know either. The two big monopoly fighting against each other is super interesting. Also highlight the fact that we do need some diversity and competition in these segment. Rooting for AMD on the GPU front to give Cuda a good challenge and also on the Intel front for their CPU.
- koffiezet 9y ago
  > and directx was their main drive to keep Linux out OS/2 Warp could have been a target, but Linux was nowhere near a threat to Microsoft when they came up with DirectX...
frozenport 9y ago
This post doesn't mention the strategic dilemma Intel has with GPUs. GPUs are fast because they have fast local memory, the core count is secondary, and some Intel chips have 22x16 vector processing elements clocked 2x higher than NVIDIA ( around 3k CUDA core equivalent ). Why doesn't Intel make a fast local memory GPU style device. Well, they did (Xeon Phi) but if they give it a reasonable price point it would cannibalize their existing x86 market, which is more than the GPU market.
SirFatty 9y ago
I've always found Alex St. John to be a blow-hard, having read his articles dating back to Boot (Maximum PC). No to mention the fact that he creates malware like WildTangent.
21 9y ago
> Intel keeps the PCIe bus slow and limits the number of IO lanes that an Intel CPU supports thus ensuring that GPU’s are always dependent on an Intel CPU to serve their workload I was looking into building a multi-GPU machine for ML and was very confused as to why the latest Intel CPU has less PCIe lanes than the previous one, it didn't made any sense to me. Those sneaky Intel bastards...
- dingo_bat 9y ago
  Let's hope AMD's threadripper really offers 64 lanes.
- deleted 9y ago
  [deleted]
- zokier 9y ago
  Intel products are heavily segmented. Did you do your comparison within the same segment?
- arnon 9y ago
  They want you to buy the expensive Xeons, or higher end i7s to get the proper amount of PCI lanes.
TheRealDunkirk 9y ago
What struck me most was the part about another giant corporation abusing its monopoly position to screw a competitor, at the of the expense of the customer.
- Steltek 9y ago
  An intelligent market will always fall victim to a tragedy of the commons scenario. Consumers will take the quick buck and let someone else worry about the collective future impact.
microcolonel 9y ago
His main idea is that x86 will decline in GPU host roles because Intel is deliberately limiting PCIe bus width, but then he shows AMD (another manufacturer of x86 chips) which has no such interest, delivering what he's asking for basically right now. He also seems to think that NVIDIA will be the sole winner, but if you think about it, AMD seems much better positioned. They manufacture chips which implement the most popular server ISA (x86) and on the same process they manufacture GPUs which are objectively more suitable for the tasks that are driving adoption(and starting to be, now that the software is catching up). So really the situation is not "NVIDIA is killing x86", it's "Discrete GPU vendors are selling more GPUs than fit on Intel's x86 implementation. Which will drive the adoption of systems with lots of bus width."
- jjm 9y ago
  It is, I think he ends with the possibility that with Arm being integrated into the GPU - a new possible world exists with the lack is a lack of Intel. With Linux and Windows Arm ready it could be very interesting the kind of possible system builds. For one having as many pci express lanes (or something completely different). Would be great to see a platform that is more friendly to extracting as much out of it as possible.
  - microcolonel 9y ago
    Wouldn't that be basically exactly the same as integrating the GPU into the CPU? This is something that AMD already does. The question is: does it have a considerable positive impact on compute workloads?. It doesn't seem like anyone is clambering for it, so I doubt it matters much. More peripheral bus bandwidth and connectors is probably all that anyone cares about.
deepnotderp 9y ago
One disagreement i have with this otherwise great piece is that the future may very well end up becoming MIMT/D. MIMD is a more flexible programming model than the lockstep SIMT of GPGPU and examples like the Rex Neo, Adapteva Epiphany and OpenPiton show that efficiency often supersedes GPUs.
- omikun 9y ago
  SIMT is inherently more power efficient than MIMD. Less control flow logic/flop. Even then, it makes sense to devote dedicated logic for specific algorithms. Even NVIDIA GPUs (Volta) are going to have special matrix multiply hardware (tensor cores) to increase power efficiency and performance. The future lies not in flexible programming model but dedicated hardware/IP. Look at the crypto block, ISP, h264/265 encoding/decoding, and now tensor cores. It's mentioned in what seems like every architecture paper in the last ten years, but dark silicon is driving the need to differentiate compute into smaller blocks. We can pack more and more transistors into a chip, but we can only power a smaller and smaller section of it at any given time. It only makes sense that we make whatever that can be powered on be as efficient as possible.
  - deepnotderp 9y ago
    In theory yes, but clean restarts like the Adapteva Epiphany and the Rex Neo can get better efficiency than GPUs because they don't suffer from legacy issues while still running legacy OpenCL code. As for the matrix multiplier ASICs like the tpu and Volta, I consider them to be incredibly uncreative and an insult of sorts to computer architecture to call that a "deep learning processor". What happens when tomorrow SPNs or graph ConvNets dominate? A proper application specific processor will be able to adapt and still maintain efficiency. Obviously i have some bias and hubris here, but our simulations show consistently superior efficiency to the tpu while running the same workloads while still retaining the ability to adapt to other computational graphs that TensorFlow may choose to run.
lrenaud 9y ago
It's a fun article, but it actually drives me to a question: is it possible (by which I mean remotely feasible) to design a user experience from the ground up around a massively parallel system? Let's say you had some low speed hardware to support a beefy GPU in a case, could a manageable operating system be built that wouldn't spend all of it's time in IO-wait? I like the idea and challenge in designing this way, but I don't see a way forward that doesn't just fall back on the architecture of core computational unit with specialized auxiliary units quite rapidly.
anonu 9y ago
Even with NVDA's recent rally, INTC is still very close to double NVDA's size ($168bn vs $88bn market cap). The point being, the author's statement that the x86 party is over seems a bit far off at the moment...
arnon 9y ago
One of the things Nvidia excels at, and has done very well in is supplying the right ecosystem for writing performant GPU code. This really pushed the adoption of GPUs forward. Writing good, fast, high performance code for many x86 nodes is still quite difficult. Nvidia's CUDA stack including Thrust, CUB, cuBLAS and other libraries really made it easy to write parallel code, without thinking too much about the lower level operations.
bazizbaziz 9y ago
> "The growing body of Big-data, HPC, and especially machine learning applications don’t need Windows and don’t perform on X86. So 2017 is the year Nvidia slips its leash and breaks free to become a genuinely viable competitive alternative to x86 based enterprise computing in valuable new markets that are unsuited to x86 based solutions." Google's TPU paper [0] showed the CPUs were relatively competitive in the machine learning space (within 2x of a K80). It's not true that x86 doesn't perform on these workloads. The existence of the TPU itself threatens Nvidia's dominance in the ML processor space. Google built an ASIC in a short time period that more than rivals a GPU on these tasks. The TPU performance improvements (section 7) make it look very straightforward to get even better performance with a few more years of development effort. With developers moving to higher level libraries, migration between GPU/CPU/TPU becomes painless, so they'll just go with whatever has the lowest TCO. (Google hosted TPUs?) Aside from machine learning tasks, the author seems to be advocating for the cpu/gpu combinations that AMD is already selling to game console manufacturers. Granted, Nvidia has a piece of this via the Switch. If Microsoft/Qualcomm goes full-on with their ARM-based x86 emulation, then perhaps a future ARM-based Xbox is in the cards driven by an Nvidia chip? /speculation [0] https://arxiv.org/abs/1704.04760 https://arxiv.org/abs/1704.04760
rcdmd 9y ago
The article's thesis, "Well it appears that the GPU era of computing is finally here! Intel is in deep trouble," has an implicit assumption that Intel's future mostly depends on processing power. Is that really the case?
combatentropy 9y ago
I had wondered whether Microsoft and Intel were more a fraternity or an uneasy alliance. This article gives hard details about how they have always been forging sabers to cut each other's throats whenever the time was ripe. It also woke me up to Intel's hardware throttling, starving the graphics unit of enough busses. I had been in awe of PCIe's bandwidth, but now I know it could be so much more. It reminds me that if you focus on microspecs doubling every few years, you think there's breathtaking progress. But if you take a step back at the overall result, computing is moving much more slowly. For example, it seems like the average laptop has always had 6-9 hours of battery life.
rsp1984 9y ago
Because this is the year that the first generation of self-hosting GPU’s are widely available on the market Really? Then what's inside of 99% of smartphones and tablets today already? It's not like ARM cores + beefy GPUs is a brand new concept. In fact even the first Raspberry Pi featured such a combination. Intel plays no role in the smartphones and tablets business already and anyone who doesn't care about x86 compatibility has been free to use whatever non-Intel accelerator in the datacenter / HPC space for years now. Not quite sure what the dramatic thing is that the author is implying has changed.

213 points by douche 9y ago

Cached: http://webcache.googleusercontent.com/search?q=cache:JDuUy_X4iUcJ:www.alexstjohn.com/WP/2017/06/10/the-age-of-nvidia/ http://webcache.googleusercontent.com/search?q=cache:JDuUy_X...

0x4f3759df 9y ago

The youtube of "NVidia GTC 2017 conference" is worth a look.

visarga 9y ago
For a moment they made me feel like it was 2007 again and I was watching an Apple keynote. I didn't expect to have that feeling again from any company.
- 0x4f3759df 9y ago
  Elon Musk was at the 2015 conf, so you might like that too.

ethbro 9y ago

In the DirectX comments, I'm struck by how interesting the strategic development of Intel and Microsoft is. I'd never thought about it before, but there aren't many industries that could instantly (2-4 years) be killed solely by technical development. If Intel had developed an OS-independent abstraction layer for devices, if Microsoft had pushed harder on ISA-independent programs, etc, how would today look?

Symmetry 9y ago
Oh, Intel was certainly a contributor to Linux and Microsoft flirted with having Windows run on other hardware. But backwards compatibility prevented either from commoditizing their complement on the desktop.
- AceJohnny2 9y ago
  Talk about co-dependent relationships...
- tachyonbeam 9y ago
  FYI, Microsoft now has an emulator to run x86 programs on ARM. This is for future laptops/tablet products.
mtgx 9y ago
There is a reason this partnership/cartel has been called "Wintel." Both knew it's in their best interest not to screw each other over by adopting other platforms more aggressively. I think they've been increasingly more dissatisfied with that arrangement since Intel was trying to get into iPads/iPhones, and then Microsoft built Windows RT and Windows Phone for ARM-only. Similarly, Google has been pushing Intel chips in Chromebooks (even though Chrome OS is architecture-agnostic, as you were hoping Microsoft to do with Windows), because they get those sweet discounts and early-access to Intel server chips. Plus, I think Intel is still on Google's board.
- StillBored 9y ago
  Well if google is pushing "intel" in chrombooks, it still doesn't explain why the intel chrombooks tend to have far more ram/disk space/etc than the ARM ones. Side by side, the ARM chrombooks seem like phones with big screens while the intel devices appear to be inexpensive ultrabooks.

Apreche 9y ago

Knowing this, what should a developer learn if they want to get a job in the brave new non-wintel world? ARM? CUDA?

zanny 9y ago
Knowing the architecture of ARM is a perk, but the software is going to be all CUDA / OpenCL / Vulkan Compute - the CPU side is just going to be a shim layer to get kernels to execute on that graphics card and won't take much development and will hopefully just be the same open standard everyone uses.
- visarga 9y ago
  Yep. Same programming. Details hidden by compiler.
  - sqeaky 9y ago
    I agree with you, except when they aren't hidden... For some large percentage of the problems we fix I agree with you, but every once in a while you need to peel back a layer or two of abstraction and ask "Why did it do that?" Most abstractions leak, despite whatever heavy lifting they do for us. Having familiarity with those underlying implementations can make that trouble-shooting much easier. So I guess it is up to each developer to understand how likely they are to peel back different abstractions. I suspect a JS dev will rarely want to look at the assembly and C/C++ might want to every week.
golergka 9y ago
Most of developers are not working with code that requires top-notch computer power; I don't think that Nvidia/Intel rivalry will affect the market for people who know JS, CSS and HTML to a significant degree. But generally speaking, learning Caffe, Tensorflow and math in general sounds like a much more important investment than low-level libraries for specific hardware.
- pm90 9y ago

Pigo 9y ago

All of this did give me some insight into why I see ARM-based Windows servers in Azure.

flipgimble 9y ago

" 2017, the year GPU’s finally begin to permanently displace the venerated x86 based CPU" he never specifies what he means by displace exactly, and follows up with vacuous statements like "x86 party ends in 2017". The entire thing sounds like trying to manufacture a momentous event out of a gradual shift of market profits and priorities over time. Best you could hope for is to state that Intel has failed to take a significant market in high end throughput computing (ie. desktop and HPC GPUs), but its doing quite well with integrated GPUs. Don't forget this is the same visionary that wrote the game industry playbook on how to hire and exploit developers: https://www.kotaku.com.au/2016/04/alex-st-johns-ideas-about-game-development-are-terrifying/ https://www.kotaku.com.au/2016/04/alex-st-johns-ideas-about-...

ballenf 9y ago
Even gradual events have recognizable tipping points or points of no return. I think that is what he is calling 2017 for the gradual transition from CPU dominance to the future of GPU/FPGA dominance. Or just CPU irrelevance. Don't know how that article you link has any relevance to the facts and analysis he presents here.
gambiting 9y ago
It's the same as calling 20XX "year of linux" for the n-th time. Sure there is a gradual shift and things are getting better, but X86 is not dead, just like windows is still doing fine.
- II2II 9y ago
  If you are trying to compare it to the year of Linux, it would be wise to look at how the market changed in general rather than how the market changed for Microsoft. Linux did not decimate Windows, but it has certainly hurt other platforms and it has likely hurt Microsoft's reach. Likewise, I doubt that GPGPU applications will decimate Intel. It will likely hurt Intel's reach. This is particularly true if AMD, ARM, and IBM are facilitating the use of to GPU as a high performance coprocessor.
lnanek2 9y ago

Hydraulix989 9y ago

Uh... GPUs and CPUs fill very different niches. GPUs aren't going to _replace_ CPUs any time soon because GPUs aren't capable nor designed for general-purpose processing. It's a fundamentally REALLY HARD problem to use GPUs to speed up any arbitrary computations (solve it and you'll be a shoo-in for the Turing Award). Businesses aren't going to be "moving" to GPUs. The "age of NVIDIA" is primarily predicated on its role in accelerating training the machine learning algorithms hyped up as "deep learning." And AMD still very much has an uphill battle on both the CPU and the GPU fronts. But yes, the excitement around Intel that used to be there is now gone. You can probably blame the "death of the PC" -- we teach kids coding in the Bay Area, and every single kid has an iPad tablet, but not a laptop. Sure professional engineers like us still have x86 laptops, but the average person does not.

visarga 9y ago
> Uh... GPUs and CPUs fill very different niches. I think they actually mean CPU = Intel, GPU = NVIDIA because strictly speaking Intel is also a GPU manufacturer, albeit low end GPUs. NVIDIA will play a major role in AI and simulation, that is clear.
ethikal 9y ago
It's not just about accelerating ML, specifically deep learning. There are many other enterprise technologies that can benefit from GPUs. One example: OLAP-focused databases (such as MapD - https://www.mapd.com/ https://www.mapd.com/). For some benchmarks, check out this blog: http://tech.marksblogg.com/benchmarks.html http://tech.marksblogg.com/benchmarks.html. The DL "training" use-case is well-known at this point, but there are many others which are emerging.
- deleted 9y ago
  [deleted]
- jhj 9y ago
  A GPU database isn't that useful, because the arithmetic intensity (ops/byte) is relatively low. Cross-sectional memory bandwidth is what really matters; you can get similar effects with a cluster of CPU machines provisioned appropriately, with a shard or a replica of the database on each CPU machine. I say this as someone who has written a GPU in-memory database of sorts that is used at Facebook (Faiss), but what is interesting if you can tie that to something that has higher arithmetic intensity before or after the database lookup on the GPU. GPUs are only really being used for machine learning due to the sequential dependence of SGD and the relatively high arithmetic intensity (flops/byte) of convolutions or certain GEMMs. The faster you can take a gradient descent step means the faster wall clock time to converge, and you would lose by limiting memory reuse (for conv/GEMM) or on communication overhead or latency if you attempt to split a single computation between multiple nodes. The Volta "tensor cores" (fp16 units) make the GPU less arithmetic bound for operations such as convolution that require a GEMM-like operation, but the fact that the memory bandwidth did not increase by a similar factor means that Volta is fairly unbalanced. The point about Intel not increasing their headline performance by as much as GPUs is also misleading. Intel CPUs are very good at branchy codes and are latency optimized, not throughput optimized (as far as a general purpose computer can be). Not everything we want to do, even in deep learning, will necessarily run well on a throughput-optimized machine.

digitalzombie 9y ago

I always thought Microsoft keeping dominance via video game and directx was their main drive to keep Linux out. Didn't realize directx was their move to fight against Intel. The unintentional effect of this was GPU manufacturer florish, which is something I didn't know either. The two big monopoly fighting against each other is super interesting. Also highlight the fact that we do need some diversity and competition in these segment. Rooting for AMD on the GPU front to give Cuda a good challenge and also on the Intel front for their CPU.

koffiezet 9y ago
> and directx was their main drive to keep Linux out OS/2 Warp could have been a target, but Linux was nowhere near a threat to Microsoft when they came up with DirectX...

frozenport 9y ago

This post doesn't mention the strategic dilemma Intel has with GPUs. GPUs are fast because they have fast local memory, the core count is secondary, and some Intel chips have 22x16 vector processing elements clocked 2x higher than NVIDIA ( around 3k CUDA core equivalent ). Why doesn't Intel make a fast local memory GPU style device. Well, they did (Xeon Phi) but if they give it a reasonable price point it would cannibalize their existing x86 market, which is more than the GPU market.

SirFatty 9y ago

I've always found Alex St. John to be a blow-hard, having read his articles dating back to Boot (Maximum PC). No to mention the fact that he creates malware like WildTangent.

21 9y ago

> Intel keeps the PCIe bus slow and limits the number of IO lanes that an Intel CPU supports thus ensuring that GPU’s are always dependent on an Intel CPU to serve their workload I was looking into building a multi-GPU machine for ML and was very confused as to why the latest Intel CPU has less PCIe lanes than the previous one, it didn't made any sense to me. Those sneaky Intel bastards...

dingo_bat 9y ago
Let's hope AMD's threadripper really offers 64 lanes.
deleted 9y ago
[deleted]
zokier 9y ago
Intel products are heavily segmented. Did you do your comparison within the same segment?
arnon 9y ago
They want you to buy the expensive Xeons, or higher end i7s to get the proper amount of PCI lanes.

TheRealDunkirk 9y ago

What struck me most was the part about another giant corporation abusing its monopoly position to screw a competitor, at the of the expense of the customer.

Steltek 9y ago
An intelligent market will always fall victim to a tragedy of the commons scenario. Consumers will take the quick buck and let someone else worry about the collective future impact.

microcolonel 9y ago

His main idea is that x86 will decline in GPU host roles because Intel is deliberately limiting PCIe bus width, but then he shows AMD (another manufacturer of x86 chips) which has no such interest, delivering what he's asking for basically right now. He also seems to think that NVIDIA will be the sole winner, but if you think about it, AMD seems much better positioned. They manufacture chips which implement the most popular server ISA (x86) and on the same process they manufacture GPUs which are objectively more suitable for the tasks that are driving adoption(and starting to be, now that the software is catching up). So really the situation is not "NVIDIA is killing x86", it's "Discrete GPU vendors are selling more GPUs than fit on Intel's x86 implementation. Which will drive the adoption of systems with lots of bus width."

jjm 9y ago
It is, I think he ends with the possibility that with Arm being integrated into the GPU - a new possible world exists with the lack is a lack of Intel. With Linux and Windows Arm ready it could be very interesting the kind of possible system builds. For one having as many pci express lanes (or something completely different). Would be great to see a platform that is more friendly to extracting as much out of it as possible.
- microcolonel 9y ago
  Wouldn't that be basically exactly the same as integrating the GPU into the CPU? This is something that AMD already does. The question is: does it have a considerable positive impact on compute workloads?. It doesn't seem like anyone is clambering for it, so I doubt it matters much. More peripheral bus bandwidth and connectors is probably all that anyone cares about.

deepnotderp 9y ago

One disagreement i have with this otherwise great piece is that the future may very well end up becoming MIMT/D. MIMD is a more flexible programming model than the lockstep SIMT of GPGPU and examples like the Rex Neo, Adapteva Epiphany and OpenPiton show that efficiency often supersedes GPUs.

omikun 9y ago
SIMT is inherently more power efficient than MIMD. Less control flow logic/flop. Even then, it makes sense to devote dedicated logic for specific algorithms. Even NVIDIA GPUs (Volta) are going to have special matrix multiply hardware (tensor cores) to increase power efficiency and performance. The future lies not in flexible programming model but dedicated hardware/IP. Look at the crypto block, ISP, h264/265 encoding/decoding, and now tensor cores. It's mentioned in what seems like every architecture paper in the last ten years, but dark silicon is driving the need to differentiate compute into smaller blocks. We can pack more and more transistors into a chip, but we can only power a smaller and smaller section of it at any given time. It only makes sense that we make whatever that can be powered on be as efficient as possible.
- deepnotderp 9y ago
  In theory yes, but clean restarts like the Adapteva Epiphany and the Rex Neo can get better efficiency than GPUs because they don't suffer from legacy issues while still running legacy OpenCL code. As for the matrix multiplier ASICs like the tpu and Volta, I consider them to be incredibly uncreative and an insult of sorts to computer architecture to call that a "deep learning processor". What happens when tomorrow SPNs or graph ConvNets dominate? A proper application specific processor will be able to adapt and still maintain efficiency. Obviously i have some bias and hubris here, but our simulations show consistently superior efficiency to the tpu while running the same workloads while still retaining the ability to adapt to other computational graphs that TensorFlow may choose to run.

lrenaud 9y ago

It's a fun article, but it actually drives me to a question: is it possible (by which I mean remotely feasible) to design a user experience from the ground up around a massively parallel system? Let's say you had some low speed hardware to support a beefy GPU in a case, could a manageable operating system be built that wouldn't spend all of it's time in IO-wait? I like the idea and challenge in designing this way, but I don't see a way forward that doesn't just fall back on the architecture of core computational unit with specialized auxiliary units quite rapidly.

anonu 9y ago

Even with NVDA's recent rally, INTC is still very close to double NVDA's size ($168bn vs $88bn market cap). The point being, the author's statement that the x86 party is over seems a bit far off at the moment...

arnon 9y ago

One of the things Nvidia excels at, and has done very well in is supplying the right ecosystem for writing performant GPU code. This really pushed the adoption of GPUs forward. Writing good, fast, high performance code for many x86 nodes is still quite difficult. Nvidia's CUDA stack including Thrust, CUB, cuBLAS and other libraries really made it easy to write parallel code, without thinking too much about the lower level operations.

bazizbaziz 9y ago

> "The growing body of Big-data, HPC, and especially machine learning applications don’t need Windows and don’t perform on X86. So 2017 is the year Nvidia slips its leash and breaks free to become a genuinely viable competitive alternative to x86 based enterprise computing in valuable new markets that are unsuited to x86 based solutions." Google's TPU paper [0] showed the CPUs were relatively competitive in the machine learning space (within 2x of a K80). It's not true that x86 doesn't perform on these workloads. The existence of the TPU itself threatens Nvidia's dominance in the ML processor space. Google built an ASIC in a short time period that more than rivals a GPU on these tasks. The TPU performance improvements (section 7) make it look very straightforward to get even better performance with a few more years of development effort. With developers moving to higher level libraries, migration between GPU/CPU/TPU becomes painless, so they'll just go with whatever has the lowest TCO. (Google hosted TPUs?) Aside from machine learning tasks, the author seems to be advocating for the cpu/gpu combinations that AMD is already selling to game console manufacturers. Granted, Nvidia has a piece of this via the Switch. If Microsoft/Qualcomm goes full-on with their ARM-based x86 emulation, then perhaps a future ARM-based Xbox is in the cards driven by an Nvidia chip? /speculation [0] https://arxiv.org/abs/1704.04760 https://arxiv.org/abs/1704.04760

rcdmd 9y ago

The article's thesis, "Well it appears that the GPU era of computing is finally here! Intel is in deep trouble," has an implicit assumption that Intel's future mostly depends on processing power. Is that really the case?

combatentropy 9y ago

I had wondered whether Microsoft and Intel were more a fraternity or an uneasy alliance. This article gives hard details about how they have always been forging sabers to cut each other's throats whenever the time was ripe. It also woke me up to Intel's hardware throttling, starving the graphics unit of enough busses. I had been in awe of PCIe's bandwidth, but now I know it could be so much more. It reminds me that if you focus on microspecs doubling every few years, you think there's breathtaking progress. But if you take a step back at the overall result, computing is moving much more slowly. For example, it seems like the average laptop has always had 6-9 hours of battery life.

rsp1984 9y ago

Because this is the year that the first generation of self-hosting GPU’s are widely available on the market Really? Then what's inside of 99% of smartphones and tablets today already? It's not like ARM cores + beefy GPUs is a brand new concept. In fact even the first Raspberry Pi featured such a combination. Intel plays no role in the smartphones and tablets business already and anyone who doesn't care about x86 compatibility has been free to use whatever non-Intel accelerator in the datacenter / HPC space for years now. Not quite sure what the dramatic thing is that the author is implying has changed.