15 ms·
The link is incredibly light on actual content but this seems to be good news for AI enthusiasts as perhaps now we'll get a reasonable competitor to CUDA/CUDNN
by diab0lic 9y ago
The link is incredibly light on actual content but this seems to be good news for AI enthusiasts as perhaps now we'll get a reasonable competitor to CUDA/CUDNN and their associated hardware for running GPU accelerated machine learning. Intel seems to be taking the ML/AI space seriously and this move seems very likely to be related. Yes I'm aware of OpenCL as I am also aware of it's level of support with libraries such as PyTorch, Tensorflow, Theano -- it isn't the first class citizen that CUDA is. While those libraries aren't perfect they offer the experience of writing the experiment on your laptop without a GPU, validating, then running the full experiment on larger hardware.
In my ideal world competition from intel would force NVidia to play nice with OpenCL or something similar, and encourage competition in the hardware space instead of driver support space. Unfortunately the worst-case looks something more like CUDA, OpenCL and a third option from Intel with OpenCL like adoption. :(
- moonbug22 9y agoOnly armchair spectators are still talking about OpenCL. It's as dead as disco.
- diab0lic 9y agoThat is more or less what I was getting at, while trying to remain polite on the subject. The truth is that none of the deep learning backends have real support for it and OSX supports an outdated version (IIRC). It has seen very little adoption outside of a handful of niche use cases.
- abiox 9y agoare there insurmountable technical constraints there, or could it be resuscitated?
- diab0lic 9y agoIt could be resuscitated but it is unlikely without the backing of a major player.
- Athas 9y agoOpenCL is a very sound design on a technical level. Unfortunately, it's mostly designed to be a good target for code generation/libraries, and is awkward to use directly. The problems are that everything is very explicit and spelled out, and the kernel language itself is based on C, not C++, and segmented from the rest of the code base. In practice, you use OpenCL API calls to submit strings containing code to the OpenCL backend, which then compiles it and hands you back an opaque compiled program that you can execute. This makes it hard to write modular and maintainable code with OpenCL directly. The issues go away if you use a good OpenCL frontend. PyOpenCL for Python goes a long way towards this, and is not really any more awkward than the corresponding PyCUDA, and higher-level languages that generate OpenCL code, like Lift[0] or Futhark[1] (tooting my own horn here), remove the awkwardness completely. [0]: http://www.lift-project.org/ http://www.lift-project.org/ [1]: https://futhark-lang.org https://futhark-lang.org
- pjmlp 9y agoI think that has been the major failure from Khronos, being stuck with C mentality for their APIs. OpenGL ES only took off thanks to gaming on the iPhone, and now is deprecated on Apple platforms. Vulkan still lives in a C world, and the semi-official C++ bindings only exist thanks NVidia. OpenCL waited too long to support C++, Fortran and providing an infrastructure for compiler writers to add GPU support to their own languages. And two years later the majority of drivers are not there yet.
- roel_v 9y agoKhronos' problem is that they want to do everything, or not do it at all. Sycl would be 'real' c++ support, way beyond the 'oo wrapper around c api' that already exists in opencl headers. Nvidia is more 'good enough? Ship it' which has shown itself many times in the past to be a dominant strategy. (As much as I hate to admit that).
- mellinoe 9y agoIt seems like a C API is a much better choice if you want your library to be callable from many different languages. What is a reasonable alternative without giving that up?
- roel_v 9y agoIt's purely political. If nvidia would have a team of 3-5 ppl working op their opencl drivers/tooling, opencl would be as good as cuda. Nvidia would be stupid to do that, of course. Hence the Khronos play to fold opencl into vulkan. We'll see how that plays out.
- joefourier 9y agoI still use OpenCL on the daily, it allows me to program for both NVidia and AMD GPUS simultaneously with the minimum amount of pain. And you can still use it for mobile GPUs and specialized chips like the Myriad. Do you have another open, cross-platform, widely compatible GPU programming framework to recommend?
- nicwilson 9y agoThere is DCompute[1] which is able to generate SPIR-V and PTX simultaneously from the same D source. See my comment to the parent. http://github.com/libmir/dcompute http://github.com/libmir/dcompute
- viewtransform 9y agoHIP allows developers to convert CUDA code to portable C++. The same source code can be compiled to run on NVIDIA or AMD GPUs https://github.com/ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP
- pjmlp 9y agoWhat mobile GPUs? On iOS it is kind of deprecated and the way forward is Metal Compute. On Android it never happened, Google created their own Renderscript dialect instead.
- gcp 9y agoSame here. What else can you do to ship GPU acceleration to AMD and NVIDIA people? The alternatives recommended here aren't even serious IMHO. I'd rather switch to CUDA and wait till Intel/AMD sort out a REAL compatibility layer than deal with those. Unless I'm mistaken, HIP still requires a separate compile for either platform and what runtime do they expect end users to have exactly?! At least CUDA and OpenCL are integrated in the vendor drivers. Vulkan compute with SPIR-V seems to be the only real solution, but even that is still very early. Sill waiting for proper OpenCL 2.0 support in NVIDIA drivers :P
- freeone3000 9y agoYou simply don't ship. Enterprise deep learning doesn't ship their training code - large models are trained on purpose-designed, dedicated hardware. Hardware compatability doesn't matter, software does. Even that's flexible if it's significantly faster. (The models can be executed on low-powered, commodity CPUs. No need for any GPU there.)
- nicwilson 9y agoThere is no reason that you can't create both OpenCL (i.e. SPIR-V) and PTX from the same source. Thats what I do for [DCompute](https://github.com/libmir/dcompute https://github.com/libmir/dcompute). Agnosticism at the library level/CPU side is next on my list.
- roel_v 9y agoThe problem isn't getting it to run, the problem is getting similar performance. You need self-tweaking algos to get some level of performance portability using just opencl, let alone across two apis.
- nicwilson 9y agoIndeed, there is a very large amount of parameterisation you can do in D with ease. You can, if you have prior knowledge of the hardware that it will run on, create different versions of the kernels to exploit that (e.g. with template kernels, which we support directly unlike OpenCL C++). Do you mean adaptive algorithms or dynamic recompilation? And yes I do expect that cross API will be difficult, for both running and getting good perf. But it is not just the room at the high performance end of the spectrum that is important, but also the lower end that is stifled by the barrier to entry that would benefit from the extra compute.
- roel_v 9y agoI meant dynamic (re)compilation yes - like adapting the stride in your kernel when you vary work group size for OpenCL kernels. I mean, I have something like that (a primitive self-tuning pre-run calibration step) and I only vary work group-, local- and vector size; and that's already a massive pain in the ass. It's ok on toy kernels but once you move beyond that, it just seeps into all your kernel code, making it almost into a meta-language. And then I'm not even talking about differences like using image types vs arrays for data that is 2d by nature. I don't see how I can generalize that; I just write various versions of my kernels. Which doesn't scale very much, to put it mildly. My point is - was drawn to OpenCL for its 'portability' claim, and yes kernels will 'run' on various hardware, but with massive differences in speed. what good does that portability do me? My workloads (scientific computing, branch heavy) are different from the typical ML half- or single precision MulDiv()/linear algebra applications, so all the hand-tuned CUDA libraries aren't even my concern. The elephant in the room is that performance doesn't depend on this API or that; it's in how you tune your algorithm to the actual hardware you're running on. Which is the direct opposite of portability. To come back to the post I was replying to - yes it's 'trivial' (I mean, a lot of work, but technically not hard, not to belittle your work) to compile almost any statically typed code into either SPIR-V or PTX or any other future format for that matter. But that doesn't mean that it will work 'well' (not even 'optimal') on other hardware. In the real world, you're almost always better off just spending a few hundred to get the same GPU as whoever wrote the kernels tested them on (or if you're running your kernels on existing clusters, to focus on optimizing them for what you know you'll be running them on). Oh and all of that is just considering GPU's. I mean, when you read an OpenCL book they make it seem (in the first few chapters) that you don't even have to think about whether you'll be running on a CPU or a GPU. And then you accidentally use USE_HOST_PTR instead of COPY_HOST_PTR (or the other way around) in the wrong place, and all of a sudden your code is 10x slower than the sequential version of your algo even. What I'm saying is - I no longer believe in easy to use abstractions for these purposes. If you want speed, you code to the metal, and/or you tweak your abstractions to your specific use case. Yes this is a lot of work. And if you don't need speed, you just throw in a few std::thread's here and there and call it a day. But that's just my conclusion for my use cases.
- chadnickbok 9y agoDisco will never die.
- gcp 9y agoI'll continue using it as long as the users are still split between NVIDIA and AMD cards. If AMD GPUs die out, CUDA it is. Or maybe give up and use a wrapper library for whatever 10 alternatives-only-supported-by-one-marginal-vendor there are. (like Apple)
- krona 9y agoMy view is CUDA has already won, and everyone else needs to get over it. Even clang supports PTX now, which is a reasonably device agnostic representation, albeit controlled mostly by Nvidia. Perhaps intel will introduce their own extensions to this ISA. Even if my precompiled CUDA application could run on Intel GPUs at 50% of the throughput, I'd be happy if I could later tweak and recompile it to get the full benefits from their hardware.
- deleted 9y ago[deleted]
- sounds 9y agoYes, Intel knows how to play when they own the instruction set (x86, sse1,2,3,4 and amd64). No, amd64 is just cross-licensing between two and only two companies. AMD will never catch Intel. Ergo, Intel owns amd64 too. It shouldn't be a surprise then that along comes NVidia with their own instruction set, PTX, and Intel's desire to own the instruction set will be their undoing.
- wmf 9y agoIntel has failed in most of the markets that they have taken seriously, especially software, so I don't see why this time would be different.
- giobox 9y agoWhile I hate to put down an effort before it even gets off the ground, there is some truth to this. Their recent failed push into 'wearables' was a great example. They bought up a number of small(ish) but interesting players (Basis, Recon Jet etc) and squandered them completely. Their complete missing the boat on smartphones, save perhaps a small amount of modem chip business the past couple of years, is especially damning. With GPUs there's the whole failed Larrabee thing as well. If Intel ever acquire a company I care about I will be extremely concerned.
- tanilama 9y agoThey have to if they want to stay relevant. They already lost Mobile, if they failed to catch the AI/ML bus it will be their last.
- p1esk 9y agoOne way Intel can compete with CUDA is if they allow us to write plain Numpy code, and magically compile it for their GPUs with similar performance to what we get from Nvidia when writing in CUDA. An alternative would be for them to (again magically) modify all major DL frameworks to support their GPUs. I don't even know which option is more realistic.
- AnkhMorporkian 9y agoI think #1 is the most realistic. Intel does some very significant work with Numpy as it is with MKL. #2 seems like dark voodoo magic that'd be neat, but pretty much impossible.
- nl 9y agoOne way Intel can compete with CUDA is to show a benchmark where they outperform NVidia in training speed. Not speed per watt, not inference, not oddly priced speed per dollar comparisons. If the speed is there the world will write the code.
- p1esk 9y agoSure, they can "show a benchmark", however, to be more than 30% faster in reality (not just on a marketing slide) than the next generation Nvidia chip is probably just as unlikely as the first two options. I'd love to get surprised though.
- Athas 9y agoI just hope it won't be all too focused on deep learning. That's boring. We need massively parallel machines that are more general than that. Modern NVIDIA-style GPUs are actually doing quite well, although they've started adding stuff that is only marginally useful outside of straightline matrix operations.